-- Logs begin at Mon 2022-08-01 15:28:52 UTC, end at Mon 2022-08-01 17:23:31 UTC. -- Aug 01 16:35:12.427633 dv-cinder-849890-2-iscsi systemd[1]: Starting Devstack devstack@keystone.service... Aug 01 16:35:12.437410 dv-cinder-849890-2-iscsi devstack@keystone.service[87046]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini Aug 01 16:35:12.439440 dv-cinder-849890-2-iscsi devstack@keystone.service[87046]: *** Starting uWSGI 2.0.18-debian (64bit) on [Mon Aug 1 16:35:12 2022] *** Aug 01 16:35:12.439440 dv-cinder-849890-2-iscsi devstack@keystone.service[87046]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Aug 01 16:35:12.439440 dv-cinder-849890-2-iscsi devstack@keystone.service[87046]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Aug 01 16:35:12.439440 dv-cinder-849890-2-iscsi devstack@keystone.service[87046]: nodename: dv-cinder-849890-2-iscsi Aug 01 16:35:12.439440 dv-cinder-849890-2-iscsi devstack@keystone.service[87046]: machine: x86_64 Aug 01 16:35:12.439440 dv-cinder-849890-2-iscsi devstack@keystone.service[87046]: clock source: unix Aug 01 16:35:12.439440 dv-cinder-849890-2-iscsi devstack@keystone.service[87046]: pcre jit disabled Aug 01 16:35:12.439440 dv-cinder-849890-2-iscsi devstack@keystone.service[87046]: detected number of CPU cores: 4 Aug 01 16:35:12.439440 dv-cinder-849890-2-iscsi devstack@keystone.service[87046]: current working directory: / Aug 01 16:35:12.439440 dv-cinder-849890-2-iscsi devstack@keystone.service[87046]: detected binary path: /usr/bin/uwsgi-core Aug 01 16:35:12.439440 dv-cinder-849890-2-iscsi devstack@keystone.service[87046]: your processes number limit is 47680 Aug 01 16:35:12.440215 dv-cinder-849890-2-iscsi devstack@keystone.service[87046]: your memory page size is 4096 bytes Aug 01 16:35:12.440215 dv-cinder-849890-2-iscsi devstack@keystone.service[87046]: detected max file descriptor number: 2048 Aug 01 16:35:12.440215 dv-cinder-849890-2-iscsi devstack@keystone.service[87046]: lock engine: pthread robust mutexes Aug 01 16:35:12.440215 dv-cinder-849890-2-iscsi devstack@keystone.service[87046]: thunder lock: enabled Aug 01 16:35:12.440215 dv-cinder-849890-2-iscsi devstack@keystone.service[87046]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 Aug 01 16:35:12.440215 dv-cinder-849890-2-iscsi devstack@keystone.service[87046]: Python version: 3.8.10 (default, Jun 22 2022, 20:18:18) [GCC 9.4.0] Aug 01 16:35:12.480401 dv-cinder-849890-2-iscsi devstack@keystone.service[87046]: Python main interpreter initialized at 0x55f81eee69c0 Aug 01 16:35:12.480401 dv-cinder-849890-2-iscsi devstack@keystone.service[87046]: python threads support enabled Aug 01 16:35:12.480401 dv-cinder-849890-2-iscsi devstack@keystone.service[87046]: your server socket listen backlog is limited to 100 connections Aug 01 16:35:12.480401 dv-cinder-849890-2-iscsi devstack@keystone.service[87046]: your mercy for graceful operations on workers is 90 seconds Aug 01 16:35:12.480743 dv-cinder-849890-2-iscsi devstack@keystone.service[87046]: mapped 403077 bytes (393 KB) for 2 cores Aug 01 16:35:12.480824 dv-cinder-849890-2-iscsi devstack@keystone.service[87046]: *** Operational MODE: preforking *** Aug 01 16:35:12.481082 dv-cinder-849890-2-iscsi devstack@keystone.service[87046]: *** uWSGI is running in multiple interpreter mode *** Aug 01 16:35:12.481082 dv-cinder-849890-2-iscsi devstack@keystone.service[87046]: spawned uWSGI master process (pid: 87046) Aug 01 16:35:12.481286 dv-cinder-849890-2-iscsi systemd[1]: Started Devstack devstack@keystone.service. Aug 01 16:35:12.483205 dv-cinder-849890-2-iscsi devstack@keystone.service[87046]: spawned uWSGI worker 1 (pid: 87057, cores: 1) Aug 01 16:35:12.483205 dv-cinder-849890-2-iscsi devstack@keystone.service[87046]: spawned uWSGI worker 2 (pid: 87058, cores: 1) Aug 01 16:35:12.483205 dv-cinder-849890-2-iscsi devstack@keystone.service[87046]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Aug 01 16:35:18.021608 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 01 16:35:18.021608 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: from cryptography.utils import int_from_bytes Aug 01 16:35:18.022470 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 01 16:35:18.022470 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: from cryptography.utils import int_from_bytes Aug 01 16:35:18.022887 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 01 16:35:18.022887 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: from cryptography.utils import int_from_bytes Aug 01 16:35:18.023008 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 01 16:35:18.023008 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: from cryptography.utils import int_from_bytes Aug 01 16:35:18.071324 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=87058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Aug 01 16:35:18.071518 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=87058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Aug 01 16:35:18.071709 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG uwsgi [-] command line args: [] {{(pid=87058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Aug 01 16:35:18.071893 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=87057) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Aug 01 16:35:18.072480 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=87057) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Aug 01 16:35:18.072593 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=87058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Aug 01 16:35:18.072730 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG uwsgi [-] command line args: [] {{(pid=87057) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Aug 01 16:35:18.072820 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG uwsgi [-] ================================================================================ {{(pid=87058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Aug 01 16:35:18.072998 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG uwsgi [-] admin_token = **** {{(pid=87058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 16:35:18.073107 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=87057) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Aug 01 16:35:18.073219 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG uwsgi [-] ================================================================================ {{(pid=87057) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Aug 01 16:35:18.073303 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG uwsgi [-] config_dir = [] {{(pid=87058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 16:35:18.073408 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=87058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 16:35:18.073494 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG uwsgi [-] admin_token = **** {{(pid=87057) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 16:35:18.073589 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG uwsgi [-] config_source = [] {{(pid=87058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 16:35:18.073716 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG uwsgi [-] config_dir = [] {{(pid=87057) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 16:35:18.073802 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG uwsgi [-] debug = True {{(pid=87058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 16:35:18.073905 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=87057) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 16:35:18.073991 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=87058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 16:35:18.074225 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG uwsgi [-] config_source = [] {{(pid=87057) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 16:35:18.074303 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG uwsgi [-] debug = True {{(pid=87057) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 16:35:18.074370 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=87058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 16:35:18.074508 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=87057) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 16:35:18.074611 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG uwsgi [-] insecure_debug = False {{(pid=87058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 16:35:18.074725 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=87057) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 16:35:18.074808 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=87058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 16:35:18.074876 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG uwsgi [-] insecure_debug = False {{(pid=87057) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 16:35:18.074985 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=87058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 16:35:18.075172 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=87057) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 16:35:18.075262 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG uwsgi [-] list_limit = None {{(pid=87058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 16:35:18.075367 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG uwsgi [-] log_config_append = None {{(pid=87058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 16:35:18.075453 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=87057) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 16:35:18.075516 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=87058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 16:35:18.075580 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG uwsgi [-] list_limit = None {{(pid=87057) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 16:35:18.075700 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG uwsgi [-] log_dir = None {{(pid=87058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 16:35:18.075781 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG uwsgi [-] log_config_append = None {{(pid=87057) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 16:35:18.075843 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG uwsgi [-] log_file = None {{(pid=87058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 16:35:18.075912 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=87057) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 16:35:18.076069 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=87058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 16:35:18.076151 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG uwsgi [-] log_dir = None {{(pid=87057) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 16:35:18.076322 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG uwsgi [-] log_file = None {{(pid=87057) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 16:35:18.076432 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=87058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 16:35:18.076572 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=87057) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 16:35:18.076662 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=87058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 16:35:18.076737 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=87057) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 16:35:18.076815 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=87058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 16:35:18.076947 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=87058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 16:35:18.077055 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=87057) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 16:35:18.077130 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=87058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 16:35:18.077212 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=87057) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 16:35:18.077334 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=87058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 16:35:18.077456 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=87057) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 16:35:18.077558 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=87057) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 16:35:18.077632 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=87058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 16:35:18.077772 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=87057) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 16:35:18.077866 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=87058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 16:35:18.078005 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=87058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 16:35:18.078112 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=87057) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 16:35:18.078261 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=87058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 16:35:18.078371 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=87057) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 16:35:18.078494 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=87058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 16:35:18.078594 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=87057) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 16:35:18.078726 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=87057) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 16:35:18.078808 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=87058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 16:35:18.078956 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=87057) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 16:35:18.079043 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG uwsgi [-] notification_format = cadf {{(pid=87058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 16:35:18.079202 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=87057) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 16:35:18.079289 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=87058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 16:35:18.079445 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG uwsgi [-] notification_format = cadf {{(pid=87057) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 16:35:18.079703 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.20/identity {{(pid=87058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 16:35:18.079797 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=87057) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 16:35:18.079872 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG uwsgi [-] publish_errors = False {{(pid=87058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 16:35:18.080096 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=87058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 16:35:18.080204 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.20/identity {{(pid=87057) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 16:35:18.080378 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=87058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 16:35:18.080496 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG uwsgi [-] publish_errors = False {{(pid=87057) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 16:35:18.080579 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=87058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 16:35:18.080652 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=87057) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 16:35:18.080783 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=87058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 16:35:18.080869 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=87057) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 16:35:18.080943 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=87058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 16:35:18.081077 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=87057) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 16:35:18.081160 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG uwsgi [-] standard_threads = False {{(pid=87058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 16:35:18.081234 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=87057) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 16:35:18.081359 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG uwsgi [-] strict_password_check = False {{(pid=87058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 16:35:18.081445 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=87057) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 16:35:18.081596 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=87058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 16:35:18.081675 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG uwsgi [-] standard_threads = False {{(pid=87057) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 16:35:18.081754 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG uwsgi [-] use_eventlog = False {{(pid=87058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 16:35:18.081879 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG uwsgi [-] strict_password_check = False {{(pid=87057) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 16:35:18.081966 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG uwsgi [-] use_journal = False {{(pid=87058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 16:35:18.082114 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG uwsgi [-] use_json = False {{(pid=87058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 16:35:18.082206 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=87057) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 16:35:18.082282 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG uwsgi [-] use_stderr = False {{(pid=87058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 16:35:18.082354 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG uwsgi [-] use_eventlog = False {{(pid=87057) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 16:35:18.082452 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG uwsgi [-] use_syslog = False {{(pid=87058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 16:35:18.082543 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG uwsgi [-] use_journal = False {{(pid=87057) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 16:35:18.082618 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG uwsgi [-] use_json = False {{(pid=87057) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 16:35:18.082689 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG uwsgi [-] watch_log_file = False {{(pid=87058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 16:35:18.082795 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG uwsgi [-] use_stderr = False {{(pid=87057) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 16:35:18.082917 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=87058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:35:18.082998 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG uwsgi [-] use_syslog = False {{(pid=87057) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 16:35:18.083147 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=87058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:35:18.083229 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG uwsgi [-] watch_log_file = False {{(pid=87057) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 16:35:18.083395 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=87058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:35:18.083476 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=87057) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:35:18.083630 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=87058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:35:18.083715 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=87057) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:35:18.083874 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=87058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:35:18.083954 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=87057) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:35:18.084108 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=87058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:35:18.084231 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=87057) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:35:18.084353 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=87058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:35:18.084439 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=87057) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:35:18.084584 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=87058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:35:18.084670 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=87057) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:35:18.084827 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=87057) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:35:18.084904 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=87058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:35:18.085062 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=87057) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:35:18.085142 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=87058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:35:18.085305 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=87058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:35:18.085384 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=87057) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:35:18.085537 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=87058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:35:18.085617 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=87057) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:35:18.085779 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=87058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:35:18.085866 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=87057) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:35:18.086010 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=87057) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:35:18.086150 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=87058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:35:18.086278 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=87057) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:35:18.086365 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=87058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:35:18.086514 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=87057) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:35:18.086610 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=87058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:35:18.086745 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=87058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:35:18.086824 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=87057) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:35:18.086992 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=87057) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:35:18.087073 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=87058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:35:18.087226 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=87057) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:35:18.087313 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=87058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:35:18.087513 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=87057) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:35:18.087616 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=87058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:35:18.087826 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=87058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:35:18.087935 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=87057) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:35:18.088082 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=87058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:35:18.088171 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=87057) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:35:18.088343 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=87058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:35:18.088431 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=87057) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:35:18.088600 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=87058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:35:18.088685 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=87057) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:35:18.088846 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=87058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:35:18.089003 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=87057) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:35:18.089080 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=87058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:35:18.089158 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=87057) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:35:18.089291 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=87058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:35:18.089377 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=87057) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:35:18.089504 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=87058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:35:18.089590 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=87057) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:35:18.089739 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=87058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:35:18.089818 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=87057) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:35:18.089984 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=87058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:35:18.090109 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=87057) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:35:18.090254 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=87058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:35:18.090345 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=87057) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:35:18.090481 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=87058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:35:18.090567 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=87057) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:35:18.090711 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=87058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:35:18.090790 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=87057) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:35:18.090955 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=87057) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:35:18.091042 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=87058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:35:18.091186 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=87057) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:35:18.091266 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=87058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:35:18.091418 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=87058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:35:18.091496 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=87057) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:35:18.091803 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=87058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:35:18.091939 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=87057) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:35:18.092046 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=87058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:35:18.092146 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=87057) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:35:18.092244 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=87058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:35:18.092341 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=87057) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:35:18.092438 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=87058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:35:18.092544 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=87057) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:35:18.092643 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=87058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:35:18.092723 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=87057) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:35:18.092833 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=87058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:35:18.092943 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=87057) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:35:18.093067 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=87058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:35:18.093184 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=87057) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:35:18.093262 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=87058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:35:18.093386 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=87057) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:35:18.093464 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=87058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:35:18.093539 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=87057) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:35:18.093665 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=87058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:35:18.093782 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=87057) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:35:18.093860 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=87058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:35:18.093977 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=87057) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:35:18.094104 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=87058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:35:18.094229 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=87057) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:35:18.094320 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=87058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:35:18.094405 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=87057) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:35:18.094558 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=87058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:35:18.094654 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=87057) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:35:18.094794 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=87058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:35:18.094872 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=87057) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:35:18.095032 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=87057) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:35:18.095119 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=87058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:35:18.095259 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=87058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:35:18.095339 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=87057) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:35:18.095494 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=87058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:35:18.095621 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=87057) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:35:18.095759 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=87057) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:35:18.095848 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=87058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:35:18.095975 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=87058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:35:18.096052 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=87057) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:35:18.096169 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=87058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:35:18.096245 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=87057) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:35:18.096395 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=87058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:35:18.096490 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=87057) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:35:18.096622 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=87058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:35:18.096702 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=87057) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:35:18.096856 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=87057) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:35:18.096936 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=87058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:35:18.097087 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=87058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:35:18.097218 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=87057) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:35:18.097320 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=87058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:35:18.097407 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=87057) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:35:18.097549 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=87058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:35:18.097638 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=87057) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:35:18.097782 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=87057) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:35:18.097859 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=87058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:35:18.097984 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=87057) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:35:18.098116 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=87058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:35:18.098264 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=87058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:35:18.098352 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=87057) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:35:18.098499 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=87057) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:35:18.098578 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=87058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:35:18.098729 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=87058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:35:18.098811 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=87057) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:35:18.098962 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=87057) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:35:18.099047 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=87058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:35:18.099198 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=87057) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:35:18.099328 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=87058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:35:18.099446 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=87058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:35:18.099526 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=87057) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:35:18.099687 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=87058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:35:18.099767 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=87057) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:35:18.099919 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=87058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:35:18.100006 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=87057) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:35:18.100157 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=87058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:35:18.100234 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=87057) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:35:18.100371 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=87058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:35:18.100451 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=87057) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:35:18.100606 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=87057) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:35:18.100684 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG uwsgi [-] cache.enabled = True {{(pid=87058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:35:18.100836 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=87058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:35:18.100916 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=87057) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:35:18.101068 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=87058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:35:18.101148 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG uwsgi [-] cache.enabled = True {{(pid=87057) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:35:18.101304 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=87057) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:35:18.101382 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=87058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:35:18.101535 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=87058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:35:18.101626 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=87057) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:35:18.101767 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=87058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:35:18.101847 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=87057) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:35:18.101997 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=87058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:35:18.102101 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=87057) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:35:18.102257 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=87057) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:35:18.102348 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=87058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:35:18.102487 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=87058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:35:18.102572 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=87057) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:35:18.102721 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=87057) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:35:18.102803 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=87058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:35:18.102957 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=87057) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:35:18.103069 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=87058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:35:18.103231 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=87058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:35:18.103326 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=87057) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:35:18.103467 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=87058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:35:18.103555 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=87057) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:35:18.103735 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=87057) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:35:18.103823 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=87058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:35:18.103972 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=87057) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:35:18.104053 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=87058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:35:18.104213 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=87057) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:35:18.104294 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=87058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:35:18.104451 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=87057) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:35:18.104540 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=87058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:35:18.104678 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=87058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:35:18.104764 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=87057) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:35:18.104907 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=87058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:35:18.104988 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=87057) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:35:18.105137 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=87058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:35:18.105226 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=87057) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:35:18.105368 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=87057) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:35:18.105447 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=87058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:35:18.105593 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=87058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:35:18.105672 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=87057) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:35:18.105828 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=87057) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:35:18.105915 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=87058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:35:18.106082 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=87058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:35:18.106167 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=87057) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:35:18.106317 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=87057) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:35:18.106397 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=87058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:35:18.106549 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=87058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:35:18.106641 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=87057) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:35:18.106795 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=87057) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:35:18.106874 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=87058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:35:18.107031 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=87057) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:35:18.107111 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=87058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:35:18.107275 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=87058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:35:18.107362 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=87057) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:35:18.107504 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG uwsgi [-] auth.external = None {{(pid=87058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:35:18.107584 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=87057) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:35:18.107735 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG uwsgi [-] auth.mapped = None {{(pid=87058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:35:18.107815 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=87057) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:35:18.107978 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG uwsgi [-] auth.external = None {{(pid=87057) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:35:18.108065 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=87058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:35:18.108209 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG uwsgi [-] auth.mapped = None {{(pid=87057) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:35:18.108317 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=87058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:35:18.108467 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG uwsgi [-] auth.password = None {{(pid=87058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:35:18.108549 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=87057) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:35:18.108695 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG uwsgi [-] auth.token = None {{(pid=87058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:35:18.108775 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=87057) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:35:18.108929 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=87058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:35:18.109022 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG uwsgi [-] auth.password = None {{(pid=87057) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:35:18.109159 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG uwsgi [-] catalog.caching = True {{(pid=87058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:35:18.109248 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG uwsgi [-] auth.token = None {{(pid=87057) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:35:18.109397 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=87057) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:35:18.109477 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=87058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:35:18.109607 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=87058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:35:18.109694 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG uwsgi [-] catalog.caching = True {{(pid=87057) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:35:18.109850 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=87058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:35:18.109940 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=87057) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:35:18.110100 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=87057) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:35:18.110181 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=87058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:35:18.110341 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=87058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:35:18.110434 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=87057) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:35:18.110573 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG uwsgi [-] credential.caching = True {{(pid=87058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:35:18.110656 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=87057) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:35:18.110811 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=87057) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:35:18.110898 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG uwsgi [-] credential.driver = sql {{(pid=87058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:35:18.111047 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG uwsgi [-] credential.caching = True {{(pid=87057) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:35:18.111137 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=87058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:35:18.111289 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=87058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:35:18.111369 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG uwsgi [-] credential.driver = sql {{(pid=87057) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:35:18.111520 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=87058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:35:18.111603 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=87057) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:35:18.111766 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=87058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:35:18.111849 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=87057) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:35:18.111999 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=87058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:35:18.112081 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=87057) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:35:18.112241 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=87057) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:35:18.112327 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=87058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:35:18.112475 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=87057) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:35:18.112557 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=87058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:35:18.112724 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=87058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:35:18.112804 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=87057) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:35:18.112971 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=87058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:35:18.113061 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=87057) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:35:18.113207 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=87057) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:35:18.113288 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=87058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:35:18.113459 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=87057) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:35:18.113541 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=87058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:35:18.113722 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=87058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:35:18.113803 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=87057) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:35:18.113956 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=87058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:35:18.114072 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=87057) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:35:18.114229 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=87058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:35:18.114312 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=87057) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:35:18.114462 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG uwsgi [-] federation.caching = True {{(pid=87058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:35:18.114544 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=87057) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:35:18.114696 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=87058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:35:18.114780 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=87057) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:35:18.114938 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG uwsgi [-] federation.driver = sql {{(pid=87058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:35:18.115019 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG uwsgi [-] federation.caching = True {{(pid=87057) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:35:18.115172 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=87057) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:35:18.115253 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=87058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:35:18.115398 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=87058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:35:18.115479 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG uwsgi [-] federation.driver = sql {{(pid=87057) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:35:18.115641 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=87058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:35:18.115724 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=87057) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:35:18.115873 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=87057) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:35:18.115952 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=87058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:35:18.116122 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=87057) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:35:18.116204 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=87058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:35:18.116353 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=87058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:35:18.116432 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=87057) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:35:18.116605 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=87058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:35:18.116686 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=87057) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:35:18.116838 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=87058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:35:18.116921 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=87057) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:35:18.117081 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=87058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:35:18.117163 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=87057) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:35:18.117312 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG uwsgi [-] identity.caching = True {{(pid=87058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:35:18.117393 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=87057) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:35:18.117554 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=87058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:35:18.117637 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=87057) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:35:18.117787 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG uwsgi [-] identity.caching = True {{(pid=87057) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:35:18.117874 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=87058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:35:18.118017 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=87058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:35:18.118117 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=87057) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:35:18.118277 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=87058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:35:18.118361 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=87057) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:35:18.118520 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=87057) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:35:18.118599 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG uwsgi [-] identity.driver = sql {{(pid=87058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:35:18.119291 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=87057) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:35:18.119374 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=87058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:35:18.119540 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG uwsgi [-] identity.driver = sql {{(pid=87057) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:35:18.119668 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=87058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:35:18.119774 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=87057) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:35:18.119856 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=87058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:35:18.120014 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=87057) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:35:18.120095 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=87058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:35:18.120265 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=87057) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:35:18.120348 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=87058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:35:18.120517 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=87057) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:35:18.120599 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=87058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:35:18.120749 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=87057) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:35:18.120830 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=87058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:35:18.120982 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=87057) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:35:18.121065 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=87058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:35:18.121218 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=87057) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:35:18.121299 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=87058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:35:18.121461 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=87058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:35:18.121553 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=87057) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:35:18.121706 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=87057) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:35:18.121787 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=87058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:35:18.121948 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=87058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:35:18.122075 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=87057) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:35:18.122224 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=87057) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:35:18.122304 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=87058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:35:18.122464 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=87058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:35:18.122606 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=87057) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:35:18.122699 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=87058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:35:18.122769 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=87057) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:35:18.124081 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=87058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:35:18.124189 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=87057) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:35:18.124268 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=87058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:35:18.124342 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=87057) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:35:18.124421 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=87058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:35:18.124496 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=87057) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:35:18.124569 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=87058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:35:18.124650 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=87057) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:35:18.124725 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=87058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:35:18.124799 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=87057) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:35:18.124871 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=87058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:35:18.124950 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=87057) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:35:18.125025 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=87058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:35:18.125105 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=87057) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:35:18.125181 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=87058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:35:18.125262 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=87057) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:35:18.125337 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=87058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:35:18.125410 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=87057) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:35:18.125483 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=87058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:35:18.125563 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=87057) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:35:18.125638 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=87058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:35:18.125711 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=87057) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:35:18.125783 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=87058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:35:18.125855 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=87057) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:35:18.125928 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=87058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:35:18.126008 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=87057) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:35:18.126103 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=87058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:35:18.126178 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=87057) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:35:18.126254 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=87058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:35:18.126328 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=87057) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:35:18.126401 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG uwsgi [-] ldap.password = **** {{(pid=87058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:35:18.126486 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=87057) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:35:18.126562 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=87058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:35:18.126635 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=87057) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:35:18.126735 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=87058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:35:18.126818 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG uwsgi [-] ldap.password = **** {{(pid=87057) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:35:18.126972 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=87058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:35:18.127061 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=87057) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:35:18.127207 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=87058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:35:18.127289 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=87057) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:35:18.127440 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=87058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:35:18.127520 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=87057) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:35:18.127683 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=87058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:35:18.127772 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=87057) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:35:18.127916 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=87057) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:35:18.127996 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=87058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:35:18.128144 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=87058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:35:18.128264 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=87057) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:35:18.128380 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=87058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:35:18.128467 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=87057) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:35:18.128612 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=87058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:35:18.128699 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=87057) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:35:18.128841 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=87057) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:35:18.128921 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=87058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:35:18.129074 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=87058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:35:18.129163 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=87057) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:35:18.129306 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=87058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:35:18.129386 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=87057) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:35:18.129538 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=87058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:35:18.129619 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=87057) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:35:18.129766 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG uwsgi [-] ldap.user = None {{(pid=87058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:35:18.129860 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=87057) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:35:18.130000 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=87057) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:35:18.130105 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=87058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:35:18.130253 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG uwsgi [-] ldap.user = None {{(pid=87057) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:35:18.130334 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=87058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:35:18.130494 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=87057) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:35:18.130589 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=87058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:35:18.130735 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=87058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:35:18.130816 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=87057) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:35:18.130969 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=87057) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:35:18.131049 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=87058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:35:18.131208 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=87058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:35:18.131296 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=87057) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:35:18.131440 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=87058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:35:18.131522 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=87057) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:35:18.131666 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=87058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:35:18.131747 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=87057) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:35:18.131897 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=87058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:35:18.131986 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=87057) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:35:18.132127 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=87057) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:35:18.132207 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=87058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:35:18.132357 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=87058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:35:18.132437 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=87057) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:35:18.132584 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=87058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:35:18.132674 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=87057) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:35:18.132818 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=87057) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:35:18.132898 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=87058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:35:18.133048 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=87057) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:35:18.133129 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=87058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:35:18.133288 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=87058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:35:18.133376 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=87057) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:35:18.133531 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=87057) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:35:18.133633 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=87058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:35:18.133776 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=87057) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:35:18.133879 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=87058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:35:18.134132 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=87058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:35:18.134218 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=87057) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:35:18.134928 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=87058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:35:18.135069 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=87057) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:35:18.135201 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=87058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:35:18.135329 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=87057) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:35:18.135470 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=87058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:35:18.135599 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=87057) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:35:18.135728 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=87058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:35:18.135858 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=87057) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:35:18.135998 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=87058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:35:18.136128 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=87057) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:35:18.136266 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=87058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:35:18.136394 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=87057) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:35:18.136523 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=87058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:35:18.136650 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=87057) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:35:18.136779 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=87058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:35:18.136910 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=87057) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:35:18.137047 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG uwsgi [-] policy.driver = sql {{(pid=87058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:35:18.137174 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=87057) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:35:18.137304 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=87058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:35:18.137432 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=87057) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:35:18.137559 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=87058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:35:18.137685 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG uwsgi [-] policy.driver = sql {{(pid=87057) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:35:18.137897 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=87058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:35:18.138050 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=87057) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:35:18.138181 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG uwsgi [-] receipt.caching = True {{(pid=87058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:35:18.138312 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=87057) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:35:18.138443 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=87058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:35:18.138572 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=87057) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:35:18.138712 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=87058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:35:18.138840 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG uwsgi [-] receipt.caching = True {{(pid=87057) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:35:18.138971 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=87058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:35:18.139100 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=87057) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:35:18.139230 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=87058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:35:18.139359 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=87057) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:35:18.139501 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=87058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:35:18.139630 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=87057) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:35:18.139770 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG uwsgi [-] resource.caching = True {{(pid=87058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:35:18.139904 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=87057) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:35:18.140036 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=87058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:35:18.140166 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=87057) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:35:18.140303 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG uwsgi [-] resource.driver = sql {{(pid=87058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:35:18.140433 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG uwsgi [-] resource.caching = True {{(pid=87057) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:35:18.140564 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=87058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:35:18.140693 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=87057) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:35:18.140822 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=87058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:35:18.140951 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG uwsgi [-] resource.driver = sql {{(pid=87057) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:35:18.141090 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=87058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:35:18.141225 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=87057) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:35:18.141355 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG uwsgi [-] revoke.caching = True {{(pid=87058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:35:18.141484 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=87057) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:35:18.141568 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=87058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:35:18.141687 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=87057) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:35:18.141825 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=87058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:35:18.141952 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG uwsgi [-] revoke.caching = True {{(pid=87057) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:35:18.142107 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG uwsgi [-] role.cache_time = None {{(pid=87058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:35:18.142234 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=87057) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:35:18.142364 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG uwsgi [-] role.caching = True {{(pid=87058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:35:18.142491 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=87057) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:35:18.142646 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG uwsgi [-] role.driver = sql {{(pid=87058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:35:18.142783 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG uwsgi [-] role.cache_time = None {{(pid=87057) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:35:18.142912 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG uwsgi [-] role.list_limit = None {{(pid=87058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:35:18.143040 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG uwsgi [-] role.caching = True {{(pid=87057) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:35:18.143170 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=87058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:35:18.143308 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG uwsgi [-] role.driver = sql {{(pid=87057) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:35:18.143444 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=87058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:35:18.143823 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG uwsgi [-] role.list_limit = None {{(pid=87057) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:35:18.143954 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=87058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:35:18.144081 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=87057) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:35:18.144211 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=87058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:35:18.144340 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=87057) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:35:18.144476 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=87058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:35:18.144604 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=87057) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:35:18.144731 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=87058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:35:18.144857 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=87057) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:35:18.144985 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=87058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:35:18.145112 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=87057) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:35:18.145257 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=87058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:35:18.145385 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=87057) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:35:18.145513 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=87058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:35:18.145639 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=87057) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:35:18.145767 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=87058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:35:18.145893 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=87057) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:35:18.146057 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=87058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:35:18.146188 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=87057) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:35:18.146319 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=87058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:35:18.146449 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=87057) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:35:18.146578 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=87058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:35:18.146716 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=87057) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:35:18.146846 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=87058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:35:18.146975 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=87057) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:35:18.147114 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=87058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:35:18.147245 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=87057) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:35:18.147374 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=87058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:35:18.147511 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=87057) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:35:18.147639 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=87058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:35:18.147768 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=87057) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:35:18.147914 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=87058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:35:18.147995 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=87057) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:35:18.148121 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=87058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:35:18.148209 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=87057) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:35:18.148286 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=87058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:35:18.148366 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=87057) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:35:18.148441 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=87058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:35:18.148514 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=87057) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:35:18.148587 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=87058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:35:18.148668 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=87057) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:35:18.148743 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=87058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:35:18.148743 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=87058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:35:18.148874 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=87057) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:35:18.148946 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=87058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:35:18.149019 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=87057) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:35:18.149099 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=87058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:35:18.149175 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=87057) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:35:18.149248 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=87058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:35:18.149321 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=87057) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:35:18.149394 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=87058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:35:18.149466 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=87057) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:35:18.149547 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=87058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:35:18.149622 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=87057) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:35:18.149695 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=87058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:35:18.149767 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=87057) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:35:18.149836 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=87058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:35:18.149942 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=87057) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:35:18.150094 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG uwsgi [-] token.cache_time = None {{(pid=87058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:35:18.150175 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=87057) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:35:18.150249 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG uwsgi [-] token.caching = True {{(pid=87058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:35:18.150321 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=87057) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:35:18.150394 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=87058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:35:18.150466 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=87057) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:35:18.150466 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG uwsgi [-] token.cache_time = None {{(pid=87057) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:35:18.150619 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG uwsgi [-] token.provider = fernet {{(pid=87058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:35:18.150619 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=87058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:35:18.150751 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG uwsgi [-] token.caching = True {{(pid=87057) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:35:18.150823 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=87058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:35:18.150896 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=87057) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:35:18.151052 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=87058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:35:18.151135 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG uwsgi [-] token.provider = fernet {{(pid=87057) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:35:18.151289 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=87058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:35:18.151370 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=87057) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:35:18.151531 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=87058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:35:18.151612 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=87057) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:35:18.151773 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=87058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:35:18.152017 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG uwsgi [-] trust.driver = sql {{(pid=87058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:35:18.152098 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=87057) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:35:18.152253 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=87058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:35:18.152334 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=87057) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:35:18.152489 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=87058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:35:18.152579 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=87057) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:35:18.152721 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=87058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:35:18.152803 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=87057) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:35:18.152964 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=87058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:35:18.153046 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG uwsgi [-] trust.driver = sql {{(pid=87057) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:35:18.153210 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=87058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:35:18.153300 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=87057) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:35:18.153438 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=87058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:35:18.153519 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=87057) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:35:18.153679 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=87058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:35:18.153768 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=87057) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:35:18.153924 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=87057) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:35:18.154005 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=87058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:35:18.154196 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=87057) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:35:18.154281 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG uwsgi [-] database.connection = **** {{(pid=87058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:35:18.154428 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=87057) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:35:18.154519 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=87058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:35:18.154712 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=87057) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:35:18.154801 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=87058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:35:18.154924 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=87058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:35:18.155006 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=87057) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:35:18.155161 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=87058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:35:18.155242 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG uwsgi [-] database.connection = **** {{(pid=87057) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:35:18.155395 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=87058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:35:18.155486 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=87057) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:35:18.155630 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=87058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:35:18.155720 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=87057) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:35:18.155863 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=87058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:35:18.155945 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=87057) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:35:18.156097 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=87058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:35:18.156179 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=87057) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:35:18.156339 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=87058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:35:18.156498 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=87057) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:35:18.156577 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=87058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:35:18.156651 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=87057) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:35:18.156795 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=87057) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:35:18.156876 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=87058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:35:18.157024 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=87058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:35:18.157113 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=87057) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:35:18.157265 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=87057) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:35:18.157346 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=87058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:35:18.157495 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=87058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:35:18.157575 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=87057) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:35:18.157734 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=87057) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:35:18.157823 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=87058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:35:18.157959 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=87058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:35:18.158143 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=87057) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:35:18.158224 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=87058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:35:18.158297 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=87057) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:35:18.158444 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=87058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:35:18.158538 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=87057) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:35:18.158615 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=87058) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Aug 01 16:35:18.158689 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=87057) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:35:18.158900 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=87057) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:35:18.159140 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=87057) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:35:18.159377 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=87057) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 16:35:18.159546 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=87057) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Aug 01 16:35:18.187933 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=87058) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 16:35:18.188160 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=87058) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 16:35:18.188393 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=87058) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 16:35:18.188967 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=87057) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 16:35:18.189203 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=87057) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 16:35:18.189432 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=87057) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 16:35:18.196647 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=87058) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 16:35:18.198320 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=87057) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 16:35:18.226590 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=87058) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 16:35:18.226819 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=87058) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 16:35:18.227676 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=87057) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 16:35:18.227911 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=87057) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 16:35:18.279992 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=87058) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 16:35:18.280131 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=87057) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 16:35:18.330857 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=87058) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 16:35:18.331091 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=87058) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 16:35:18.331528 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=87057) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 16:35:18.331765 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=87057) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 16:35:18.396904 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=87058) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 16:35:18.397127 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=87058) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 16:35:18.397274 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=87057) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 16:35:18.397365 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=87058) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 16:35:18.397509 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=87057) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 16:35:18.397595 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=87058) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 16:35:18.397736 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=87057) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 16:35:18.397830 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=87058) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 16:35:18.397974 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=87058) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 16:35:18.398081 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=87057) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 16:35:18.398199 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=87057) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 16:35:18.398415 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=87057) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 16:35:18.422056 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=87057) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 16:35:18.422178 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=87058) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 16:35:18.422261 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=87057) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 16:35:18.422338 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=87058) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 16:35:18.422449 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=87057) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 16:35:18.422535 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=87058) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 16:35:18.422669 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=87057) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 16:35:18.422755 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=87058) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 16:35:18.422885 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=87057) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 16:35:18.422970 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=87058) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 16:35:18.423150 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=87057) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 16:35:18.423355 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=87058) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 16:35:18.423355 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=87058) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 16:35:18.423499 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=87057) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 16:35:18.434991 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=87058) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 16:35:18.435187 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=87057) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 16:35:18.447716 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=87058) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 16:35:18.447964 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=87058) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 16:35:18.448165 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=87058) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 16:35:18.448383 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=87058) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 16:35:18.448598 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=87058) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 16:35:18.448756 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=87057) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 16:35:18.448858 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=87058) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 16:35:18.448994 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=87057) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 16:35:18.449080 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=87058) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 16:35:18.449222 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=87057) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 16:35:18.449450 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=87057) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 16:35:18.449670 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=87057) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 16:35:18.449887 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=87057) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 16:35:18.450136 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=87057) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 16:35:18.466362 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=87058) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 16:35:18.466563 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=87058) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 01 16:35:18.466663 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=87057) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 16:35:18.466777 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=87057) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 01 16:35:18.466892 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=87058) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 16:35:18.467088 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=87058) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 01 16:35:18.467171 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=87057) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 16:35:18.467299 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=87057) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 01 16:35:18.467392 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=87058) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 16:35:18.467569 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=87057) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 16:35:18.467653 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=87058) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 16:35:18.467826 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=87057) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 16:35:18.467909 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=87058) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 16:35:18.468067 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=87058) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 16:35:18.468158 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=87057) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 16:35:18.468317 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=87057) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 16:35:18.477752 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=87058) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 16:35:18.478610 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=87058) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 16:35:18.478708 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=87057) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 16:35:18.478920 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=87058) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 16:35:18.479088 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=87057) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 16:35:18.479330 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=87058) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 16:35:18.479420 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=87057) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 16:35:18.479628 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=87057) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 16:35:18.486079 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=87058) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 01 16:35:18.486608 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=87057) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 01 16:35:18.490807 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=87058) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 01 16:35:18.491080 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=87058) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 16:35:18.491324 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=87057) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 01 16:35:18.491410 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=87058) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 16:35:18.491600 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=87057) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 16:35:18.491686 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=87058) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 16:35:18.491873 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=87058) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 16:35:18.491957 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=87057) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 16:35:18.492126 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=87058) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 16:35:18.492220 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=87057) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 16:35:18.492396 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=87058) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 16:35:18.492481 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=87057) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 16:35:18.492659 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=87057) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 16:35:18.492740 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=87058) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 16:35:18.492920 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=87058) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 16:35:18.493003 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=87057) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 16:35:18.493182 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=87058) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 16:35:18.493266 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=87057) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 16:35:18.493439 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=87058) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 16:35:18.493521 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=87057) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 16:35:18.493872 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=87057) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 16:35:18.494145 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=87057) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 16:35:18.516751 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=87058) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 16:35:18.517633 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=87057) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 16:35:18.518251 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=87058) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 01 16:35:18.518783 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=87058) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 16:35:18.519423 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=87057) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 01 16:35:18.519700 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=87057) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 16:35:18.523378 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=87058) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 01 16:35:18.523653 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=87058) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 16:35:18.523926 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=87058) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 16:35:18.524329 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=87057) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 01 16:35:18.524609 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=87057) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 16:35:18.524949 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=87057) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 16:35:18.530750 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=87058) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 01 16:35:18.531028 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=87058) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 16:35:18.531732 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=87057) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 01 16:35:18.532017 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=87057) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 16:35:18.535112 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=87058) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 01 16:35:18.535394 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=87058) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 16:35:18.535671 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=87058) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 16:35:18.535948 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=87058) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 16:35:18.536228 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=87058) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 16:35:18.536497 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=87058) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 16:35:18.536740 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=87057) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 01 16:35:18.536824 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=87058) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 16:35:18.537029 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=87058) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 16:35:18.537112 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=87057) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 16:35:18.537318 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=87057) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 16:35:18.537598 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=87057) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 16:35:18.537872 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=87057) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 16:35:18.538188 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=87057) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 16:35:18.538461 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=87057) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 16:35:18.538734 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=87057) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 16:35:18.554009 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=87058) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 16:35:18.554305 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=87058) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 16:35:18.555490 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=87057) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 16:35:18.555769 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=87057) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 16:35:18.558837 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=87058) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 01 16:35:18.560285 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=87057) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 01 16:35:18.561956 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=87058) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 16:35:18.562267 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=87058) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 16:35:18.563482 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=87057) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 16:35:18.563775 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=87057) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 16:35:18.566647 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=87058) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 01 16:35:18.568166 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=87057) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 01 16:35:18.569787 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=87058) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 01 16:35:18.571366 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=87057) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 01 16:35:18.573122 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=87058) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 16:35:18.573417 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=87058) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 16:35:18.573695 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=87058) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 16:35:18.573989 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=87058) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 16:35:18.574284 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=87058) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 16:35:18.574572 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=87058) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 16:35:18.574683 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=87057) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 16:35:18.574977 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=87057) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 16:35:18.575260 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=87057) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 16:35:18.575551 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=87057) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 16:35:18.575835 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=87057) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 16:35:18.576120 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=87057) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 16:35:18.590662 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=87058) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 01 16:35:18.590939 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=87058) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 16:35:18.591200 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=87058) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 16:35:18.591454 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=87058) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 16:35:18.592140 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=87057) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 01 16:35:18.592419 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=87057) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 16:35:18.592683 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=87057) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 16:35:18.592942 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=87057) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 16:35:18.598287 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=87058) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 16:35:18.599819 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=87058) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 16:35:18.599904 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=87057) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 16:35:18.600088 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=87058) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 16:35:18.601396 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=87057) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 16:35:18.601669 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=87057) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 16:35:18.602914 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=87058) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 01 16:35:18.603715 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=87058) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 16:35:18.604139 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=87058) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 16:35:18.604430 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=87058) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 16:35:18.604717 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=87058) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 16:35:18.605049 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=87057) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 01 16:35:18.605328 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=87057) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 16:35:18.605629 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=87057) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 16:35:18.605909 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=87057) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 16:35:18.606217 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=87057) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 16:35:18.616067 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=87058) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 01 16:35:18.616341 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=87058) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 16:35:18.616612 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=87058) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 16:35:18.616884 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=87058) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 16:35:18.617154 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=87058) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 16:35:18.617418 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=87058) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 16:35:18.617685 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=87058) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 16:35:18.617767 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=87057) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 01 16:35:18.617959 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=87057) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 16:35:18.618260 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=87057) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 16:35:18.618535 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=87057) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 16:35:18.619376 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=87057) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 16:35:18.619644 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=87057) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 16:35:18.619926 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=87057) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 16:35:18.633854 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=87058) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 01 16:35:18.635594 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=87057) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 01 16:35:18.636833 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=87058) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 01 16:35:18.638612 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=87057) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 01 16:35:18.639793 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=87058) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 16:35:18.641582 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=87057) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 16:35:18.642277 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=87058) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 16:35:18.644036 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=87058) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 01 16:35:18.644313 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=87058) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 16:35:18.644396 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=87057) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 16:35:18.644587 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=87058) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 16:35:18.645869 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=87057) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 01 16:35:18.646180 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=87057) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 16:35:18.646463 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=87057) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 16:35:18.651031 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=87058) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 16:35:18.652629 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=87058) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 01 16:35:18.653072 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=87057) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 16:35:18.654701 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=87057) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 01 16:35:18.655642 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=87058) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 16:35:18.655929 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=87058) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 16:35:18.656204 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=87058) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 16:35:18.656485 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=87058) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 16:35:18.657726 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=87057) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 16:35:18.657998 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=87057) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 16:35:18.658312 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=87057) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 16:35:18.658586 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=87057) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 16:35:18.664546 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=87058) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 01 16:35:18.664825 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=87058) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 16:35:18.665100 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=87058) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 16:35:18.667289 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=87057) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 01 16:35:18.667572 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=87057) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 16:35:18.667851 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=87057) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 16:35:18.672510 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=87058) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 01 16:35:18.672739 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=87058) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 16:35:18.673011 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=87058) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 16:35:18.673303 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=87058) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 16:35:18.673572 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=87058) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 16:35:18.673845 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=87058) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 16:35:18.674132 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=87058) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 16:35:18.674394 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=87058) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 16:35:18.674727 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=87057) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 01 16:35:18.674810 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=87058) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 16:35:18.675006 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=87057) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 16:35:18.675091 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=87058) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 16:35:18.675284 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=87057) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 16:35:18.675367 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=87058) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 16:35:18.675542 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=87058) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 16:35:18.675633 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=87057) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 16:35:18.675810 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=87057) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 16:35:18.675892 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=87058) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 16:35:18.676063 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=87058) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 16:35:18.676144 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=87057) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 16:35:18.676341 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=87057) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 16:35:18.676614 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=87057) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 16:35:18.676981 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=87057) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 16:35:18.677259 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=87057) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 16:35:18.677542 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=87057) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 16:35:18.677802 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=87057) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 16:35:18.678091 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=87057) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 16:35:18.678364 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=87057) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 16:35:18.709897 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x55f81eee69c0 pid: 87058 (default app) Aug 01 16:35:18.713332 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x55f81eee69c0 pid: 87057 (default app) Aug 01 16:35:18.713818 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b84abe-dd01-4669-a2a2-391ed2d7b834 None None] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:35:18.714131 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b84abe-dd01-4669-a2a2-391ed2d7b834 None None] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:35:18.714407 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b84abe-dd01-4669-a2a2-391ed2d7b834 None None] PATH_INFO: `/v3` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:35:18.718682 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 1/1] 10.222.0.20 () {50 vars in 791 bytes} [Mon Aug 1 16:35:18 2022] GET /identity/v3/ => generated 255 bytes in 8 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:35:27.251417 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728944ee-e10a-4f09-a274-df7c5ffb6295 None None] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:35:27.251961 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728944ee-e10a-4f09-a274-df7c5ffb6295 None None] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:35:27.251961 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728944ee-e10a-4f09-a274-df7c5ffb6295 None None] PATH_INFO: `/` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:35:27.256339 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 1/2] 10.222.0.20 () {54 vars in 926 bytes} [Mon Aug 1 16:35:27 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 01 16:35:27.262213 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232f6f8e-9914-44de-93fb-8f2cde9ad677 None None] REQUEST_METHOD: `POST` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:35:27.262647 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232f6f8e-9914-44de-93fb-8f2cde9ad677 None None] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:35:27.262749 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232f6f8e-9914-44de-93fb-8f2cde9ad677 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:35:27.333878 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: INFO dbcounter [None req-232f6f8e-9914-44de-93fb-8f2cde9ad677 None None] Registered counter for database keystone Aug 01 16:35:27.353627 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG oslo_db.sqlalchemy.engines [None req-232f6f8e-9914-44de-93fb-8f2cde9ad677 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=87058) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Aug 01 16:35:27.359887 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG dbcounter [-] [87058] Writer thread running {{(pid=87058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Aug 01 16:35:27.538701 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG passlib.handlers.bcrypt [None req-232f6f8e-9914-44de-93fb-8f2cde9ad677 None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=87058) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Aug 01 16:35:27.539130 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG passlib.handlers.bcrypt [None req-232f6f8e-9914-44de-93fb-8f2cde9ad677 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=87058) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Aug 01 16:35:27.593423 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.auth.core [None req-232f6f8e-9914-44de-93fb-8f2cde9ad677 None None] MFA Rules not processed for user `1007f8e379b74e169b98fb7c3c0ba5b4`. Rule list: `[]` (Enabled: `True`). {{(pid=87058) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:35:27.595087 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.fernet_utils [None req-232f6f8e-9914-44de-93fb-8f2cde9ad677 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:35:27.723340 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 2/3] 10.222.0.20 () {58 vars in 1023 bytes} [Mon Aug 1 16:35:27 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 462 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Aug 01 16:35:27.844229 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-728944ee-e10a-4f09-a274-df7c5ffb6295 None None] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:35:27.915190 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: INFO dbcounter [None req-728944ee-e10a-4f09-a274-df7c5ffb6295 None None] Registered counter for database keystone Aug 01 16:35:27.936873 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG oslo_db.sqlalchemy.engines [None req-728944ee-e10a-4f09-a274-df7c5ffb6295 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=87057) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Aug 01 16:35:27.943692 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG dbcounter [-] [87057] Writer thread running {{(pid=87057) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Aug 01 16:35:28.064498 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-728944ee-e10a-4f09-a274-df7c5ffb6295 None None] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:35:28.068818 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-901620c8-8ab4-49f6-82ad-cd1ce9820b66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:35:28.069445 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901620c8-8ab4-49f6-82ad-cd1ce9820b66 admin admin] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:35:28.069726 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901620c8-8ab4-49f6-82ad-cd1ce9820b66 admin admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:35:28.069998 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901620c8-8ab4-49f6-82ad-cd1ce9820b66 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:35:28.078816 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-901620c8-8ab4-49f6-82ad-cd1ce9820b66 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:35:28.130209 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: WARNING keystone.common.rbac_enforcer.enforcer [None req-901620c8-8ab4-49f6-82ad-cd1ce9820b66 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Aug 01 16:35:28.130492 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-901620c8-8ab4-49f6-82ad-cd1ce9820b66 admin admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:35:28.140109 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 2/4] 10.222.0.20 () {58 vars in 1324 bytes} [Mon Aug 1 16:35:27 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 296 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 01 16:35:28.146480 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-232f6f8e-9914-44de-93fb-8f2cde9ad677 None None] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:35:28.160901 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-232f6f8e-9914-44de-93fb-8f2cde9ad677 None None] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:35:28.165853 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8cddfc7-90ab-42f1-af61-8d5e344b65dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:35:28.166996 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8cddfc7-90ab-42f1-af61-8d5e344b65dc admin admin] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:35:28.167274 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8cddfc7-90ab-42f1-af61-8d5e344b65dc admin admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:35:28.167545 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8cddfc7-90ab-42f1-af61-8d5e344b65dc admin admin] PATH_INFO: `/v3/projects/3e43f22f36744d079b60704aff4969d4` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:35:28.169050 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8cddfc7-90ab-42f1-af61-8d5e344b65dc admin admin] RBAC: Authorizing `identity:get_project(project_id=3e43f22f36744d079b60704aff4969d4)` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:35:28.219559 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: WARNING keystone.common.rbac_enforcer.enforcer [None req-b8cddfc7-90ab-42f1-af61-8d5e344b65dc admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Aug 01 16:35:28.219912 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8cddfc7-90ab-42f1-af61-8d5e344b65dc admin admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:35:28.224530 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 3/5] 10.222.0.20 () {56 vars in 1207 bytes} [Mon Aug 1 16:35:28 2022] GET /identity/v3/projects/3e43f22f36744d079b60704aff4969d4 => generated 344 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 16:35:30.216078 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b2adaf3-2b0a-4e78-9c06-d98cc1b19b5b None None] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:35:30.216571 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b2adaf3-2b0a-4e78-9c06-d98cc1b19b5b None None] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:35:30.216571 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b2adaf3-2b0a-4e78-9c06-d98cc1b19b5b None None] PATH_INFO: `/` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:35:30.221361 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 3/6] 10.222.0.20 () {54 vars in 926 bytes} [Mon Aug 1 16:35:30 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 16:35:30.226809 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb14007-0eec-463a-885a-fb3b24c666bd None None] REQUEST_METHOD: `POST` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:35:30.227234 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb14007-0eec-463a-885a-fb3b24c666bd None None] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:35:30.227355 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb14007-0eec-463a-885a-fb3b24c666bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:35:30.247651 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.auth.core [None req-deb14007-0eec-463a-885a-fb3b24c666bd None None] MFA Rules not processed for user `1007f8e379b74e169b98fb7c3c0ba5b4`. Rule list: `[]` (Enabled: `True`). {{(pid=87058) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:35:30.249153 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.fernet_utils [None req-deb14007-0eec-463a-885a-fb3b24c666bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:35:30.256895 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 4/7] 10.222.0.20 () {58 vars in 1023 bytes} [Mon Aug 1 16:35:30 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Aug 01 16:35:30.279012 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b2adaf3-2b0a-4e78-9c06-d98cc1b19b5b None None] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:35:30.290950 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b2adaf3-2b0a-4e78-9c06-d98cc1b19b5b None None] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:35:30.295063 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be7c6f7-80ef-4b2e-8b9d-6f07787689b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:35:30.295665 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be7c6f7-80ef-4b2e-8b9d-6f07787689b2 admin admin] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:35:30.295942 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be7c6f7-80ef-4b2e-8b9d-6f07787689b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:35:30.296212 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be7c6f7-80ef-4b2e-8b9d-6f07787689b2 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:35:30.303802 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9be7c6f7-80ef-4b2e-8b9d-6f07787689b2 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:35:30.305575 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9be7c6f7-80ef-4b2e-8b9d-6f07787689b2 admin admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:35:30.314833 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 4/8] 10.222.0.20 () {58 vars in 1324 bytes} [Mon Aug 1 16:35:30 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 01 16:35:30.319938 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb14007-0eec-463a-885a-fb3b24c666bd None None] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:35:30.329586 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb14007-0eec-463a-885a-fb3b24c666bd None None] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:35:30.333955 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98edd7ca-b3fb-424f-bca1-23679b856949 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:35:30.334612 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98edd7ca-b3fb-424f-bca1-23679b856949 admin admin] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:35:30.334895 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98edd7ca-b3fb-424f-bca1-23679b856949 admin admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:35:30.335168 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98edd7ca-b3fb-424f-bca1-23679b856949 admin admin] PATH_INFO: `/v3/users/1007f8e379b74e169b98fb7c3c0ba5b4` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:35:30.336535 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98edd7ca-b3fb-424f-bca1-23679b856949 admin admin] RBAC: Authorizing `identity:get_user(user_id=1007f8e379b74e169b98fb7c3c0ba5b4)` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:35:30.338530 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98edd7ca-b3fb-424f-bca1-23679b856949 admin admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:35:30.347575 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 5/9] 10.222.0.20 () {56 vars in 1198 bytes} [Mon Aug 1 16:35:30 2022] GET /identity/v3/users/1007f8e379b74e169b98fb7c3c0ba5b4 => generated 247 bytes in 21 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 16:35:32.370584 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322dc846-3ddf-4bd1-9ad1-1521fec8744f None None] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:35:32.371117 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322dc846-3ddf-4bd1-9ad1-1521fec8744f None None] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:35:32.371117 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322dc846-3ddf-4bd1-9ad1-1521fec8744f None None] PATH_INFO: `/` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:35:32.372476 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 5/10] 10.222.0.20 () {54 vars in 926 bytes} [Mon Aug 1 16:35:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 16:35:32.377627 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19011842-699c-416b-8fc5-0d07cab31c3f None None] REQUEST_METHOD: `POST` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:35:32.378006 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19011842-699c-416b-8fc5-0d07cab31c3f None None] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:35:32.378327 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19011842-699c-416b-8fc5-0d07cab31c3f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:35:32.397652 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.auth.core [None req-19011842-699c-416b-8fc5-0d07cab31c3f None None] MFA Rules not processed for user `1007f8e379b74e169b98fb7c3c0ba5b4`. Rule list: `[]` (Enabled: `True`). {{(pid=87058) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:35:32.398888 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.fernet_utils [None req-19011842-699c-416b-8fc5-0d07cab31c3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:35:32.424198 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c04a109-0023-485b-b5c0-c4a8267849fa None None] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:35:32.424478 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c04a109-0023-485b-b5c0-c4a8267849fa None None] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:35:32.424746 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c04a109-0023-485b-b5c0-c4a8267849fa None None] PATH_INFO: `/` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:35:32.426089 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 6/11] 10.222.0.20 () {54 vars in 926 bytes} [Mon Aug 1 16:35:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 01 16:35:32.431044 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a259559-6e17-4b1e-81fa-f10f9c3344d6 None None] REQUEST_METHOD: `POST` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:35:32.431328 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a259559-6e17-4b1e-81fa-f10f9c3344d6 None None] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:35:32.431600 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a259559-6e17-4b1e-81fa-f10f9c3344d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:35:32.472864 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG passlib.handlers.bcrypt [None req-7a259559-6e17-4b1e-81fa-f10f9c3344d6 None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=87057) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Aug 01 16:35:32.473269 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG passlib.handlers.bcrypt [None req-7a259559-6e17-4b1e-81fa-f10f9c3344d6 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=87057) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Aug 01 16:35:32.478878 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 6/12] 10.222.0.20 () {58 vars in 1023 bytes} [Mon Aug 1 16:35:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 01 16:35:32.501430 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19011842-699c-416b-8fc5-0d07cab31c3f None None] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:35:32.519138 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.auth.core [None req-7a259559-6e17-4b1e-81fa-f10f9c3344d6 None None] MFA Rules not processed for user `1007f8e379b74e169b98fb7c3c0ba5b4`. Rule list: `[]` (Enabled: `True`). {{(pid=87057) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:35:32.520716 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.fernet_utils [None req-7a259559-6e17-4b1e-81fa-f10f9c3344d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87057) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:35:32.525362 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG dbcounter [-] [87058] Writing DB stats keystone:SELECT=100 {{(pid=87058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:35:32.606951 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19011842-699c-416b-8fc5-0d07cab31c3f None None] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:35:32.624684 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 7/13] 10.222.0.20 () {58 vars in 1023 bytes} [Mon Aug 1 16:35:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 01 16:35:32.642620 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a4b4654-fa55-4122-8a84-26fc029e2b02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:35:32.643520 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4b4654-fa55-4122-8a84-26fc029e2b02 None admin] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:35:32.643799 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4b4654-fa55-4122-8a84-26fc029e2b02 None admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:35:32.644074 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4b4654-fa55-4122-8a84-26fc029e2b02 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:35:32.646649 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a259559-6e17-4b1e-81fa-f10f9c3344d6 None None] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:35:32.656411 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a4b4654-fa55-4122-8a84-26fc029e2b02 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 16:35:32.664822 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application [None req-5a4b4654-fa55-4122-8a84-26fc029e2b02 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 16:35:32.664822 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 16:35:32.664822 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 16:35:32.664822 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 16:35:32.664822 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 16:35:32.664822 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 16:35:32.664822 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 16:35:32.664822 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 16:35:32.664822 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 16:35:32.664822 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 16:35:32.664822 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 16:35:32.664822 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 16:35:32.664822 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 16:35:32.664822 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 16:35:32.664822 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 16:35:32.664822 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 16:35:32.666202 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 16:35:32.666202 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 16:35:32.666202 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 16:35:32.666202 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 16:35:32.666202 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 16:35:32.666202 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 16:35:32.666202 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 16:35:32.666202 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 16:35:32.666202 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 16:35:32.666202 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 16:35:32.666202 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 16:35:32.666202 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application with Lock( Aug 01 16:35:32.666202 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 16:35:32.666202 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self._enter() Aug 01 16:35:32.666202 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 16:35:32.666202 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 16:35:32.667222 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 16:35:32.667222 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self.creator() Aug 01 16:35:32.667222 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 16:35:32.667222 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application created_value = creator( Aug 01 16:35:32.667222 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 16:35:32.667222 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 16:35:32.667222 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 16:35:32.667222 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 16:35:32.667222 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 16:35:32.667222 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 16:35:32.667222 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 16:35:32.667222 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application Aug 01 16:35:32.667222 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 7/14] 10.222.0.20 () {56 vars in 1096 bytes} [Mon Aug 1 16:35:32 2022] GET /identity/v3/roles/admin => generated 83 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 01 16:35:32.671875 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a4b4654-fa55-4122-8a84-26fc029e2b02 None admin] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:35:32.751726 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a259559-6e17-4b1e-81fa-f10f9c3344d6 None None] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:35:32.752995 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG dbcounter [-] [87058] Writing DB stats keystone:SELECT=100 {{(pid=87058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:35:32.754072 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG dbcounter [-] [87057] Writing DB stats keystone:SELECT=100 {{(pid=87057) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:35:32.772147 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a4b4654-fa55-4122-8a84-26fc029e2b02 None admin] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:35:32.791655 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf0284b3-a17a-4148-93a6-e98a6bbcea32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:35:32.792586 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0284b3-a17a-4148-93a6-e98a6bbcea32 None admin] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:35:32.792874 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0284b3-a17a-4148-93a6-e98a6bbcea32 None admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:35:32.793148 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0284b3-a17a-4148-93a6-e98a6bbcea32 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:35:32.806815 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b136d996-244f-4615-8bb4-292a8ff162ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:35:32.807429 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b136d996-244f-4615-8bb4-292a8ff162ff None admin] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:35:32.807711 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b136d996-244f-4615-8bb4-292a8ff162ff None admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:35:32.807980 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b136d996-244f-4615-8bb4-292a8ff162ff None admin] PATH_INFO: `/v3/roles` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:35:32.808702 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b136d996-244f-4615-8bb4-292a8ff162ff None admin] RBAC: Adding query filter params (name=admin) {{(pid=87058) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:35:32.809004 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b136d996-244f-4615-8bb4-292a8ff162ff None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:35:32.810408 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b136d996-244f-4615-8bb4-292a8ff162ff None admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:35:32.822530 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 8/15] 10.222.0.20 () {56 vars in 1099 bytes} [Mon Aug 1 16:35:32 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 16:35:32.828120 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b136d996-244f-4615-8bb4-292a8ff162ff None admin] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:35:32.861065 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf0284b3-a17a-4148-93a6-e98a6bbcea32 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:35:32.862634 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf0284b3-a17a-4148-93a6-e98a6bbcea32 None admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:35:32.928251 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b136d996-244f-4615-8bb4-292a8ff162ff None admin] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:35:32.965625 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 8/16] 10.222.0.20 () {58 vars in 1282 bytes} [Mon Aug 1 16:35:32 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 01 16:35:32.971439 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf0284b3-a17a-4148-93a6-e98a6bbcea32 None admin] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:35:32.974694 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG dbcounter [-] [87057] Writing DB stats keystone:SELECT=100 {{(pid=87057) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:35:33.068646 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71948d66-a00e-40cb-9baf-4607a5e5979d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:35:33.069436 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71948d66-a00e-40cb-9baf-4607a5e5979d None admin] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:35:33.069605 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71948d66-a00e-40cb-9baf-4607a5e5979d None admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:35:33.069879 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71948d66-a00e-40cb-9baf-4607a5e5979d None admin] PATH_INFO: `/v3/users/1007f8e379b74e169b98fb7c3c0ba5b4` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:35:33.071087 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71948d66-a00e-40cb-9baf-4607a5e5979d None admin] RBAC: Authorizing `identity:get_user(user_id=1007f8e379b74e169b98fb7c3c0ba5b4)` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:35:33.073458 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71948d66-a00e-40cb-9baf-4607a5e5979d None admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:35:33.075092 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 9/17] 10.222.0.20 () {56 vars in 1177 bytes} [Mon Aug 1 16:35:32 2022] GET /identity/v3/users/1007f8e379b74e169b98fb7c3c0ba5b4 => generated 247 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 16:35:33.076240 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf0284b3-a17a-4148-93a6-e98a6bbcea32 None admin] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:35:33.080553 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71948d66-a00e-40cb-9baf-4607a5e5979d None admin] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:35:33.095729 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG dbcounter [-] [87058] Writing DB stats keystone:SELECT=100 {{(pid=87058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:35:33.110208 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf85013-7115-457d-97e1-8b3505cf2bbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:35:33.111072 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf85013-7115-457d-97e1-8b3505cf2bbe None admin] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:35:33.111349 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf85013-7115-457d-97e1-8b3505cf2bbe None admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:35:33.111620 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf85013-7115-457d-97e1-8b3505cf2bbe None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:35:33.129976 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bf85013-7115-457d-97e1-8b3505cf2bbe None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:35:33.131474 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bf85013-7115-457d-97e1-8b3505cf2bbe None admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:35:33.150707 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application [None req-1bf85013-7115-457d-97e1-8b3505cf2bbe None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 16:35:33.150707 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 16:35:33.150707 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 16:35:33.150707 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 16:35:33.150707 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 16:35:33.150707 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 16:35:33.150707 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 16:35:33.150707 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 16:35:33.150707 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 16:35:33.150707 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 16:35:33.150707 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 16:35:33.150707 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 16:35:33.150707 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 16:35:33.150707 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 16:35:33.150707 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 16:35:33.151594 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 16:35:33.151594 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 16:35:33.151594 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 16:35:33.151594 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 16:35:33.151594 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 16:35:33.151594 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 16:35:33.151594 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 16:35:33.151594 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 16:35:33.151594 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application with Lock( Aug 01 16:35:33.151594 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 16:35:33.151594 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self._enter() Aug 01 16:35:33.151594 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 16:35:33.151594 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 16:35:33.151594 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 16:35:33.151594 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self.creator() Aug 01 16:35:33.151594 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 16:35:33.151594 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application created_value = creator( Aug 01 16:35:33.152388 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 16:35:33.152388 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 16:35:33.152388 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 16:35:33.152388 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application Aug 01 16:35:33.152388 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 9/18] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Aug 1 16:35:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 01 16:35:33.157689 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf85013-7115-457d-97e1-8b3505cf2bbe None admin] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:35:33.184305 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71948d66-a00e-40cb-9baf-4607a5e5979d None admin] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:35:33.214733 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG dbcounter [-] [87057] Writing DB stats keystone:SELECT=100 {{(pid=87057) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:35:33.218168 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-873b1b57-e0d8-4e11-a95f-60650fcd3395 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:35:33.218817 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873b1b57-e0d8-4e11-a95f-60650fcd3395 None admin] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:35:33.219097 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873b1b57-e0d8-4e11-a95f-60650fcd3395 None admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:35:33.219371 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873b1b57-e0d8-4e11-a95f-60650fcd3395 None admin] PATH_INFO: `/v3/domains/default` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:35:33.220535 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-873b1b57-e0d8-4e11-a95f-60650fcd3395 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:35:33.222089 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-873b1b57-e0d8-4e11-a95f-60650fcd3395 None admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:35:33.227143 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 10/19] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Aug 1 16:35:33 2022] GET /identity/v3/domains/default => generated 199 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 16:35:33.229764 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-873b1b57-e0d8-4e11-a95f-60650fcd3395 None admin] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:35:33.259517 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf85013-7115-457d-97e1-8b3505cf2bbe None admin] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:35:33.292529 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c34271d-bbba-4917-b39f-7c849e2f5a89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:35:33.293147 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c34271d-bbba-4917-b39f-7c849e2f5a89 None admin] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:35:33.293426 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c34271d-bbba-4917-b39f-7c849e2f5a89 None admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:35:33.293699 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c34271d-bbba-4917-b39f-7c849e2f5a89 None admin] PATH_INFO: `/v3/domains` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:35:33.294421 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c34271d-bbba-4917-b39f-7c849e2f5a89 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87057) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:35:33.294722 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c34271d-bbba-4917-b39f-7c849e2f5a89 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:35:33.296118 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c34271d-bbba-4917-b39f-7c849e2f5a89 None admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:35:33.309969 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG dbcounter [-] [87058] Writing DB stats keystone:SELECT=100 {{(pid=87058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:35:33.313740 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 10/20] 10.222.0.20 () {56 vars in 1109 bytes} [Mon Aug 1 16:35:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 16:35:33.329771 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-873b1b57-e0d8-4e11-a95f-60650fcd3395 None admin] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:35:33.367582 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2996648-924d-4a94-b9a7-90edef334b98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:35:33.368857 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2996648-924d-4a94-b9a7-90edef334b98 None admin] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:35:33.369147 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2996648-924d-4a94-b9a7-90edef334b98 None admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:35:33.369424 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2996648-924d-4a94-b9a7-90edef334b98 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:35:33.370274 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2996648-924d-4a94-b9a7-90edef334b98 None admin] RBAC: Adding query filter params (role.id=440bb943dd5b46fea038b903b6eaaf81, scope.domain.id=default, user.id=1007f8e379b74e169b98fb7c3c0ba5b4) {{(pid=87058) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:35:33.370587 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2996648-924d-4a94-b9a7-90edef334b98 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:35:33.372081 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2996648-924d-4a94-b9a7-90edef334b98 None admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:35:33.379906 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 11/21] 10.222.0.20 () {56 vars in 1322 bytes} [Mon Aug 1 16:35:33 2022] GET /identity/v3/role_assignments?user.id=1007f8e379b74e169b98fb7c3c0ba5b4&scope.domain.id=default&role.id=440bb943dd5b46fea038b903b6eaaf81 => generated 233 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 16:35:35.423413 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e08022-11e2-424c-a3e4-2717878bf0d0 None None] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:35:35.424451 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e08022-11e2-424c-a3e4-2717878bf0d0 None None] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:35:35.424940 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e08022-11e2-424c-a3e4-2717878bf0d0 None None] PATH_INFO: `/` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:35:35.426782 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 11/22] 10.222.0.20 () {54 vars in 926 bytes} [Mon Aug 1 16:35:35 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 16:35:35.433200 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf52882-43eb-4cc1-a277-7d942e298bcc None None] REQUEST_METHOD: `POST` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:35:35.434094 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf52882-43eb-4cc1-a277-7d942e298bcc None None] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:35:35.434582 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf52882-43eb-4cc1-a277-7d942e298bcc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:35:35.454069 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.auth.core [None req-8cf52882-43eb-4cc1-a277-7d942e298bcc None None] MFA Rules not processed for user `1007f8e379b74e169b98fb7c3c0ba5b4`. Rule list: `[]` (Enabled: `True`). {{(pid=87058) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:35:35.455571 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.fernet_utils [None req-8cf52882-43eb-4cc1-a277-7d942e298bcc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:35:35.550154 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 12/23] 10.222.0.20 () {58 vars in 1023 bytes} [Mon Aug 1 16:35:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 16:35:35.595483 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e08022-11e2-424c-a3e4-2717878bf0d0 None None] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:35:35.739035 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e08022-11e2-424c-a3e4-2717878bf0d0 None None] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:35:35.760354 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG dbcounter [-] [87057] Writing DB stats keystone:SELECT=100 {{(pid=87057) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:35:35.784684 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0344336c-cd2a-4fc5-9695-82e08ad203fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:35:35.785335 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0344336c-cd2a-4fc5-9695-82e08ad203fa None admin] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:35:35.785614 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0344336c-cd2a-4fc5-9695-82e08ad203fa None admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:35:35.788807 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0344336c-cd2a-4fc5-9695-82e08ad203fa None admin] PATH_INFO: `/v3/roles/admin` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:35:35.802920 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0344336c-cd2a-4fc5-9695-82e08ad203fa None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 16:35:35.811578 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application [None req-0344336c-cd2a-4fc5-9695-82e08ad203fa None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 16:35:35.811578 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 16:35:35.811578 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 16:35:35.811578 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 16:35:35.811578 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 16:35:35.811578 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 16:35:35.811578 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 16:35:35.811578 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 16:35:35.811578 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 16:35:35.811578 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 16:35:35.811578 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 16:35:35.811578 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 16:35:35.811578 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 16:35:35.811578 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 16:35:35.811578 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 16:35:35.811578 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 16:35:35.812436 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 16:35:35.812436 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 16:35:35.812436 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 16:35:35.812436 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 16:35:35.812436 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 16:35:35.812436 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 16:35:35.812436 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 16:35:35.812436 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 16:35:35.812436 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 16:35:35.812436 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 16:35:35.812436 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 16:35:35.812436 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application with Lock( Aug 01 16:35:35.812436 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 16:35:35.812436 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self._enter() Aug 01 16:35:35.812436 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 16:35:35.812436 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 16:35:35.813671 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 16:35:35.813671 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self.creator() Aug 01 16:35:35.813671 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 16:35:35.813671 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application created_value = creator( Aug 01 16:35:35.813671 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 16:35:35.813671 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 16:35:35.813671 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 16:35:35.813671 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 16:35:35.813671 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 16:35:35.813671 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 16:35:35.813671 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 16:35:35.813671 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application Aug 01 16:35:35.813671 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 12/24] 10.222.0.20 () {56 vars in 1096 bytes} [Mon Aug 1 16:35:35 2022] GET /identity/v3/roles/admin => generated 83 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 16:35:35.818613 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cf52882-43eb-4cc1-a277-7d942e298bcc None None] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:35:35.910051 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG dbcounter [-] [87058] Writing DB stats keystone:SELECT=100 {{(pid=87058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:35:35.935153 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cf52882-43eb-4cc1-a277-7d942e298bcc None None] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:35:35.983937 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd079d67-b723-411d-8758-6dc07b0c2743 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:35:35.984570 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd079d67-b723-411d-8758-6dc07b0c2743 None admin] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:35:35.984846 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd079d67-b723-411d-8758-6dc07b0c2743 None admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:35:35.985140 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd079d67-b723-411d-8758-6dc07b0c2743 None admin] PATH_INFO: `/v3/roles` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:35:35.985834 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd079d67-b723-411d-8758-6dc07b0c2743 None admin] RBAC: Adding query filter params (name=admin) {{(pid=87058) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:35:35.986155 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd079d67-b723-411d-8758-6dc07b0c2743 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:35:35.987543 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd079d67-b723-411d-8758-6dc07b0c2743 None admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:35:36.001385 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 13/25] 10.222.0.20 () {56 vars in 1099 bytes} [Mon Aug 1 16:35:35 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:35:36.007001 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0344336c-cd2a-4fc5-9695-82e08ad203fa None admin] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:35:36.048406 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0b520a-9f25-473b-8565-28b70ecebb26 None None] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:35:36.048722 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0b520a-9f25-473b-8565-28b70ecebb26 None None] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:35:36.048962 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0b520a-9f25-473b-8565-28b70ecebb26 None None] PATH_INFO: `/` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:35:36.050407 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 14/26] 10.222.0.20 () {54 vars in 926 bytes} [Mon Aug 1 16:35:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 16:35:36.064452 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d38f19-95b3-4640-93ad-fef865e3a8b9 None None] REQUEST_METHOD: `POST` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:35:36.065064 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d38f19-95b3-4640-93ad-fef865e3a8b9 None None] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:35:36.065535 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d38f19-95b3-4640-93ad-fef865e3a8b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:35:36.112685 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.auth.core [None req-b3d38f19-95b3-4640-93ad-fef865e3a8b9 None None] MFA Rules not processed for user `1007f8e379b74e169b98fb7c3c0ba5b4`. Rule list: `[]` (Enabled: `True`). {{(pid=87058) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:35:36.114320 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.fernet_utils [None req-b3d38f19-95b3-4640-93ad-fef865e3a8b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:35:36.299387 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 15/27] 10.222.0.20 () {58 vars in 1023 bytes} [Mon Aug 1 16:35:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 236 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 01 16:35:36.319755 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c615bb0e-a372-407e-8588-5120fd9b3e37 None None] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:35:36.320340 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c615bb0e-a372-407e-8588-5120fd9b3e37 None None] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:35:36.320809 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c615bb0e-a372-407e-8588-5120fd9b3e37 None None] PATH_INFO: `/` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:35:36.322557 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 16/28] 10.222.0.20 () {54 vars in 926 bytes} [Mon Aug 1 16:35:36 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 01 16:35:36.327839 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce2103a-8890-48c5-86e7-1a2197beda3e None None] REQUEST_METHOD: `POST` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:35:36.328353 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce2103a-8890-48c5-86e7-1a2197beda3e None None] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:35:36.328817 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce2103a-8890-48c5-86e7-1a2197beda3e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:35:36.332782 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0344336c-cd2a-4fc5-9695-82e08ad203fa None admin] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:35:36.349989 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.auth.core [None req-5ce2103a-8890-48c5-86e7-1a2197beda3e None None] MFA Rules not processed for user `1007f8e379b74e169b98fb7c3c0ba5b4`. Rule list: `[]` (Enabled: `True`). {{(pid=87058) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:35:36.353295 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.fernet_utils [None req-5ce2103a-8890-48c5-86e7-1a2197beda3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:35:36.394927 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b212aa-5414-48e6-86b4-7676129d576a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:35:36.395910 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b212aa-5414-48e6-86b4-7676129d576a None admin] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:35:36.396495 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b212aa-5414-48e6-86b4-7676129d576a None admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:35:36.397948 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b212aa-5414-48e6-86b4-7676129d576a None admin] PATH_INFO: `/v3/users/1007f8e379b74e169b98fb7c3c0ba5b4` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:35:36.400812 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3b212aa-5414-48e6-86b4-7676129d576a None admin] RBAC: Authorizing `identity:get_user(user_id=1007f8e379b74e169b98fb7c3c0ba5b4)` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:35:36.403337 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3b212aa-5414-48e6-86b4-7676129d576a None admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:35:36.406446 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 13/29] 10.222.0.20 () {56 vars in 1177 bytes} [Mon Aug 1 16:35:36 2022] GET /identity/v3/users/1007f8e379b74e169b98fb7c3c0ba5b4 => generated 247 bytes in 400 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:35:36.412278 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b212aa-5414-48e6-86b4-7676129d576a None admin] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:35:36.493773 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG dbcounter [-] [87058] Writing DB stats keystone:SELECT=100 {{(pid=87058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:35:36.495200 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG dbcounter [-] [87057] Writing DB stats keystone:SELECT=100 {{(pid=87057) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:35:36.504537 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 17/30] 10.222.0.20 () {58 vars in 1023 bytes} [Mon Aug 1 16:35:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 16:35:36.505839 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce2103a-8890-48c5-86e7-1a2197beda3e None None] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:35:36.558863 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b212aa-5414-48e6-86b4-7676129d576a None admin] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:35:36.608661 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-163a62ba-9557-423a-8d55-d0618d06d20c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:35:36.609428 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163a62ba-9557-423a-8d55-d0618d06d20c None admin] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:35:36.609775 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163a62ba-9557-423a-8d55-d0618d06d20c None admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:35:36.610243 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163a62ba-9557-423a-8d55-d0618d06d20c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:35:36.622370 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-163a62ba-9557-423a-8d55-d0618d06d20c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:35:36.624155 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-163a62ba-9557-423a-8d55-d0618d06d20c None admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:35:36.641212 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application [None req-163a62ba-9557-423a-8d55-d0618d06d20c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 16:35:36.641212 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 16:35:36.641212 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 16:35:36.641212 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 16:35:36.641212 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 16:35:36.641212 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 16:35:36.641212 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 16:35:36.641212 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 16:35:36.641212 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 16:35:36.641212 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 16:35:36.641212 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 16:35:36.641212 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 16:35:36.641212 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 16:35:36.641212 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 16:35:36.641212 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 16:35:36.642271 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 16:35:36.642271 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 16:35:36.642271 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 16:35:36.642271 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 16:35:36.642271 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 16:35:36.642271 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 16:35:36.642271 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 16:35:36.642271 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 16:35:36.642271 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application with Lock( Aug 01 16:35:36.642271 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 16:35:36.642271 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self._enter() Aug 01 16:35:36.642271 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 16:35:36.642271 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 16:35:36.642271 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 16:35:36.642271 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self.creator() Aug 01 16:35:36.642271 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 16:35:36.642271 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application created_value = creator( Aug 01 16:35:36.643570 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 16:35:36.643570 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 16:35:36.643570 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 16:35:36.643570 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application Aug 01 16:35:36.645202 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 14/31] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Aug 1 16:35:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 237 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 16:35:36.650650 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce2103a-8890-48c5-86e7-1a2197beda3e None None] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:35:36.652133 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8fb0f55-63c9-4e21-88f4-d578f8e28965 None None] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:35:36.652582 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8fb0f55-63c9-4e21-88f4-d578f8e28965 None None] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:35:36.653001 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8fb0f55-63c9-4e21-88f4-d578f8e28965 None None] PATH_INFO: `/` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:35:36.654816 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 15/32] 10.222.0.20 () {54 vars in 926 bytes} [Mon Aug 1 16:35:36 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 16:35:36.655668 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8fb0f55-63c9-4e21-88f4-d578f8e28965 None None] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:35:36.687755 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc8c2ed-94a8-4594-a20d-93623b12176b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:35:36.688561 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc8c2ed-94a8-4594-a20d-93623b12176b None admin] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:35:36.689025 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc8c2ed-94a8-4594-a20d-93623b12176b None admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:35:36.689383 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc8c2ed-94a8-4594-a20d-93623b12176b None admin] PATH_INFO: `/v3/domains/default` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:35:36.690708 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bc8c2ed-94a8-4594-a20d-93623b12176b None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:35:36.692524 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bc8c2ed-94a8-4594-a20d-93623b12176b None admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:35:36.694540 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 18/33] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Aug 1 16:35:36 2022] GET /identity/v3/domains/default => generated 199 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:35:36.695621 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc8c2ed-94a8-4594-a20d-93623b12176b None admin] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:35:36.764855 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG dbcounter [-] [87058] Writing DB stats keystone:SELECT=100 {{(pid=87058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:35:36.771449 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG dbcounter [-] [87057] Writing DB stats keystone:SELECT=100 {{(pid=87057) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:35:36.779136 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8fb0f55-63c9-4e21-88f4-d578f8e28965 None None] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:35:36.810528 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc8c2ed-94a8-4594-a20d-93623b12176b None admin] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:35:36.817389 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5bf499a-a086-44b8-9971-62d0eb006202 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:35:36.818146 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5bf499a-a086-44b8-9971-62d0eb006202 None admin] REQUEST_METHOD: `POST` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:35:36.818441 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5bf499a-a086-44b8-9971-62d0eb006202 None admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:35:36.818720 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5bf499a-a086-44b8-9971-62d0eb006202 None admin] PATH_INFO: `/v3/roles` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:35:36.819508 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5bf499a-a086-44b8-9971-62d0eb006202 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:35:36.821078 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5bf499a-a086-44b8-9971-62d0eb006202 None admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:35:36.850348 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef7ab16-bc5e-4b21-b23e-f80afceafdae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:35:36.851051 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef7ab16-bc5e-4b21-b23e-f80afceafdae None admin] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:35:36.853361 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef7ab16-bc5e-4b21-b23e-f80afceafdae None admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:35:36.853361 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef7ab16-bc5e-4b21-b23e-f80afceafdae None admin] PATH_INFO: `/v3/domains` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:35:36.853361 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ef7ab16-bc5e-4b21-b23e-f80afceafdae None admin] RBAC: Adding query filter params (name=Default) {{(pid=87058) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:35:36.853361 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ef7ab16-bc5e-4b21-b23e-f80afceafdae None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:35:36.854789 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ef7ab16-bc5e-4b21-b23e-f80afceafdae None admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:35:36.877048 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 19/34] 10.222.0.20 () {56 vars in 1109 bytes} [Mon Aug 1 16:35:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:35:36.877048 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38f0847-9039-4f40-bff1-773145fba3ab None None] REQUEST_METHOD: `POST` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:35:36.877048 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38f0847-9039-4f40-bff1-773145fba3ab None None] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:35:36.877048 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38f0847-9039-4f40-bff1-773145fba3ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:35:36.894094 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.auth.core [None req-f38f0847-9039-4f40-bff1-773145fba3ab None None] MFA Rules not processed for user `1007f8e379b74e169b98fb7c3c0ba5b4`. Rule list: `[]` (Enabled: `True`). {{(pid=87058) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:35:36.895312 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.fernet_utils [None req-f38f0847-9039-4f40-bff1-773145fba3ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:35:36.979438 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 20/35] 10.222.0.20 () {58 vars in 1023 bytes} [Mon Aug 1 16:35:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 16:35:36.980597 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f38f0847-9039-4f40-bff1-773145fba3ab None None] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:35:37.051238 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG dbcounter [-] [87058] Writing DB stats keystone:SELECT=100 {{(pid=87058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:35:37.078154 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 16/36] 10.222.0.20 () {60 vars in 1131 bytes} [Mon Aug 1 16:35:36 2022] POST /identity/v3/roles => generated 219 bytes in 415 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 16:35:37.078154 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5bf499a-a086-44b8-9971-62d0eb006202 None admin] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:35:37.111702 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f38f0847-9039-4f40-bff1-773145fba3ab None None] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:35:37.164287 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb9fa90-e35b-4e84-ad28-d6ab7a093007 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:35:37.165535 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb9fa90-e35b-4e84-ad28-d6ab7a093007 None admin] REQUEST_METHOD: `PUT` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:35:37.165828 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb9fa90-e35b-4e84-ad28-d6ab7a093007 None admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:35:37.166343 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb9fa90-e35b-4e84-ad28-d6ab7a093007 None admin] PATH_INFO: `/v3/domains/default/users/1007f8e379b74e169b98fb7c3c0ba5b4/roles/440bb943dd5b46fea038b903b6eaaf81` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:35:37.169474 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cb9fa90-e35b-4e84-ad28-d6ab7a093007 None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=1007f8e379b74e169b98fb7c3c0ba5b4, role_id=440bb943dd5b46fea038b903b6eaaf81)` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:35:37.171572 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cb9fa90-e35b-4e84-ad28-d6ab7a093007 None admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:35:37.198221 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5bf499a-a086-44b8-9971-62d0eb006202 None admin] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:35:37.235545 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c339db7-1f36-4791-877b-6080013b0f42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:35:37.236183 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c339db7-1f36-4791-877b-6080013b0f42 None admin] REQUEST_METHOD: `POST` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:35:37.236469 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c339db7-1f36-4791-877b-6080013b0f42 None admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:35:37.236744 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c339db7-1f36-4791-877b-6080013b0f42 None admin] PATH_INFO: `/v3/projects` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:35:37.237442 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c339db7-1f36-4791-877b-6080013b0f42 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:35:37.239064 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c339db7-1f36-4791-877b-6080013b0f42 None admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:35:37.320226 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 17/37] 10.222.0.20 () {60 vars in 1141 bytes} [Mon Aug 1 16:35:37 2022] POST /identity/v3/projects => generated 301 bytes in 249 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 16:35:37.321054 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c339db7-1f36-4791-877b-6080013b0f42 None admin] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:35:37.335409 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG dbcounter [-] [87057] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=87057) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:35:37.374159 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 21/38] 10.222.0.20 () {58 vars in 1361 bytes} [Mon Aug 1 16:35:36 2022] PUT /identity/v3/domains/default/users/1007f8e379b74e169b98fb7c3c0ba5b4/roles/440bb943dd5b46fea038b903b6eaaf81 => generated 0 bytes in 387 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 16:35:37.374159 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c17c4d-bba6-4418-924f-9fc608242612 None None] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:35:37.374159 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c17c4d-bba6-4418-924f-9fc608242612 None None] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:35:37.374159 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c17c4d-bba6-4418-924f-9fc608242612 None None] PATH_INFO: `/` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:35:37.374829 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 22/39] 10.222.0.20 () {54 vars in 926 bytes} [Mon Aug 1 16:35:37 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 16:35:37.379176 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a6f265-6d50-471e-b3c6-498af89caa7f None None] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:35:37.379176 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a6f265-6d50-471e-b3c6-498af89caa7f None None] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:35:37.379176 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a6f265-6d50-471e-b3c6-498af89caa7f None None] PATH_INFO: `/` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:35:37.379176 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 23/40] 10.222.0.20 () {54 vars in 926 bytes} [Mon Aug 1 16:35:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 16:35:37.381743 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7000795f-8f89-49b0-b989-8ea2e9313561 None None] REQUEST_METHOD: `POST` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:35:37.381743 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7000795f-8f89-49b0-b989-8ea2e9313561 None None] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:35:37.381743 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7000795f-8f89-49b0-b989-8ea2e9313561 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:35:37.403082 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.auth.core [None req-7000795f-8f89-49b0-b989-8ea2e9313561 None None] MFA Rules not processed for user `1007f8e379b74e169b98fb7c3c0ba5b4`. Rule list: `[]` (Enabled: `True`). {{(pid=87058) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:35:37.404724 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.fernet_utils [None req-7000795f-8f89-49b0-b989-8ea2e9313561 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:35:37.479301 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c339db7-1f36-4791-877b-6080013b0f42 None admin] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:35:37.486443 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 24/41] 10.222.0.20 () {58 vars in 1023 bytes} [Mon Aug 1 16:35:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 16:35:37.487917 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe7e5f9-c6f5-4e3f-a6d6-a471a1075578 None None] REQUEST_METHOD: `POST` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:35:37.488208 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe7e5f9-c6f5-4e3f-a6d6-a471a1075578 None None] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:35:37.488485 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe7e5f9-c6f5-4e3f-a6d6-a471a1075578 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:35:37.510970 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.auth.core [None req-8fe7e5f9-c6f5-4e3f-a6d6-a471a1075578 None None] MFA Rules not processed for user `1007f8e379b74e169b98fb7c3c0ba5b4`. Rule list: `[]` (Enabled: `True`). {{(pid=87058) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:35:37.512261 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.fernet_utils [None req-8fe7e5f9-c6f5-4e3f-a6d6-a471a1075578 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:35:37.541905 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23995ce8-729a-4462-abf3-535f18d033c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:35:37.542590 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23995ce8-729a-4462-abf3-535f18d033c9 None admin] REQUEST_METHOD: `POST` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:35:37.542863 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23995ce8-729a-4462-abf3-535f18d033c9 None admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:35:37.543253 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23995ce8-729a-4462-abf3-535f18d033c9 None admin] PATH_INFO: `/v3/roles` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:35:37.544100 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23995ce8-729a-4462-abf3-535f18d033c9 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:35:37.545598 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23995ce8-729a-4462-abf3-535f18d033c9 None admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:35:37.566643 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG dbcounter [-] [87058] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:35:37.592937 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 25/42] 10.222.0.20 () {58 vars in 1023 bytes} [Mon Aug 1 16:35:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 16:35:37.593723 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fe7e5f9-c6f5-4e3f-a6d6-a471a1075578 None None] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:35:37.617714 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 18/43] 10.222.0.20 () {60 vars in 1131 bytes} [Mon Aug 1 16:35:37 2022] POST /identity/v3/roles => generated 225 bytes in 297 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 16:35:37.618697 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23995ce8-729a-4462-abf3-535f18d033c9 None admin] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:35:37.699856 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG dbcounter [-] [87057] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87057) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:35:37.718630 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fe7e5f9-c6f5-4e3f-a6d6-a471a1075578 None None] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:35:37.742500 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23995ce8-729a-4462-abf3-535f18d033c9 None admin] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:35:37.760600 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ee48ef-7b35-4bde-8bdd-406fb294b65b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:35:37.761275 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ee48ef-7b35-4bde-8bdd-406fb294b65b None admin] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:35:37.761558 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ee48ef-7b35-4bde-8bdd-406fb294b65b None admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:35:37.761834 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ee48ef-7b35-4bde-8bdd-406fb294b65b None admin] PATH_INFO: `/v3/domains/default` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:35:37.763158 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57ee48ef-7b35-4bde-8bdd-406fb294b65b None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:35:37.764788 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57ee48ef-7b35-4bde-8bdd-406fb294b65b None admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:35:37.766791 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 26/44] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Aug 1 16:35:37 2022] GET /identity/v3/domains/default => generated 199 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:35:37.772846 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ee48ef-7b35-4bde-8bdd-406fb294b65b None admin] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:35:37.783692 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0196c4e0-d237-4615-b8d2-19687c7e31db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:35:37.784337 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0196c4e0-d237-4615-b8d2-19687c7e31db None admin] REQUEST_METHOD: `POST` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:35:37.784623 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0196c4e0-d237-4615-b8d2-19687c7e31db None admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:35:37.784902 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0196c4e0-d237-4615-b8d2-19687c7e31db None admin] PATH_INFO: `/v3/roles` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:35:37.785631 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0196c4e0-d237-4615-b8d2-19687c7e31db None admin] RBAC: Authorizing `identity:create_role()` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:35:37.787222 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0196c4e0-d237-4615-b8d2-19687c7e31db None admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:35:37.826125 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG dbcounter [-] [87058] Writing DB stats keystone:SELECT=100 {{(pid=87058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:35:37.884432 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ee48ef-7b35-4bde-8bdd-406fb294b65b None admin] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:35:37.899080 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 19/45] 10.222.0.20 () {60 vars in 1131 bytes} [Mon Aug 1 16:35:37 2022] POST /identity/v3/roles => generated 223 bytes in 279 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 16:35:37.934110 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce23694d-1a97-49bf-9dad-eb6365aac3d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:35:37.934771 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce23694d-1a97-49bf-9dad-eb6365aac3d9 None admin] REQUEST_METHOD: `POST` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:35:37.935079 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce23694d-1a97-49bf-9dad-eb6365aac3d9 None admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:35:37.935333 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce23694d-1a97-49bf-9dad-eb6365aac3d9 None admin] PATH_INFO: `/v3/projects` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:35:37.936130 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce23694d-1a97-49bf-9dad-eb6365aac3d9 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:35:37.937681 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce23694d-1a97-49bf-9dad-eb6365aac3d9 None admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:35:38.071572 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 27/46] 10.222.0.20 () {60 vars in 1141 bytes} [Mon Aug 1 16:35:37 2022] POST /identity/v3/projects => generated 312 bytes in 299 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 16:35:39.455744 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089d0c6e-c58d-4e8a-a958-61126cdc8b7b None None] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:35:39.456263 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089d0c6e-c58d-4e8a-a958-61126cdc8b7b None None] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:35:39.456368 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089d0c6e-c58d-4e8a-a958-61126cdc8b7b None None] PATH_INFO: `/` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:35:39.457711 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 20/47] 10.222.0.20 () {54 vars in 926 bytes} [Mon Aug 1 16:35:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 16:35:39.463823 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ca997d-6bc8-4da7-aba7-d0b065a3bcbe None None] REQUEST_METHOD: `POST` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:35:39.464561 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ca997d-6bc8-4da7-aba7-d0b065a3bcbe None None] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:35:39.464849 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ca997d-6bc8-4da7-aba7-d0b065a3bcbe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:35:39.482910 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.auth.core [None req-a0ca997d-6bc8-4da7-aba7-d0b065a3bcbe None None] MFA Rules not processed for user `1007f8e379b74e169b98fb7c3c0ba5b4`. Rule list: `[]` (Enabled: `True`). {{(pid=87058) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:35:39.484146 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.fernet_utils [None req-a0ca997d-6bc8-4da7-aba7-d0b065a3bcbe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:35:39.550327 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 28/48] 10.222.0.20 () {58 vars in 1023 bytes} [Mon Aug 1 16:35:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 16:35:39.573523 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-089d0c6e-c58d-4e8a-a958-61126cdc8b7b None None] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:35:39.674449 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-089d0c6e-c58d-4e8a-a958-61126cdc8b7b None None] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:35:39.701497 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG dbcounter [-] [87057] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87057) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:35:39.709078 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7d7d0ed-ec7f-44c6-bc92-330117fc587e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:35:39.709674 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d7d0ed-ec7f-44c6-bc92-330117fc587e None admin] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:35:39.709960 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d7d0ed-ec7f-44c6-bc92-330117fc587e None admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:35:39.710270 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d7d0ed-ec7f-44c6-bc92-330117fc587e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:35:39.720843 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7d7d0ed-ec7f-44c6-bc92-330117fc587e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 16:35:39.725915 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application [None req-b7d7d0ed-ec7f-44c6-bc92-330117fc587e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 16:35:39.725915 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 16:35:39.725915 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 16:35:39.725915 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 16:35:39.725915 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 16:35:39.725915 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 16:35:39.725915 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 16:35:39.725915 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 16:35:39.725915 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 16:35:39.725915 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 16:35:39.725915 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 16:35:39.725915 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 16:35:39.725915 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 16:35:39.725915 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 16:35:39.725915 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 16:35:39.725915 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 16:35:39.726799 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 16:35:39.726799 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 16:35:39.726799 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 16:35:39.726799 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 16:35:39.726799 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 16:35:39.726799 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 16:35:39.726799 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 16:35:39.726799 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 16:35:39.726799 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 16:35:39.726799 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 16:35:39.726799 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 16:35:39.726799 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application with Lock( Aug 01 16:35:39.726799 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 16:35:39.726799 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self._enter() Aug 01 16:35:39.726799 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 16:35:39.726799 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 16:35:39.728207 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 16:35:39.728207 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self.creator() Aug 01 16:35:39.728207 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 16:35:39.728207 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application created_value = creator( Aug 01 16:35:39.728207 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 16:35:39.728207 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 16:35:39.728207 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 16:35:39.728207 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 16:35:39.728207 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 16:35:39.728207 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 16:35:39.728207 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 16:35:39.728207 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application Aug 01 16:35:39.728207 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 21/49] 10.222.0.20 () {56 vars in 1096 bytes} [Mon Aug 1 16:35:39 2022] GET /identity/v3/roles/admin => generated 83 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 16:35:39.733570 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0ca997d-6bc8-4da7-aba7-d0b065a3bcbe None None] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:35:39.759469 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG dbcounter [-] [87058] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:35:39.832221 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0ca997d-6bc8-4da7-aba7-d0b065a3bcbe None None] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:35:39.865841 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-341be8b1-025e-469c-bcb4-6d64b420cdb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:35:39.866502 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341be8b1-025e-469c-bcb4-6d64b420cdb9 None admin] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:35:39.866788 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341be8b1-025e-469c-bcb4-6d64b420cdb9 None admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:35:39.867066 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341be8b1-025e-469c-bcb4-6d64b420cdb9 None admin] PATH_INFO: `/v3/roles` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:35:39.867795 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-341be8b1-025e-469c-bcb4-6d64b420cdb9 None admin] RBAC: Adding query filter params (name=admin) {{(pid=87058) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:35:39.868653 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-341be8b1-025e-469c-bcb4-6d64b420cdb9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:35:39.870060 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-341be8b1-025e-469c-bcb4-6d64b420cdb9 None admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:35:39.878002 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 29/50] 10.222.0.20 () {56 vars in 1099 bytes} [Mon Aug 1 16:35:39 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:35:39.884843 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7d7d0ed-ec7f-44c6-bc92-330117fc587e None admin] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:35:39.985130 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7d7d0ed-ec7f-44c6-bc92-330117fc587e None admin] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:35:40.019279 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4842b813-909b-4206-9ceb-881a2259dc43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:35:40.019925 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4842b813-909b-4206-9ceb-881a2259dc43 None admin] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:35:40.020207 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4842b813-909b-4206-9ceb-881a2259dc43 None admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:35:40.020485 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4842b813-909b-4206-9ceb-881a2259dc43 None admin] PATH_INFO: `/v3/users/1007f8e379b74e169b98fb7c3c0ba5b4` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:35:40.021648 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4842b813-909b-4206-9ceb-881a2259dc43 None admin] RBAC: Authorizing `identity:get_user(user_id=1007f8e379b74e169b98fb7c3c0ba5b4)` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:35:40.023151 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4842b813-909b-4206-9ceb-881a2259dc43 None admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:35:40.024853 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 22/51] 10.222.0.20 () {56 vars in 1177 bytes} [Mon Aug 1 16:35:39 2022] GET /identity/v3/users/1007f8e379b74e169b98fb7c3c0ba5b4 => generated 247 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:35:40.030438 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-341be8b1-025e-469c-bcb4-6d64b420cdb9 None admin] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:35:40.064832 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc445786-6507-482a-ad57-9c2d32ca54aa None None] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:35:40.065120 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc445786-6507-482a-ad57-9c2d32ca54aa None None] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:35:40.065402 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc445786-6507-482a-ad57-9c2d32ca54aa None None] PATH_INFO: `/` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:35:40.066801 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 23/52] 10.222.0.20 () {54 vars in 926 bytes} [Mon Aug 1 16:35:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 01 16:35:40.072222 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c6a658-3fbf-426e-88b2-a21bc797c412 None None] REQUEST_METHOD: `POST` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:35:40.072546 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c6a658-3fbf-426e-88b2-a21bc797c412 None None] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:35:40.072826 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c6a658-3fbf-426e-88b2-a21bc797c412 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:35:40.091312 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.auth.core [None req-25c6a658-3fbf-426e-88b2-a21bc797c412 None None] MFA Rules not processed for user `1007f8e379b74e169b98fb7c3c0ba5b4`. Rule list: `[]` (Enabled: `True`). {{(pid=87057) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:35:40.092841 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.fernet_utils [None req-25c6a658-3fbf-426e-88b2-a21bc797c412 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87057) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:35:40.115723 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG dbcounter [-] [87058] Writing DB stats keystone:SELECT=100 {{(pid=87058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:35:40.131452 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-341be8b1-025e-469c-bcb4-6d64b420cdb9 None admin] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:35:40.143371 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG dbcounter [-] [87057] Writing DB stats keystone:SELECT=100 {{(pid=87057) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:35:40.159330 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 24/53] 10.222.0.20 () {58 vars in 1023 bytes} [Mon Aug 1 16:35:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 01 16:35:40.166613 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77b0cb31-8c0f-4958-b719-4fece1abc910 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:35:40.167252 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b0cb31-8c0f-4958-b719-4fece1abc910 None admin] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:35:40.167627 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b0cb31-8c0f-4958-b719-4fece1abc910 None admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:35:40.167806 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b0cb31-8c0f-4958-b719-4fece1abc910 None admin] PATH_INFO: `/v3/domains/default` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:35:40.168949 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77b0cb31-8c0f-4958-b719-4fece1abc910 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:35:40.170469 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77b0cb31-8c0f-4958-b719-4fece1abc910 None admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:35:40.172037 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 30/54] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Aug 1 16:35:40 2022] GET /identity/v3/domains/default => generated 199 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:35:40.177959 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c6a658-3fbf-426e-88b2-a21bc797c412 None None] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:35:40.183564 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77b0cb31-8c0f-4958-b719-4fece1abc910 None admin] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:35:40.277578 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c6a658-3fbf-426e-88b2-a21bc797c412 None None] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:35:40.281349 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77b0cb31-8c0f-4958-b719-4fece1abc910 None admin] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:35:40.312446 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9268eece-3075-4b96-8471-557f85bc8205 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:35:40.313068 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9268eece-3075-4b96-8471-557f85bc8205 None admin] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:35:40.313352 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9268eece-3075-4b96-8471-557f85bc8205 None admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:35:40.313636 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9268eece-3075-4b96-8471-557f85bc8205 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:35:40.314454 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9268eece-3075-4b96-8471-557f85bc8205 None admin] RBAC: Adding query filter params (role.id=440bb943dd5b46fea038b903b6eaaf81, scope.domain.id=default, user.id=1007f8e379b74e169b98fb7c3c0ba5b4) {{(pid=87057) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:35:40.314767 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9268eece-3075-4b96-8471-557f85bc8205 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:35:40.316233 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9268eece-3075-4b96-8471-557f85bc8205 None admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:35:40.316786 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c0a7924-6000-4ae0-909d-5f9c97735222 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:35:40.317427 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0a7924-6000-4ae0-909d-5f9c97735222 None admin] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:35:40.317707 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0a7924-6000-4ae0-909d-5f9c97735222 None admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:35:40.317979 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0a7924-6000-4ae0-909d-5f9c97735222 None admin] PATH_INFO: `/v3/domains/default` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:35:40.319318 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c0a7924-6000-4ae0-909d-5f9c97735222 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:35:40.320754 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c0a7924-6000-4ae0-909d-5f9c97735222 None admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:35:40.323286 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 31/55] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Aug 1 16:35:40 2022] GET /identity/v3/domains/default => generated 199 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 16:35:40.325546 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 25/56] 10.222.0.20 () {56 vars in 1322 bytes} [Mon Aug 1 16:35:40 2022] GET /identity/v3/role_assignments?user.id=1007f8e379b74e169b98fb7c3c0ba5b4&scope.domain.id=default&role.id=440bb943dd5b46fea038b903b6eaaf81 => generated 530 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:35:40.328581 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c0a7924-6000-4ae0-909d-5f9c97735222 None admin] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:35:40.355902 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG dbcounter [-] [87058] Writing DB stats keystone:SELECT=100 {{(pid=87058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:35:40.439425 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c0a7924-6000-4ae0-909d-5f9c97735222 None admin] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:35:40.476589 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9acd6b4-576d-41d0-a90d-22666e380a11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:35:40.477201 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9acd6b4-576d-41d0-a90d-22666e380a11 None admin] REQUEST_METHOD: `POST` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:35:40.477564 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9acd6b4-576d-41d0-a90d-22666e380a11 None admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:35:40.477847 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9acd6b4-576d-41d0-a90d-22666e380a11 None admin] PATH_INFO: `/v3/projects` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:35:40.478663 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9acd6b4-576d-41d0-a90d-22666e380a11 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:35:40.480113 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9acd6b4-576d-41d0-a90d-22666e380a11 None admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:35:40.567055 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 32/57] 10.222.0.20 () {60 vars in 1140 bytes} [Mon Aug 1 16:35:40 2022] POST /identity/v3/projects => generated 298 bytes in 239 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 16:35:42.546611 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f79f0f7-9f5c-4336-8f5c-a48ab83ae09e None None] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:35:42.547157 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f79f0f7-9f5c-4336-8f5c-a48ab83ae09e None None] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:35:42.547157 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f79f0f7-9f5c-4336-8f5c-a48ab83ae09e None None] PATH_INFO: `/` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:35:42.548522 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 26/58] 10.222.0.20 () {54 vars in 926 bytes} [Mon Aug 1 16:35:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 16:35:42.553530 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56ced53-c1a8-4b3c-b3e3-acf79706e798 None None] REQUEST_METHOD: `POST` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:35:42.553940 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56ced53-c1a8-4b3c-b3e3-acf79706e798 None None] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:35:42.554101 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56ced53-c1a8-4b3c-b3e3-acf79706e798 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:35:42.572441 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.auth.core [None req-b56ced53-c1a8-4b3c-b3e3-acf79706e798 None None] MFA Rules not processed for user `1007f8e379b74e169b98fb7c3c0ba5b4`. Rule list: `[]` (Enabled: `True`). {{(pid=87058) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:35:42.573622 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.fernet_utils [None req-b56ced53-c1a8-4b3c-b3e3-acf79706e798 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:35:42.638338 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 33/59] 10.222.0.20 () {58 vars in 1023 bytes} [Mon Aug 1 16:35:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 16:35:42.661883 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f79f0f7-9f5c-4336-8f5c-a48ab83ae09e None None] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:35:42.713831 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG dbcounter [-] [87057] Writing DB stats keystone:SELECT=100 {{(pid=87057) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:35:42.762952 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f79f0f7-9f5c-4336-8f5c-a48ab83ae09e None None] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:35:42.803501 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1915e66e-8c0e-4bb1-a4e0-98fdd1fe1469 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:35:42.804069 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1915e66e-8c0e-4bb1-a4e0-98fdd1fe1469 None admin] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:35:42.804350 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1915e66e-8c0e-4bb1-a4e0-98fdd1fe1469 None admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:35:42.804625 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1915e66e-8c0e-4bb1-a4e0-98fdd1fe1469 None admin] PATH_INFO: `/v3/domains/default` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:35:42.805839 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1915e66e-8c0e-4bb1-a4e0-98fdd1fe1469 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:35:42.807331 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1915e66e-8c0e-4bb1-a4e0-98fdd1fe1469 None admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:35:42.808914 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 27/60] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Aug 1 16:35:42 2022] GET /identity/v3/domains/default => generated 199 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:35:42.814451 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b56ced53-c1a8-4b3c-b3e3-acf79706e798 None None] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:35:42.817524 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG dbcounter [-] [87058] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:35:42.912210 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b56ced53-c1a8-4b3c-b3e3-acf79706e798 None None] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:35:42.944935 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f672fd0-31b6-4d65-a70f-5fe7ac7429b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:35:42.945546 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f672fd0-31b6-4d65-a70f-5fe7ac7429b4 None admin] REQUEST_METHOD: `POST` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:35:42.945826 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f672fd0-31b6-4d65-a70f-5fe7ac7429b4 None admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:35:42.946328 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f672fd0-31b6-4d65-a70f-5fe7ac7429b4 None admin] PATH_INFO: `/v3/users` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:35:42.947009 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f672fd0-31b6-4d65-a70f-5fe7ac7429b4 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:35:42.948449 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f672fd0-31b6-4d65-a70f-5fe7ac7429b4 None admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:35:43.062946 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 34/61] 10.222.0.20 () {60 vars in 1132 bytes} [Mon Aug 1 16:35:42 2022] POST /identity/v3/users => generated 275 bytes in 249 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 16:35:46.191440 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ae54b3-7aaf-4a73-8321-282dadbf02de None None] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:35:46.192427 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ae54b3-7aaf-4a73-8321-282dadbf02de None None] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:35:46.192918 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ae54b3-7aaf-4a73-8321-282dadbf02de None None] PATH_INFO: `/` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:35:46.194792 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 28/62] 10.222.0.20 () {54 vars in 926 bytes} [Mon Aug 1 16:35:46 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 16:35:46.200547 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694c79c8-8886-45dc-8d75-342779bfae5c None None] REQUEST_METHOD: `POST` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:35:46.201368 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694c79c8-8886-45dc-8d75-342779bfae5c None None] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:35:46.201846 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694c79c8-8886-45dc-8d75-342779bfae5c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:35:46.215566 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9cbe0ef-843a-4e90-bacc-f1a2264df633 None None] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:35:46.216190 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9cbe0ef-843a-4e90-bacc-f1a2264df633 None None] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:35:46.216682 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9cbe0ef-843a-4e90-bacc-f1a2264df633 None None] PATH_INFO: `/` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:35:46.218262 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 29/63] 10.222.0.20 () {54 vars in 926 bytes} [Mon Aug 1 16:35:46 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 01 16:35:46.223775 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7928b2f9-c6bb-451d-a621-31ff1be02c59 None None] REQUEST_METHOD: `POST` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:35:46.224305 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7928b2f9-c6bb-451d-a621-31ff1be02c59 None None] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:35:46.224777 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7928b2f9-c6bb-451d-a621-31ff1be02c59 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:35:46.228826 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.auth.core [None req-694c79c8-8886-45dc-8d75-342779bfae5c None None] MFA Rules not processed for user `1007f8e379b74e169b98fb7c3c0ba5b4`. Rule list: `[]` (Enabled: `True`). {{(pid=87058) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:35:46.230059 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.fernet_utils [None req-694c79c8-8886-45dc-8d75-342779bfae5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:35:46.244653 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.auth.core [None req-7928b2f9-c6bb-451d-a621-31ff1be02c59 None None] MFA Rules not processed for user `1007f8e379b74e169b98fb7c3c0ba5b4`. Rule list: `[]` (Enabled: `True`). {{(pid=87057) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:35:46.246150 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.fernet_utils [None req-7928b2f9-c6bb-451d-a621-31ff1be02c59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87057) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:35:46.334896 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG dbcounter [-] [87058] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:35:46.349617 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 35/64] 10.222.0.20 () {58 vars in 1023 bytes} [Mon Aug 1 16:35:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 16:35:46.362141 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 30/65] 10.222.0.20 () {58 vars in 1023 bytes} [Mon Aug 1 16:35:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 16:35:46.378510 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-694c79c8-8886-45dc-8d75-342779bfae5c None None] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:35:46.381981 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7928b2f9-c6bb-451d-a621-31ff1be02c59 None None] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:35:46.441264 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG dbcounter [-] [87057] Writing DB stats keystone:SELECT=100 {{(pid=87057) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:35:46.499355 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7928b2f9-c6bb-451d-a621-31ff1be02c59 None None] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:35:46.518764 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-694c79c8-8886-45dc-8d75-342779bfae5c None None] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:35:46.539136 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a53836-69f9-41f7-a41f-324426260936 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:35:46.540251 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a53836-69f9-41f7-a41f-324426260936 None admin] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:35:46.540682 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a53836-69f9-41f7-a41f-324426260936 None admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:35:46.541103 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a53836-69f9-41f7-a41f-324426260936 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:35:46.552255 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11a53836-69f9-41f7-a41f-324426260936 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 16:35:46.558200 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application [None req-11a53836-69f9-41f7-a41f-324426260936 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 16:35:46.558200 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 16:35:46.558200 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 16:35:46.558200 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 16:35:46.558200 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 16:35:46.558200 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 16:35:46.558200 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 16:35:46.558200 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 16:35:46.558200 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 16:35:46.558200 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 16:35:46.558200 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 16:35:46.558200 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 16:35:46.558200 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 16:35:46.558200 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 16:35:46.558200 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 16:35:46.558200 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 16:35:46.559054 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 16:35:46.559054 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 16:35:46.559054 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 16:35:46.559054 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 16:35:46.559054 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 16:35:46.559054 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 16:35:46.559054 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 16:35:46.559054 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 16:35:46.559054 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 16:35:46.559054 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 16:35:46.559054 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 16:35:46.559054 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application with Lock( Aug 01 16:35:46.559054 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 16:35:46.559054 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self._enter() Aug 01 16:35:46.559054 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 16:35:46.559054 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 16:35:46.559806 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 16:35:46.559806 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self.creator() Aug 01 16:35:46.559806 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 16:35:46.559806 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application created_value = creator( Aug 01 16:35:46.559806 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 16:35:46.559806 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 16:35:46.559806 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 16:35:46.559806 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 16:35:46.559806 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 16:35:46.559806 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 16:35:46.559806 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 16:35:46.559806 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application Aug 01 16:35:46.562123 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 31/66] 10.222.0.20 () {56 vars in 1096 bytes} [Mon Aug 1 16:35:46 2022] GET /identity/v3/roles/admin => generated 83 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 16:35:46.563971 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f2e4e7-0310-4f0c-9cad-2228615fce8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:35:46.564340 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ecf3691-b3e7-4d2a-9386-040a73d9156b None None] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:35:46.564623 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f2e4e7-0310-4f0c-9cad-2228615fce8a None admin] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:35:46.564903 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f2e4e7-0310-4f0c-9cad-2228615fce8a None admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:35:46.565182 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f2e4e7-0310-4f0c-9cad-2228615fce8a None admin] PATH_INFO: `/v3/roles/member` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:35:46.565448 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ecf3691-b3e7-4d2a-9386-040a73d9156b None None] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:35:46.565911 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ecf3691-b3e7-4d2a-9386-040a73d9156b None None] PATH_INFO: `/` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:35:46.567550 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 32/67] 10.222.0.20 () {54 vars in 926 bytes} [Mon Aug 1 16:35:46 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 16:35:46.568468 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ecf3691-b3e7-4d2a-9386-040a73d9156b None None] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:35:46.579478 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00f2e4e7-0310-4f0c-9cad-2228615fce8a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 16:35:46.586400 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application [None req-00f2e4e7-0310-4f0c-9cad-2228615fce8a None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 01 16:35:46.586400 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 16:35:46.586400 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 16:35:46.586400 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 16:35:46.586400 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 16:35:46.586400 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 16:35:46.586400 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 16:35:46.586400 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 16:35:46.586400 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 16:35:46.586400 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 16:35:46.586400 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 16:35:46.586400 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 16:35:46.586400 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 16:35:46.586400 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 16:35:46.586400 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 16:35:46.586400 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 16:35:46.588239 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 16:35:46.588239 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 16:35:46.588239 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 16:35:46.588239 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 16:35:46.588239 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 16:35:46.588239 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 16:35:46.588239 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 16:35:46.588239 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 16:35:46.588239 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 16:35:46.588239 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 16:35:46.588239 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 16:35:46.588239 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application with Lock( Aug 01 16:35:46.588239 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 16:35:46.588239 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self._enter() Aug 01 16:35:46.588239 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 16:35:46.588239 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 16:35:46.589173 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 16:35:46.589173 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self.creator() Aug 01 16:35:46.589173 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 16:35:46.589173 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application created_value = creator( Aug 01 16:35:46.589173 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 16:35:46.589173 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 16:35:46.589173 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 16:35:46.589173 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 16:35:46.589173 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 16:35:46.589173 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 16:35:46.589173 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 01 16:35:46.589173 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application Aug 01 16:35:46.596713 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 36/68] 10.222.0.20 () {56 vars in 1099 bytes} [Mon Aug 1 16:35:46 2022] GET /identity/v3/roles/member => generated 84 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 16:35:46.596713 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24c38d3-5a5c-47ee-a759-8c068b9c6264 None None] REQUEST_METHOD: `POST` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:35:46.596713 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24c38d3-5a5c-47ee-a759-8c068b9c6264 None None] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:35:46.596713 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24c38d3-5a5c-47ee-a759-8c068b9c6264 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:35:46.618892 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.auth.core [None req-c24c38d3-5a5c-47ee-a759-8c068b9c6264 None None] MFA Rules not processed for user `1007f8e379b74e169b98fb7c3c0ba5b4`. Rule list: `[]` (Enabled: `True`). {{(pid=87058) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:35:46.620093 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.fernet_utils [None req-c24c38d3-5a5c-47ee-a759-8c068b9c6264 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:35:46.672201 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG dbcounter [-] [87058] Writing DB stats keystone:SELECT=100 {{(pid=87058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:35:46.688638 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ecf3691-b3e7-4d2a-9386-040a73d9156b None None] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:35:46.695590 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 37/69] 10.222.0.20 () {58 vars in 1023 bytes} [Mon Aug 1 16:35:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 16:35:46.698906 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c24c38d3-5a5c-47ee-a759-8c068b9c6264 None None] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:35:46.708494 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG dbcounter [-] [87057] Writing DB stats keystone:SELECT=100 {{(pid=87057) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:35:46.738183 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6544da43-583d-426c-b84e-d6ca22a3236d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:35:46.738802 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6544da43-583d-426c-b84e-d6ca22a3236d None admin] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:35:46.739079 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6544da43-583d-426c-b84e-d6ca22a3236d None admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:35:46.739353 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6544da43-583d-426c-b84e-d6ca22a3236d None admin] PATH_INFO: `/v3/roles` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:35:46.740069 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6544da43-583d-426c-b84e-d6ca22a3236d None admin] RBAC: Adding query filter params (name=admin) {{(pid=87057) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:35:46.740370 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6544da43-583d-426c-b84e-d6ca22a3236d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:35:46.741751 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6544da43-583d-426c-b84e-d6ca22a3236d None admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:35:46.754072 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 33/70] 10.222.0.20 () {56 vars in 1099 bytes} [Mon Aug 1 16:35:46 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:35:46.755544 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac7fbc1-a2d9-4ecb-8100-af1f058d7bfa None None] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:35:46.755969 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac7fbc1-a2d9-4ecb-8100-af1f058d7bfa None None] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:35:46.756399 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac7fbc1-a2d9-4ecb-8100-af1f058d7bfa None None] PATH_INFO: `/` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:35:46.757752 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 34/71] 10.222.0.20 () {54 vars in 926 bytes} [Mon Aug 1 16:35:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 16:35:46.758752 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac7fbc1-a2d9-4ecb-8100-af1f058d7bfa None None] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:35:46.830203 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c24c38d3-5a5c-47ee-a759-8c068b9c6264 None None] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:35:46.866359 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2dea6bb-f7f8-4880-8a29-dc27a7a9707c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:35:46.867322 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2dea6bb-f7f8-4880-8a29-dc27a7a9707c None admin] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:35:46.867788 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac7fbc1-a2d9-4ecb-8100-af1f058d7bfa None None] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:35:46.867971 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2dea6bb-f7f8-4880-8a29-dc27a7a9707c None admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:35:46.868079 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2dea6bb-f7f8-4880-8a29-dc27a7a9707c None admin] PATH_INFO: `/v3/roles` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:35:46.868942 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2dea6bb-f7f8-4880-8a29-dc27a7a9707c None admin] RBAC: Adding query filter params (name=member) {{(pid=87058) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:35:46.869406 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2dea6bb-f7f8-4880-8a29-dc27a7a9707c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:35:46.871056 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2dea6bb-f7f8-4880-8a29-dc27a7a9707c None admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:35:46.880038 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 38/72] 10.222.0.20 () {56 vars in 1101 bytes} [Mon Aug 1 16:35:46 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:35:46.881046 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2dea6bb-f7f8-4880-8a29-dc27a7a9707c None admin] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:35:46.916073 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41242d45-ad42-4b0c-a6d3-ea94d35092ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:35:46.916732 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41242d45-ad42-4b0c-a6d3-ea94d35092ac None admin] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:35:46.917185 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41242d45-ad42-4b0c-a6d3-ea94d35092ac None admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:35:46.917466 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41242d45-ad42-4b0c-a6d3-ea94d35092ac None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:35:46.931366 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41242d45-ad42-4b0c-a6d3-ea94d35092ac None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 16:35:46.933876 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG dbcounter [-] [87058] Writing DB stats keystone:SELECT=100 {{(pid=87058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:35:46.938102 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application [None req-41242d45-ad42-4b0c-a6d3-ea94d35092ac None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 01 16:35:46.938102 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 16:35:46.938102 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 16:35:46.938102 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 16:35:46.938102 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 16:35:46.938102 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 16:35:46.938102 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 16:35:46.938102 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 16:35:46.938102 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 16:35:46.938102 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 16:35:46.938102 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 16:35:46.938102 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 16:35:46.938102 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 16:35:46.938102 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 16:35:46.938102 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 16:35:46.938102 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 16:35:46.938953 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 16:35:46.938953 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 16:35:46.938953 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 16:35:46.938953 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 16:35:46.938953 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 16:35:46.938953 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 16:35:46.938953 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 16:35:46.938953 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 16:35:46.938953 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 16:35:46.938953 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 16:35:46.938953 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 16:35:46.938953 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application with Lock( Aug 01 16:35:46.938953 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 16:35:46.938953 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self._enter() Aug 01 16:35:46.938953 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 16:35:46.938953 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 16:35:46.939724 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 16:35:46.939724 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self.creator() Aug 01 16:35:46.939724 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 16:35:46.939724 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application created_value = creator( Aug 01 16:35:46.939724 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 16:35:46.939724 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 16:35:46.939724 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 16:35:46.939724 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 16:35:46.939724 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 16:35:46.939724 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 16:35:46.939724 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 01 16:35:46.939724 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application Aug 01 16:35:46.939724 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 35/73] 10.222.0.20 () {56 vars in 1114 bytes} [Mon Aug 1 16:35:46 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 16:35:46.943879 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859fc53a-79bd-4018-9f8b-f87cc103fe42 None None] REQUEST_METHOD: `POST` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:35:46.943879 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859fc53a-79bd-4018-9f8b-f87cc103fe42 None None] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:35:46.943879 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859fc53a-79bd-4018-9f8b-f87cc103fe42 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:35:46.960505 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.auth.core [None req-859fc53a-79bd-4018-9f8b-f87cc103fe42 None None] MFA Rules not processed for user `1007f8e379b74e169b98fb7c3c0ba5b4`. Rule list: `[]` (Enabled: `True`). {{(pid=87057) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:35:46.961808 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.fernet_utils [None req-859fc53a-79bd-4018-9f8b-f87cc103fe42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87057) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:35:46.985893 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG dbcounter [-] [87057] Writing DB stats keystone:SELECT=100 {{(pid=87057) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:35:46.991434 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2dea6bb-f7f8-4880-8a29-dc27a7a9707c None admin] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:35:47.027660 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20361f2e-403e-4ab8-a353-316ec359cb85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:35:47.028306 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20361f2e-403e-4ab8-a353-316ec359cb85 None admin] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:35:47.028588 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20361f2e-403e-4ab8-a353-316ec359cb85 None admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:35:47.028864 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20361f2e-403e-4ab8-a353-316ec359cb85 None admin] PATH_INFO: `/v3/users/1007f8e379b74e169b98fb7c3c0ba5b4` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:35:47.030024 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20361f2e-403e-4ab8-a353-316ec359cb85 None admin] RBAC: Authorizing `identity:get_user(user_id=1007f8e379b74e169b98fb7c3c0ba5b4)` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:35:47.030852 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 36/74] 10.222.0.20 () {58 vars in 1023 bytes} [Mon Aug 1 16:35:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 16:35:47.044842 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20361f2e-403e-4ab8-a353-316ec359cb85 None admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:35:47.044943 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-859fc53a-79bd-4018-9f8b-f87cc103fe42 None None] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:35:47.045013 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 39/75] 10.222.0.20 () {56 vars in 1177 bytes} [Mon Aug 1 16:35:46 2022] GET /identity/v3/users/1007f8e379b74e169b98fb7c3c0ba5b4 => generated 247 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:35:47.045013 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ebe8ba7-a326-48aa-ba47-20a75a453b5a None None] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:35:47.045013 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ebe8ba7-a326-48aa-ba47-20a75a453b5a None None] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:35:47.045013 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ebe8ba7-a326-48aa-ba47-20a75a453b5a None None] PATH_INFO: `/` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:35:47.045013 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 40/76] 10.222.0.20 () {54 vars in 926 bytes} [Mon Aug 1 16:35:47 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 16:35:47.045013 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ebe8ba7-a326-48aa-ba47-20a75a453b5a None None] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:35:47.136412 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-859fc53a-79bd-4018-9f8b-f87cc103fe42 None None] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:35:47.141759 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ebe8ba7-a326-48aa-ba47-20a75a453b5a None None] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:35:47.152743 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG dbcounter [-] [87058] Writing DB stats keystone:SELECT=100 {{(pid=87058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:35:47.171216 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14dbb5d0-1ccc-43f9-8921-6cf1fc31c387 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:35:47.171853 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14dbb5d0-1ccc-43f9-8921-6cf1fc31c387 None admin] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:35:47.172142 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14dbb5d0-1ccc-43f9-8921-6cf1fc31c387 None admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:35:47.172416 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14dbb5d0-1ccc-43f9-8921-6cf1fc31c387 None admin] PATH_INFO: `/v3/users/d96b664ac2df42d2aa31b69bc44ebf19` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:35:47.175795 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-117f2e79-fbb9-43fb-8bb3-72f4a5e59c0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:35:47.176445 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117f2e79-fbb9-43fb-8bb3-72f4a5e59c0d None admin] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:35:47.176935 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117f2e79-fbb9-43fb-8bb3-72f4a5e59c0d None admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:35:47.177369 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117f2e79-fbb9-43fb-8bb3-72f4a5e59c0d None admin] PATH_INFO: `/v3/roles` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:35:47.178391 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-117f2e79-fbb9-43fb-8bb3-72f4a5e59c0d None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=87058) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:35:47.178698 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-117f2e79-fbb9-43fb-8bb3-72f4a5e59c0d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:35:47.180082 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-117f2e79-fbb9-43fb-8bb3-72f4a5e59c0d None admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:35:47.188199 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14dbb5d0-1ccc-43f9-8921-6cf1fc31c387 None admin] RBAC: Authorizing `identity:get_user(user_id=d96b664ac2df42d2aa31b69bc44ebf19)` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:35:47.188832 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 41/77] 10.222.0.20 () {56 vars in 1111 bytes} [Mon Aug 1 16:35:47 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:35:47.189750 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14dbb5d0-1ccc-43f9-8921-6cf1fc31c387 None admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:35:47.191515 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 37/78] 10.222.0.20 () {56 vars in 1177 bytes} [Mon Aug 1 16:35:47 2022] GET /identity/v3/users/d96b664ac2df42d2aa31b69bc44ebf19 => generated 275 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:35:47.194159 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-117f2e79-fbb9-43fb-8bb3-72f4a5e59c0d None admin] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:35:47.194664 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a241fc1-2fea-4488-8f73-e450c6cced63 None None] REQUEST_METHOD: `POST` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:35:47.194957 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a241fc1-2fea-4488-8f73-e450c6cced63 None None] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:35:47.195228 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a241fc1-2fea-4488-8f73-e450c6cced63 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:35:47.213208 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.auth.core [None req-6a241fc1-2fea-4488-8f73-e450c6cced63 None None] MFA Rules not processed for user `1007f8e379b74e169b98fb7c3c0ba5b4`. Rule list: `[]` (Enabled: `True`). {{(pid=87057) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:35:47.214442 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.fernet_utils [None req-6a241fc1-2fea-4488-8f73-e450c6cced63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87057) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:35:47.228081 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG dbcounter [-] [87057] Writing DB stats keystone:SELECT=100 {{(pid=87057) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:35:47.281727 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 38/79] 10.222.0.20 () {58 vars in 1023 bytes} [Mon Aug 1 16:35:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 16:35:47.282526 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a241fc1-2fea-4488-8f73-e450c6cced63 None None] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:35:47.302068 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-117f2e79-fbb9-43fb-8bb3-72f4a5e59c0d None admin] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:35:47.337068 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40931e5f-7ced-46c4-9689-33a002e432c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:35:47.337721 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40931e5f-7ced-46c4-9689-33a002e432c3 None admin] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:35:47.338009 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40931e5f-7ced-46c4-9689-33a002e432c3 None admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:35:47.338317 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40931e5f-7ced-46c4-9689-33a002e432c3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:35:47.379043 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG dbcounter [-] [87058] Writing DB stats keystone:SELECT=100 {{(pid=87058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:35:47.382681 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a241fc1-2fea-4488-8f73-e450c6cced63 None None] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:35:47.406014 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40931e5f-7ced-46c4-9689-33a002e432c3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:35:47.407875 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40931e5f-7ced-46c4-9689-33a002e432c3 None admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:35:47.416888 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c8326cf-8d2b-4710-8e09-b39358bf2dda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:35:47.417529 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8326cf-8d2b-4710-8e09-b39358bf2dda None admin] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:35:47.417809 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8326cf-8d2b-4710-8e09-b39358bf2dda None admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:35:47.418117 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8326cf-8d2b-4710-8e09-b39358bf2dda None admin] PATH_INFO: `/v3/roles/member` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:35:47.428244 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c8326cf-8d2b-4710-8e09-b39358bf2dda None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 16:35:47.433724 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application [None req-6c8326cf-8d2b-4710-8e09-b39358bf2dda None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 01 16:35:47.433724 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 16:35:47.433724 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 16:35:47.433724 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 16:35:47.433724 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 16:35:47.433724 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 16:35:47.433724 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 16:35:47.433724 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 16:35:47.433724 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 16:35:47.433724 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 16:35:47.433724 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 16:35:47.433724 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 16:35:47.433724 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 16:35:47.433724 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 16:35:47.433724 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 16:35:47.433724 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 16:35:47.434695 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 16:35:47.434695 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 16:35:47.434695 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 16:35:47.434695 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 16:35:47.434695 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 16:35:47.434695 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 16:35:47.434695 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 16:35:47.434695 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 16:35:47.434695 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 16:35:47.434695 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 16:35:47.434695 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 16:35:47.434695 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application with Lock( Aug 01 16:35:47.434695 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 16:35:47.434695 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self._enter() Aug 01 16:35:47.434695 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 16:35:47.434695 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 16:35:47.435450 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 16:35:47.435450 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self.creator() Aug 01 16:35:47.435450 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 16:35:47.435450 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application created_value = creator( Aug 01 16:35:47.435450 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 16:35:47.435450 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 16:35:47.435450 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 16:35:47.435450 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 16:35:47.435450 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 16:35:47.435450 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 16:35:47.435450 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 01 16:35:47.435450 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application Aug 01 16:35:47.435450 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 39/80] 10.222.0.20 () {56 vars in 1099 bytes} [Mon Aug 1 16:35:47 2022] GET /identity/v3/roles/member => generated 84 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 16:35:47.437116 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c8326cf-8d2b-4710-8e09-b39358bf2dda None admin] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:35:47.448381 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG dbcounter [-] [87057] Writing DB stats keystone:SELECT=100 {{(pid=87057) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:35:47.507266 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 42/81] 10.222.0.20 () {58 vars in 1282 bytes} [Mon Aug 1 16:35:47 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 01 16:35:47.508119 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40931e5f-7ced-46c4-9689-33a002e432c3 None admin] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:35:47.535342 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c8326cf-8d2b-4710-8e09-b39358bf2dda None admin] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:35:47.570644 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a876d48b-8746-4c0a-83f0-cdac8ea51eb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:35:47.571341 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a876d48b-8746-4c0a-83f0-cdac8ea51eb3 None admin] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:35:47.571640 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a876d48b-8746-4c0a-83f0-cdac8ea51eb3 None admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:35:47.571886 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a876d48b-8746-4c0a-83f0-cdac8ea51eb3 None admin] PATH_INFO: `/v3/users/d96b664ac2df42d2aa31b69bc44ebf19` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:35:47.573055 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a876d48b-8746-4c0a-83f0-cdac8ea51eb3 None admin] RBAC: Authorizing `identity:get_user(user_id=d96b664ac2df42d2aa31b69bc44ebf19)` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:35:47.574537 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a876d48b-8746-4c0a-83f0-cdac8ea51eb3 None admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:35:47.576263 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 40/82] 10.222.0.20 () {56 vars in 1177 bytes} [Mon Aug 1 16:35:47 2022] GET /identity/v3/users/d96b664ac2df42d2aa31b69bc44ebf19 => generated 275 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:35:47.576862 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a876d48b-8746-4c0a-83f0-cdac8ea51eb3 None admin] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:35:47.577768 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG dbcounter [-] [87058] Writing DB stats keystone:SELECT=100 {{(pid=87058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:35:47.606657 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40931e5f-7ced-46c4-9689-33a002e432c3 None admin] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:35:47.641368 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e815e92-4d81-4e99-9dc6-1560972cb94d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:35:47.641985 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e815e92-4d81-4e99-9dc6-1560972cb94d None admin] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:35:47.642292 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e815e92-4d81-4e99-9dc6-1560972cb94d None admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:35:47.642564 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e815e92-4d81-4e99-9dc6-1560972cb94d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:35:47.655610 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG dbcounter [-] [87057] Writing DB stats keystone:SELECT=100 {{(pid=87057) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:35:47.678268 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a876d48b-8746-4c0a-83f0-cdac8ea51eb3 None admin] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:35:47.709365 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e815e92-4d81-4e99-9dc6-1560972cb94d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:35:47.711016 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e815e92-4d81-4e99-9dc6-1560972cb94d None admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:35:47.712910 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b1b449-f8b8-41bf-b943-19754d7ccbbb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:35:47.713548 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b1b449-f8b8-41bf-b943-19754d7ccbbb None admin] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:35:47.713826 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b1b449-f8b8-41bf-b943-19754d7ccbbb None admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:35:47.714130 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b1b449-f8b8-41bf-b943-19754d7ccbbb None admin] PATH_INFO: `/v3/domains/default` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:35:47.715278 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8b1b449-f8b8-41bf-b943-19754d7ccbbb None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:35:47.716700 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8b1b449-f8b8-41bf-b943-19754d7ccbbb None admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:35:47.718498 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 41/83] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Aug 1 16:35:47 2022] GET /identity/v3/domains/default => generated 199 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:35:47.719061 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b1b449-f8b8-41bf-b943-19754d7ccbbb None admin] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:35:47.785692 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG dbcounter [-] [87058] Writing DB stats keystone:SELECT=100 {{(pid=87058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:35:47.815370 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 43/84] 10.222.0.20 () {58 vars in 1282 bytes} [Mon Aug 1 16:35:47 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 01 16:35:47.815826 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b1b449-f8b8-41bf-b943-19754d7ccbbb None admin] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:35:47.820746 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e815e92-4d81-4e99-9dc6-1560972cb94d None admin] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:35:47.850291 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f899d69e-f5c0-478c-8f91-8432c3de657d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:35:47.850908 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f899d69e-f5c0-478c-8f91-8432c3de657d None admin] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:35:47.851190 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f899d69e-f5c0-478c-8f91-8432c3de657d None admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:35:47.851469 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f899d69e-f5c0-478c-8f91-8432c3de657d None admin] PATH_INFO: `/v3/roles` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:35:47.852233 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f899d69e-f5c0-478c-8f91-8432c3de657d None admin] RBAC: Adding query filter params (name=member) {{(pid=87057) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:35:47.852540 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f899d69e-f5c0-478c-8f91-8432c3de657d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:35:47.854097 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f899d69e-f5c0-478c-8f91-8432c3de657d None admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:35:47.862329 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 42/85] 10.222.0.20 () {56 vars in 1101 bytes} [Mon Aug 1 16:35:47 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:35:47.863117 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f899d69e-f5c0-478c-8f91-8432c3de657d None admin] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:35:47.873486 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG dbcounter [-] [87057] Writing DB stats keystone:SELECT=100 {{(pid=87057) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:35:47.923742 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e815e92-4d81-4e99-9dc6-1560972cb94d None admin] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:35:47.957672 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03607407-b957-4774-9cf4-778194df139a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:35:47.958335 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03607407-b957-4774-9cf4-778194df139a None admin] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:35:47.958625 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03607407-b957-4774-9cf4-778194df139a None admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:35:47.958902 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03607407-b957-4774-9cf4-778194df139a None admin] PATH_INFO: `/v3/projects/64c2a2f6f91d4b9c8d5993d28fd19631` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:35:47.960068 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03607407-b957-4774-9cf4-778194df139a None admin] RBAC: Authorizing `identity:get_project(project_id=64c2a2f6f91d4b9c8d5993d28fd19631)` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:35:47.961535 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03607407-b957-4774-9cf4-778194df139a None admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:35:47.963508 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 44/86] 10.222.0.20 () {56 vars in 1186 bytes} [Mon Aug 1 16:35:47 2022] GET /identity/v3/projects/64c2a2f6f91d4b9c8d5993d28fd19631 => generated 298 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:35:47.964256 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03607407-b957-4774-9cf4-778194df139a None admin] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:35:47.970509 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f899d69e-f5c0-478c-8f91-8432c3de657d None admin] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:35:48.004199 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be488e11-6df0-406d-9f64-dba899ce3a98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:35:48.004815 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be488e11-6df0-406d-9f64-dba899ce3a98 None admin] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:35:48.005098 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be488e11-6df0-406d-9f64-dba899ce3a98 None admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:35:48.005218 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG dbcounter [-] [87058] Writing DB stats keystone:SELECT=100 {{(pid=87058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:35:48.005373 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be488e11-6df0-406d-9f64-dba899ce3a98 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:35:48.061845 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03607407-b957-4774-9cf4-778194df139a None admin] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:35:48.072392 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be488e11-6df0-406d-9f64-dba899ce3a98 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:35:48.073880 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be488e11-6df0-406d-9f64-dba899ce3a98 None admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:35:48.081431 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG dbcounter [-] [87057] Writing DB stats keystone:SELECT=100 {{(pid=87057) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:35:48.095484 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb6288e4-1555-48bb-9e32-f07da1ae95c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:35:48.096109 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6288e4-1555-48bb-9e32-f07da1ae95c1 None admin] REQUEST_METHOD: `POST` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:35:48.096393 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6288e4-1555-48bb-9e32-f07da1ae95c1 None admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:35:48.096668 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6288e4-1555-48bb-9e32-f07da1ae95c1 None admin] PATH_INFO: `/v3/users` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:35:48.097350 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb6288e4-1555-48bb-9e32-f07da1ae95c1 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:35:48.098824 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb6288e4-1555-48bb-9e32-f07da1ae95c1 None admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:35:48.169311 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 43/87] 10.222.0.20 () {58 vars in 1282 bytes} [Mon Aug 1 16:35:47 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 01 16:35:48.170182 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be488e11-6df0-406d-9f64-dba899ce3a98 None admin] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:35:48.187244 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 45/88] 10.222.0.20 () {60 vars in 1132 bytes} [Mon Aug 1 16:35:47 2022] POST /identity/v3/users => generated 289 bytes in 223 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 16:35:48.188361 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb6288e4-1555-48bb-9e32-f07da1ae95c1 None admin] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:35:48.287750 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be488e11-6df0-406d-9f64-dba899ce3a98 None admin] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:35:48.296396 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG dbcounter [-] [87057] Writing DB stats keystone:SELECT=100 {{(pid=87057) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:35:48.301293 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG dbcounter [-] [87058] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:35:48.309492 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb6288e4-1555-48bb-9e32-f07da1ae95c1 None admin] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:35:48.325936 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6924d3e3-2df8-4411-beaf-08b9d22e042c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:35:48.326555 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6924d3e3-2df8-4411-beaf-08b9d22e042c None admin] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:35:48.326855 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6924d3e3-2df8-4411-beaf-08b9d22e042c None admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:35:48.327319 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6924d3e3-2df8-4411-beaf-08b9d22e042c None admin] PATH_INFO: `/v3/projects/64c2a2f6f91d4b9c8d5993d28fd19631` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:35:48.328857 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6924d3e3-2df8-4411-beaf-08b9d22e042c None admin] RBAC: Authorizing `identity:get_project(project_id=64c2a2f6f91d4b9c8d5993d28fd19631)` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:35:48.330363 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6924d3e3-2df8-4411-beaf-08b9d22e042c None admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:35:48.341336 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 44/89] 10.222.0.20 () {56 vars in 1186 bytes} [Mon Aug 1 16:35:48 2022] GET /identity/v3/projects/64c2a2f6f91d4b9c8d5993d28fd19631 => generated 298 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:35:48.341336 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6924d3e3-2df8-4411-beaf-08b9d22e042c None admin] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:35:48.354513 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5fcb6bc-58e9-4582-8744-fa5b0d70ecd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:35:48.355018 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5fcb6bc-58e9-4582-8744-fa5b0d70ecd4 None admin] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:35:48.355311 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5fcb6bc-58e9-4582-8744-fa5b0d70ecd4 None admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:35:48.355588 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5fcb6bc-58e9-4582-8744-fa5b0d70ecd4 None admin] PATH_INFO: `/v3/users/d96b664ac2df42d2aa31b69bc44ebf19` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:35:48.356921 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5fcb6bc-58e9-4582-8744-fa5b0d70ecd4 None admin] RBAC: Authorizing `identity:get_user(user_id=d96b664ac2df42d2aa31b69bc44ebf19)` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:35:48.358647 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5fcb6bc-58e9-4582-8744-fa5b0d70ecd4 None admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:35:48.360512 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 46/90] 10.222.0.20 () {56 vars in 1177 bytes} [Mon Aug 1 16:35:48 2022] GET /identity/v3/users/d96b664ac2df42d2aa31b69bc44ebf19 => generated 275 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:35:48.361199 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5fcb6bc-58e9-4582-8744-fa5b0d70ecd4 None admin] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:35:48.450388 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6924d3e3-2df8-4411-beaf-08b9d22e042c None admin] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:35:48.494544 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a63482c-5a0e-48f0-8ef7-d2ed61da0ed1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:35:48.495078 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a63482c-5a0e-48f0-8ef7-d2ed61da0ed1 None admin] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:35:48.495359 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a63482c-5a0e-48f0-8ef7-d2ed61da0ed1 None admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:35:48.495630 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a63482c-5a0e-48f0-8ef7-d2ed61da0ed1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:35:48.496434 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a63482c-5a0e-48f0-8ef7-d2ed61da0ed1 None admin] RBAC: Adding query filter params (role.id=440bb943dd5b46fea038b903b6eaaf81, scope.project.id=64c2a2f6f91d4b9c8d5993d28fd19631, user.id=1007f8e379b74e169b98fb7c3c0ba5b4) {{(pid=87057) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:35:48.496742 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a63482c-5a0e-48f0-8ef7-d2ed61da0ed1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:35:48.498336 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a63482c-5a0e-48f0-8ef7-d2ed61da0ed1 None admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:35:48.508032 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 45/91] 10.222.0.20 () {56 vars in 1374 bytes} [Mon Aug 1 16:35:48 2022] GET /identity/v3/role_assignments?user.id=1007f8e379b74e169b98fb7c3c0ba5b4&scope.project.id=64c2a2f6f91d4b9c8d5993d28fd19631&role.id=440bb943dd5b46fea038b903b6eaaf81 => generated 259 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:35:48.509623 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a63482c-5a0e-48f0-8ef7-d2ed61da0ed1 None admin] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:35:48.550068 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5fcb6bc-58e9-4582-8744-fa5b0d70ecd4 None admin] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:35:48.567700 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG dbcounter [-] [87057] Writing DB stats keystone:SELECT=100 {{(pid=87057) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:35:48.610367 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb5a25b-d571-449b-82d4-a1c678a793ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:35:48.611017 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb5a25b-d571-449b-82d4-a1c678a793ae None admin] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:35:48.611301 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb5a25b-d571-449b-82d4-a1c678a793ae None admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:35:48.611576 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb5a25b-d571-449b-82d4-a1c678a793ae None admin] PATH_INFO: `/v3/projects/64c2a2f6f91d4b9c8d5993d28fd19631` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:35:48.614303 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eb5a25b-d571-449b-82d4-a1c678a793ae None admin] RBAC: Authorizing `identity:get_project(project_id=64c2a2f6f91d4b9c8d5993d28fd19631)` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:35:48.615802 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eb5a25b-d571-449b-82d4-a1c678a793ae None admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:35:48.618181 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 47/92] 10.222.0.20 () {56 vars in 1186 bytes} [Mon Aug 1 16:35:48 2022] GET /identity/v3/projects/64c2a2f6f91d4b9c8d5993d28fd19631 => generated 298 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:35:48.619615 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb5a25b-d571-449b-82d4-a1c678a793ae None admin] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:35:48.671411 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a63482c-5a0e-48f0-8ef7-d2ed61da0ed1 None admin] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:35:48.688281 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG dbcounter [-] [87058] Writing DB stats keystone:SELECT=100 {{(pid=87058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:35:48.747056 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5084ff0c-0019-4e96-9068-9d54a650820b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:35:48.748497 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5084ff0c-0019-4e96-9068-9d54a650820b None admin] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:35:48.748784 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5084ff0c-0019-4e96-9068-9d54a650820b None admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:35:48.749078 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5084ff0c-0019-4e96-9068-9d54a650820b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:35:48.749881 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5084ff0c-0019-4e96-9068-9d54a650820b None admin] RBAC: Adding query filter params (role.id=0999d3da63a545f18ff3252afdc90518, scope.project.id=64c2a2f6f91d4b9c8d5993d28fd19631, user.id=d96b664ac2df42d2aa31b69bc44ebf19) {{(pid=87057) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:35:48.750240 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5084ff0c-0019-4e96-9068-9d54a650820b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:35:48.751705 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5084ff0c-0019-4e96-9068-9d54a650820b None admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:35:48.761609 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 46/93] 10.222.0.20 () {56 vars in 1374 bytes} [Mon Aug 1 16:35:48 2022] GET /identity/v3/role_assignments?user.id=d96b664ac2df42d2aa31b69bc44ebf19&scope.project.id=64c2a2f6f91d4b9c8d5993d28fd19631&role.id=0999d3da63a545f18ff3252afdc90518 => generated 259 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:35:48.762257 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5084ff0c-0019-4e96-9068-9d54a650820b None admin] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:35:48.812118 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb5a25b-d571-449b-82d4-a1c678a793ae None admin] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:35:48.886194 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e07f724-56ad-4f9d-b6eb-65d248af2484 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:35:48.886764 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e07f724-56ad-4f9d-b6eb-65d248af2484 None admin] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:35:48.887048 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e07f724-56ad-4f9d-b6eb-65d248af2484 None admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:35:48.887328 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e07f724-56ad-4f9d-b6eb-65d248af2484 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:35:48.975381 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5084ff0c-0019-4e96-9068-9d54a650820b None admin] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:35:48.977978 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG dbcounter [-] [87057] Writing DB stats keystone:SELECT=100 {{(pid=87057) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:35:49.027041 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e07f724-56ad-4f9d-b6eb-65d248af2484 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:35:49.030350 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e07f724-56ad-4f9d-b6eb-65d248af2484 None admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:35:49.051129 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a572297d-5d83-4417-a206-ca2c90595d3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:35:49.051771 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a572297d-5d83-4417-a206-ca2c90595d3a None admin] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:35:49.052056 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a572297d-5d83-4417-a206-ca2c90595d3a None admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:35:49.052336 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a572297d-5d83-4417-a206-ca2c90595d3a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:35:49.053327 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a572297d-5d83-4417-a206-ca2c90595d3a None admin] RBAC: Adding query filter params (role.id=3fac71a39f3f4ed198a3579064c30268, scope.project.id=64c2a2f6f91d4b9c8d5993d28fd19631, user.id=d96b664ac2df42d2aa31b69bc44ebf19) {{(pid=87057) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:35:49.053640 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a572297d-5d83-4417-a206-ca2c90595d3a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:35:49.057549 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a572297d-5d83-4417-a206-ca2c90595d3a None admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:35:49.067018 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 47/94] 10.222.0.20 () {56 vars in 1374 bytes} [Mon Aug 1 16:35:48 2022] GET /identity/v3/role_assignments?user.id=d96b664ac2df42d2aa31b69bc44ebf19&scope.project.id=64c2a2f6f91d4b9c8d5993d28fd19631&role.id=3fac71a39f3f4ed198a3579064c30268 => generated 259 bytes in 305 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:35:49.078463 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG dbcounter [-] [87058] Writing DB stats keystone:SELECT=100 {{(pid=87058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:35:49.167797 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 48/95] 10.222.0.20 () {58 vars in 1282 bytes} [Mon Aug 1 16:35:48 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 549 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 01 16:35:49.173741 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a572297d-5d83-4417-a206-ca2c90595d3a None admin] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:35:49.324401 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a572297d-5d83-4417-a206-ca2c90595d3a None admin] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:35:49.374828 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53761096-426b-48a4-9cd4-ff6ea3965e56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:35:49.375503 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53761096-426b-48a4-9cd4-ff6ea3965e56 None admin] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:35:49.375787 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53761096-426b-48a4-9cd4-ff6ea3965e56 None admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:35:49.376063 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53761096-426b-48a4-9cd4-ff6ea3965e56 None admin] PATH_INFO: `/v3/projects/0c14cf5caf744ecf88597ead2702ec70` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:35:49.377323 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53761096-426b-48a4-9cd4-ff6ea3965e56 None admin] RBAC: Authorizing `identity:get_project(project_id=0c14cf5caf744ecf88597ead2702ec70)` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:35:49.378823 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53761096-426b-48a4-9cd4-ff6ea3965e56 None admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:35:49.380511 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 48/96] 10.222.0.20 () {56 vars in 1186 bytes} [Mon Aug 1 16:35:49 2022] GET /identity/v3/projects/0c14cf5caf744ecf88597ead2702ec70 => generated 312 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:35:49.386534 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e07f724-56ad-4f9d-b6eb-65d248af2484 None admin] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:35:49.505737 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e07f724-56ad-4f9d-b6eb-65d248af2484 None admin] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:35:49.541079 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG dbcounter [-] [87058] Writing DB stats keystone:SELECT=100 {{(pid=87058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:35:49.546687 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a02243-7b82-43cc-92f9-fc3f53b5f8c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:35:49.547345 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a02243-7b82-43cc-92f9-fc3f53b5f8c4 None admin] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:35:49.547708 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a02243-7b82-43cc-92f9-fc3f53b5f8c4 None admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:35:49.547986 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a02243-7b82-43cc-92f9-fc3f53b5f8c4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:35:49.548877 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39a02243-7b82-43cc-92f9-fc3f53b5f8c4 None admin] RBAC: Adding query filter params (role.id=0999d3da63a545f18ff3252afdc90518, scope.project.id=0c14cf5caf744ecf88597ead2702ec70, user.id=d96b664ac2df42d2aa31b69bc44ebf19) {{(pid=87058) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:35:49.549189 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39a02243-7b82-43cc-92f9-fc3f53b5f8c4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:35:49.550666 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39a02243-7b82-43cc-92f9-fc3f53b5f8c4 None admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:35:49.560725 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 49/97] 10.222.0.20 () {56 vars in 1374 bytes} [Mon Aug 1 16:35:49 2022] GET /identity/v3/role_assignments?user.id=d96b664ac2df42d2aa31b69bc44ebf19&scope.project.id=0c14cf5caf744ecf88597ead2702ec70&role.id=0999d3da63a545f18ff3252afdc90518 => generated 259 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:35:51.024362 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14663c2c-4cd4-4049-970f-7a05559fc26f None None] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:35:51.025259 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14663c2c-4cd4-4049-970f-7a05559fc26f None None] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:35:51.025259 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14663c2c-4cd4-4049-970f-7a05559fc26f None None] PATH_INFO: `/` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:35:51.026481 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 49/98] 10.222.0.20 () {54 vars in 926 bytes} [Mon Aug 1 16:35:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 16:35:51.031528 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d6caa6-2609-40ff-b953-8219ec97ea0e None None] REQUEST_METHOD: `POST` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:35:51.032022 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d6caa6-2609-40ff-b953-8219ec97ea0e None None] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:35:51.032170 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d6caa6-2609-40ff-b953-8219ec97ea0e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:35:51.054621 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.auth.core [None req-09d6caa6-2609-40ff-b953-8219ec97ea0e None None] MFA Rules not processed for user `1007f8e379b74e169b98fb7c3c0ba5b4`. Rule list: `[]` (Enabled: `True`). {{(pid=87058) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:35:51.056628 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.fernet_utils [None req-09d6caa6-2609-40ff-b953-8219ec97ea0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:35:51.140761 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 50/99] 10.222.0.20 () {58 vars in 1023 bytes} [Mon Aug 1 16:35:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 16:35:51.179171 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14663c2c-4cd4-4049-970f-7a05559fc26f None None] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:35:51.231605 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG dbcounter [-] [87057] Writing DB stats keystone:SELECT=100 {{(pid=87057) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:35:51.328786 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14663c2c-4cd4-4049-970f-7a05559fc26f None None] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:35:51.335076 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc0c859-70b4-4801-b050-d1697135f94e None None] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:35:51.335643 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc0c859-70b4-4801-b050-d1697135f94e None None] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:35:51.336185 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc0c859-70b4-4801-b050-d1697135f94e None None] PATH_INFO: `/` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:35:51.338137 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 51/100] 10.222.0.20 () {54 vars in 926 bytes} [Mon Aug 1 16:35:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 16:35:51.353067 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7bc6c7-58e6-4ec1-b748-9070c11df87a None None] REQUEST_METHOD: `POST` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:35:51.353630 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7bc6c7-58e6-4ec1-b748-9070c11df87a None None] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:35:51.354162 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7bc6c7-58e6-4ec1-b748-9070c11df87a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:35:51.383295 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.auth.core [None req-ff7bc6c7-58e6-4ec1-b748-9070c11df87a None None] MFA Rules not processed for user `1007f8e379b74e169b98fb7c3c0ba5b4`. Rule list: `[]` (Enabled: `True`). {{(pid=87058) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:35:51.385083 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.fernet_utils [None req-ff7bc6c7-58e6-4ec1-b748-9070c11df87a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:35:51.394063 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-370eac44-2627-498f-ad16-e6dabbf6807a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:35:51.394063 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370eac44-2627-498f-ad16-e6dabbf6807a None admin] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:35:51.394063 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370eac44-2627-498f-ad16-e6dabbf6807a None admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:35:51.394063 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370eac44-2627-498f-ad16-e6dabbf6807a None admin] PATH_INFO: `/v3/roles/reader` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:35:51.420166 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-370eac44-2627-498f-ad16-e6dabbf6807a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 16:35:51.425367 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application [None req-370eac44-2627-498f-ad16-e6dabbf6807a None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 01 16:35:51.425367 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 16:35:51.425367 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 16:35:51.425367 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 16:35:51.425367 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 16:35:51.425367 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 16:35:51.425367 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 16:35:51.425367 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 16:35:51.425367 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 16:35:51.425367 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 16:35:51.425367 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 16:35:51.425367 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 16:35:51.425367 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 16:35:51.425367 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 16:35:51.425367 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 16:35:51.425367 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 16:35:51.426244 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 16:35:51.426244 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 16:35:51.426244 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 16:35:51.426244 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 16:35:51.426244 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 16:35:51.426244 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 16:35:51.426244 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 16:35:51.426244 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 16:35:51.426244 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 16:35:51.426244 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 16:35:51.426244 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 16:35:51.426244 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application with Lock( Aug 01 16:35:51.426244 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 16:35:51.426244 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self._enter() Aug 01 16:35:51.426244 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 16:35:51.426244 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 16:35:51.427438 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 16:35:51.427438 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self.creator() Aug 01 16:35:51.427438 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 16:35:51.427438 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application created_value = creator( Aug 01 16:35:51.427438 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 16:35:51.427438 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 16:35:51.427438 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 16:35:51.427438 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 16:35:51.427438 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 16:35:51.427438 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 16:35:51.427438 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 01 16:35:51.427438 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application Aug 01 16:35:51.427438 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 50/101] 10.222.0.20 () {56 vars in 1099 bytes} [Mon Aug 1 16:35:51 2022] GET /identity/v3/roles/reader => generated 84 bytes in 248 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 16:35:51.433410 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-370eac44-2627-498f-ad16-e6dabbf6807a None admin] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:35:51.545212 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 52/102] 10.222.0.20 () {58 vars in 1023 bytes} [Mon Aug 1 16:35:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 16:35:51.572505 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2f92f7-913a-4b0a-b007-966df5611fb9 None None] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:35:51.572803 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2f92f7-913a-4b0a-b007-966df5611fb9 None None] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:35:51.573062 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2f92f7-913a-4b0a-b007-966df5611fb9 None None] PATH_INFO: `/` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:35:51.574615 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 53/103] 10.222.0.20 () {54 vars in 926 bytes} [Mon Aug 1 16:35:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 01 16:35:51.578452 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b2f92f7-913a-4b0a-b007-966df5611fb9 None None] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:35:51.622389 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG dbcounter [-] [87057] Writing DB stats keystone:SELECT=100 {{(pid=87057) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:35:51.631828 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-370eac44-2627-498f-ad16-e6dabbf6807a None admin] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:35:51.683734 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG dbcounter [-] [87058] Writing DB stats keystone:SELECT=100 {{(pid=87058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:35:51.701111 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a6737c-b3cc-4b5e-b53d-befdcde23a33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:35:51.707354 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a6737c-b3cc-4b5e-b53d-befdcde23a33 None admin] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:35:51.707354 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a6737c-b3cc-4b5e-b53d-befdcde23a33 None admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:35:51.707354 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a6737c-b3cc-4b5e-b53d-befdcde23a33 None admin] PATH_INFO: `/v3/roles` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:35:51.707354 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71a6737c-b3cc-4b5e-b53d-befdcde23a33 None admin] RBAC: Adding query filter params (name=reader) {{(pid=87057) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:35:51.707354 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71a6737c-b3cc-4b5e-b53d-befdcde23a33 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:35:51.707354 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71a6737c-b3cc-4b5e-b53d-befdcde23a33 None admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:35:51.716985 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 51/104] 10.222.0.20 () {56 vars in 1101 bytes} [Mon Aug 1 16:35:51 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 284 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:35:51.718455 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6457076e-3855-4ae0-b08b-aca414ac2c75 None None] REQUEST_METHOD: `POST` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:35:51.719278 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6457076e-3855-4ae0-b08b-aca414ac2c75 None None] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:35:51.719704 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6457076e-3855-4ae0-b08b-aca414ac2c75 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:35:51.740666 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.auth.core [None req-6457076e-3855-4ae0-b08b-aca414ac2c75 None None] MFA Rules not processed for user `1007f8e379b74e169b98fb7c3c0ba5b4`. Rule list: `[]` (Enabled: `True`). {{(pid=87057) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:35:51.742196 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.fernet_utils [None req-6457076e-3855-4ae0-b08b-aca414ac2c75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87057) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:35:51.773231 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b2f92f7-913a-4b0a-b007-966df5611fb9 None None] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:35:51.810808 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d308fd67-57ca-406a-951f-cb8d72257311 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:35:51.811900 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d308fd67-57ca-406a-951f-cb8d72257311 None admin] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:35:51.812186 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d308fd67-57ca-406a-951f-cb8d72257311 None admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:35:51.812458 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d308fd67-57ca-406a-951f-cb8d72257311 None admin] PATH_INFO: `/v3/domains/default` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:35:51.813904 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d308fd67-57ca-406a-951f-cb8d72257311 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:35:51.816293 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d308fd67-57ca-406a-951f-cb8d72257311 None admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:35:51.818209 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 54/105] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Aug 1 16:35:51 2022] GET /identity/v3/domains/default => generated 199 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 16:35:51.819562 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d308fd67-57ca-406a-951f-cb8d72257311 None admin] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:35:51.823269 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 52/106] 10.222.0.20 () {58 vars in 1023 bytes} [Mon Aug 1 16:35:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 16:35:51.824953 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc4c88b-74df-47b2-8f2f-597e46e634f0 None None] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:35:51.825460 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc4c88b-74df-47b2-8f2f-597e46e634f0 None None] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:35:51.825980 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc4c88b-74df-47b2-8f2f-597e46e634f0 None None] PATH_INFO: `/` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:35:51.827540 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 53/107] 10.222.0.20 () {54 vars in 926 bytes} [Mon Aug 1 16:35:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 16:35:51.828534 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc4c88b-74df-47b2-8f2f-597e46e634f0 None None] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:35:51.926143 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG dbcounter [-] [87057] Writing DB stats keystone:SELECT=100 {{(pid=87057) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:35:51.933732 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d308fd67-57ca-406a-951f-cb8d72257311 None admin] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:35:51.948875 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc4c88b-74df-47b2-8f2f-597e46e634f0 None None] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:35:51.957563 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG dbcounter [-] [87058] Writing DB stats keystone:SELECT=100 {{(pid=87058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:35:51.969523 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-711d15e2-06a6-4833-b6a2-9c8c0b049f80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:35:51.970247 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711d15e2-06a6-4833-b6a2-9c8c0b049f80 None admin] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:35:51.970531 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711d15e2-06a6-4833-b6a2-9c8c0b049f80 None admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:35:51.970834 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711d15e2-06a6-4833-b6a2-9c8c0b049f80 None admin] PATH_INFO: `/v3/users/cce885e3eda2452eadebf5bff42d3e8d` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:35:51.983348 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cde80c70-d984-41c7-b693-93c03438a33a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:35:51.983964 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde80c70-d984-41c7-b693-93c03438a33a None admin] REQUEST_METHOD: `POST` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:35:51.984246 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde80c70-d984-41c7-b693-93c03438a33a None admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:35:51.984536 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde80c70-d984-41c7-b693-93c03438a33a None admin] PATH_INFO: `/v3/projects` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:35:51.985228 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cde80c70-d984-41c7-b693-93c03438a33a None admin] RBAC: Authorizing `identity:create_project()` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:35:51.986266 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-711d15e2-06a6-4833-b6a2-9c8c0b049f80 None admin] RBAC: Authorizing `identity:get_user(user_id=cce885e3eda2452eadebf5bff42d3e8d)` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:35:51.986718 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cde80c70-d984-41c7-b693-93c03438a33a None admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:35:51.987716 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-711d15e2-06a6-4833-b6a2-9c8c0b049f80 None admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:35:51.994098 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 55/108] 10.222.0.20 () {56 vars in 1177 bytes} [Mon Aug 1 16:35:51 2022] GET /identity/v3/users/cce885e3eda2452eadebf5bff42d3e8d => generated 289 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:35:51.994098 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f03a43e-ef75-4f5b-8718-6ec765a2c481 None None] REQUEST_METHOD: `POST` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:35:51.994098 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f03a43e-ef75-4f5b-8718-6ec765a2c481 None None] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:35:51.994098 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f03a43e-ef75-4f5b-8718-6ec765a2c481 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:35:52.011298 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.auth.core [None req-4f03a43e-ef75-4f5b-8718-6ec765a2c481 None None] MFA Rules not processed for user `1007f8e379b74e169b98fb7c3c0ba5b4`. Rule list: `[]` (Enabled: `True`). {{(pid=87058) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:35:52.012482 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.fernet_utils [None req-4f03a43e-ef75-4f5b-8718-6ec765a2c481 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:35:52.080284 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 56/109] 10.222.0.20 () {58 vars in 1023 bytes} [Mon Aug 1 16:35:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 16:35:52.089339 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f03a43e-ef75-4f5b-8718-6ec765a2c481 None None] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:35:52.100554 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 54/110] 10.222.0.20 () {60 vars in 1141 bytes} [Mon Aug 1 16:35:51 2022] POST /identity/v3/projects => generated 302 bytes in 272 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 16:35:52.101978 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f81f32-ce18-43ac-8880-281ee2f14340 None None] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:35:52.102333 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f81f32-ce18-43ac-8880-281ee2f14340 None None] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:35:52.102613 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f81f32-ce18-43ac-8880-281ee2f14340 None None] PATH_INFO: `/` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:35:52.104191 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 55/111] 10.222.0.20 () {54 vars in 926 bytes} [Mon Aug 1 16:35:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 16:35:52.109459 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f81f32-ce18-43ac-8880-281ee2f14340 None None] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:35:52.226223 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f03a43e-ef75-4f5b-8718-6ec765a2c481 None None] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:35:52.242445 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f81f32-ce18-43ac-8880-281ee2f14340 None None] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:35:52.247167 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG dbcounter [-] [87058] Writing DB stats keystone:SELECT=100 {{(pid=87058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:35:52.275769 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c70e268-efb4-42c4-bfb3-1158aae703af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:35:52.276432 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c70e268-efb4-42c4-bfb3-1158aae703af None admin] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:35:52.276716 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c70e268-efb4-42c4-bfb3-1158aae703af None admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:35:52.276991 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c70e268-efb4-42c4-bfb3-1158aae703af None admin] PATH_INFO: `/v3/roles/member` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:35:52.291235 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ae7753-563f-4ffb-b252-fb10dbf25534 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:35:52.291651 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c70e268-efb4-42c4-bfb3-1158aae703af None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 16:35:52.292531 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ae7753-563f-4ffb-b252-fb10dbf25534 None admin] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:35:52.293010 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ae7753-563f-4ffb-b252-fb10dbf25534 None admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:35:52.293480 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ae7753-563f-4ffb-b252-fb10dbf25534 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:35:52.296886 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application [None req-6c70e268-efb4-42c4-bfb3-1158aae703af None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 01 16:35:52.296886 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 16:35:52.296886 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 16:35:52.296886 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 16:35:52.296886 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 16:35:52.296886 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 16:35:52.296886 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 16:35:52.296886 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 16:35:52.296886 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 16:35:52.296886 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 16:35:52.296886 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 16:35:52.296886 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 16:35:52.296886 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 16:35:52.296886 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 16:35:52.296886 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 16:35:52.296886 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 16:35:52.297740 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 16:35:52.297740 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 16:35:52.297740 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 16:35:52.297740 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 16:35:52.297740 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 16:35:52.297740 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 16:35:52.297740 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 16:35:52.297740 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 16:35:52.297740 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 16:35:52.297740 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 16:35:52.297740 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 16:35:52.297740 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application with Lock( Aug 01 16:35:52.297740 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 16:35:52.297740 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self._enter() Aug 01 16:35:52.297740 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 16:35:52.297740 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 16:35:52.298578 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 16:35:52.298578 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self.creator() Aug 01 16:35:52.298578 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 16:35:52.298578 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application created_value = creator( Aug 01 16:35:52.298578 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 16:35:52.298578 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 16:35:52.298578 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 16:35:52.298578 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 16:35:52.298578 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 16:35:52.298578 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 16:35:52.298578 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 01 16:35:52.298578 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application Aug 01 16:35:52.302826 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 57/112] 10.222.0.20 () {56 vars in 1099 bytes} [Mon Aug 1 16:35:52 2022] GET /identity/v3/roles/member => generated 84 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 16:35:52.303618 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c70e268-efb4-42c4-bfb3-1158aae703af None admin] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:35:52.304106 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG dbcounter [-] [87057] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87057) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:35:52.374889 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4ae7753-563f-4ffb-b252-fb10dbf25534 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:35:52.376393 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4ae7753-563f-4ffb-b252-fb10dbf25534 None admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:35:52.419382 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c70e268-efb4-42c4-bfb3-1158aae703af None admin] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:35:52.497947 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-346784f2-2a27-4f9f-b876-2f189681f285 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:35:52.498729 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346784f2-2a27-4f9f-b876-2f189681f285 None admin] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:35:52.499017 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346784f2-2a27-4f9f-b876-2f189681f285 None admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:35:52.499291 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346784f2-2a27-4f9f-b876-2f189681f285 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:35:52.513292 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-346784f2-2a27-4f9f-b876-2f189681f285 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 16:35:52.520129 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application [None req-346784f2-2a27-4f9f-b876-2f189681f285 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 16:35:52.520129 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 16:35:52.520129 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 16:35:52.520129 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 16:35:52.520129 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 16:35:52.520129 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 16:35:52.520129 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 16:35:52.520129 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 16:35:52.520129 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 16:35:52.520129 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 16:35:52.520129 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 16:35:52.520129 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 16:35:52.520129 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 16:35:52.520129 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 16:35:52.520129 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 16:35:52.520129 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 16:35:52.521046 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 16:35:52.521046 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 16:35:52.521046 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 16:35:52.521046 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 16:35:52.521046 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 16:35:52.521046 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 16:35:52.521046 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 16:35:52.521046 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 16:35:52.521046 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 16:35:52.521046 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 16:35:52.521046 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 16:35:52.521046 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application with Lock( Aug 01 16:35:52.521046 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 16:35:52.521046 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self._enter() Aug 01 16:35:52.521046 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 16:35:52.521046 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 16:35:52.521809 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 16:35:52.521809 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self.creator() Aug 01 16:35:52.521809 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 16:35:52.521809 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application created_value = creator( Aug 01 16:35:52.521809 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 16:35:52.521809 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 16:35:52.521809 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 16:35:52.521809 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 16:35:52.521809 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 16:35:52.521809 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 16:35:52.521809 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 16:35:52.521809 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application Aug 01 16:35:52.521809 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 58/113] 10.222.0.20 () {56 vars in 1096 bytes} [Mon Aug 1 16:35:52 2022] GET /identity/v3/roles/admin => generated 83 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 16:35:52.523728 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1ac65f-ef5f-44ad-b74a-920e752927bf None None] REQUEST_METHOD: `POST` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:35:52.524545 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1ac65f-ef5f-44ad-b74a-920e752927bf None None] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:35:52.524825 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1ac65f-ef5f-44ad-b74a-920e752927bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:35:52.545148 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.auth.core [None req-3d1ac65f-ef5f-44ad-b74a-920e752927bf None None] MFA Rules not processed for user `1007f8e379b74e169b98fb7c3c0ba5b4`. Rule list: `[]` (Enabled: `True`). {{(pid=87058) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:35:52.546392 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.fernet_utils [None req-3d1ac65f-ef5f-44ad-b74a-920e752927bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:35:52.559224 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 56/114] 10.222.0.20 () {58 vars in 1282 bytes} [Mon Aug 1 16:35:52 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 454 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 01 16:35:52.559987 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ae7753-563f-4ffb-b252-fb10dbf25534 None admin] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:35:52.579612 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG dbcounter [-] [87058] Writing DB stats keystone:SELECT=100 {{(pid=87058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:35:52.604672 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG dbcounter [-] [87057] Writing DB stats keystone:SELECT=100 {{(pid=87057) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:35:52.627281 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 59/115] 10.222.0.20 () {58 vars in 1023 bytes} [Mon Aug 1 16:35:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 16:35:52.628237 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d1ac65f-ef5f-44ad-b74a-920e752927bf None None] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:35:52.685278 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ae7753-563f-4ffb-b252-fb10dbf25534 None admin] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:35:52.727986 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ce0a49-a993-4b66-b84c-c16bd2d7ccb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:35:52.728649 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ce0a49-a993-4b66-b84c-c16bd2d7ccb5 None admin] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:35:52.728935 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ce0a49-a993-4b66-b84c-c16bd2d7ccb5 None admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:35:52.729274 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ce0a49-a993-4b66-b84c-c16bd2d7ccb5 None admin] PATH_INFO: `/v3/roles` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:35:52.730012 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85ce0a49-a993-4b66-b84c-c16bd2d7ccb5 None admin] RBAC: Adding query filter params (name=member) {{(pid=87057) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:35:52.730345 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85ce0a49-a993-4b66-b84c-c16bd2d7ccb5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:35:52.731727 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85ce0a49-a993-4b66-b84c-c16bd2d7ccb5 None admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:35:52.740124 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 57/116] 10.222.0.20 () {56 vars in 1101 bytes} [Mon Aug 1 16:35:52 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:35:52.743773 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ce0a49-a993-4b66-b84c-c16bd2d7ccb5 None admin] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:35:52.751865 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d1ac65f-ef5f-44ad-b74a-920e752927bf None None] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:35:52.786730 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94639a21-5cdc-4db2-91c2-5b07d3801e22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:35:52.787357 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94639a21-5cdc-4db2-91c2-5b07d3801e22 None admin] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:35:52.787638 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94639a21-5cdc-4db2-91c2-5b07d3801e22 None admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:35:52.787917 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94639a21-5cdc-4db2-91c2-5b07d3801e22 None admin] PATH_INFO: `/v3/roles` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:35:52.788640 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94639a21-5cdc-4db2-91c2-5b07d3801e22 None admin] RBAC: Adding query filter params (name=admin) {{(pid=87058) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:35:52.788949 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94639a21-5cdc-4db2-91c2-5b07d3801e22 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:35:52.790414 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94639a21-5cdc-4db2-91c2-5b07d3801e22 None admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:35:52.807857 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 60/117] 10.222.0.20 () {56 vars in 1099 bytes} [Mon Aug 1 16:35:52 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:35:52.808719 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94639a21-5cdc-4db2-91c2-5b07d3801e22 None admin] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:35:52.835938 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG dbcounter [-] [87058] Writing DB stats keystone:SELECT=100 {{(pid=87058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:35:52.861422 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG dbcounter [-] [87057] Writing DB stats keystone:SELECT=100 {{(pid=87057) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:35:52.864207 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ce0a49-a993-4b66-b84c-c16bd2d7ccb5 None admin] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:35:52.907926 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4517759e-c97d-4514-aafd-721acbeb19f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:35:52.908591 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4517759e-c97d-4514-aafd-721acbeb19f1 None admin] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:35:52.908894 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4517759e-c97d-4514-aafd-721acbeb19f1 None admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:35:52.909152 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4517759e-c97d-4514-aafd-721acbeb19f1 None admin] PATH_INFO: `/v3/projects/64c2a2f6f91d4b9c8d5993d28fd19631` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:35:52.910691 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4517759e-c97d-4514-aafd-721acbeb19f1 None admin] RBAC: Authorizing `identity:get_project(project_id=64c2a2f6f91d4b9c8d5993d28fd19631)` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:35:52.912159 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4517759e-c97d-4514-aafd-721acbeb19f1 None admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:35:52.913849 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 58/118] 10.222.0.20 () {56 vars in 1186 bytes} [Mon Aug 1 16:35:52 2022] GET /identity/v3/projects/64c2a2f6f91d4b9c8d5993d28fd19631 => generated 298 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:35:52.915195 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07dc2fb0-8ea6-400d-a2ed-0b6a5e23f1e3 None None] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:35:52.915480 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07dc2fb0-8ea6-400d-a2ed-0b6a5e23f1e3 None None] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:35:52.915762 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07dc2fb0-8ea6-400d-a2ed-0b6a5e23f1e3 None None] PATH_INFO: `/` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:35:52.917150 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 59/119] 10.222.0.20 () {54 vars in 926 bytes} [Mon Aug 1 16:35:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 16:35:52.917767 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07dc2fb0-8ea6-400d-a2ed-0b6a5e23f1e3 None None] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:35:52.935939 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94639a21-5cdc-4db2-91c2-5b07d3801e22 None admin] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:35:52.974111 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3297733-981e-4ed7-b715-28b544d35f97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:35:52.975072 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3297733-981e-4ed7-b715-28b544d35f97 None admin] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:35:52.975591 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3297733-981e-4ed7-b715-28b544d35f97 None admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:35:52.976047 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3297733-981e-4ed7-b715-28b544d35f97 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:35:52.987635 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3297733-981e-4ed7-b715-28b544d35f97 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 16:35:52.992590 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application [None req-d3297733-981e-4ed7-b715-28b544d35f97 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 01 16:35:52.992590 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 16:35:52.992590 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 16:35:52.992590 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 16:35:52.992590 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 16:35:52.992590 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 16:35:52.992590 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 16:35:52.992590 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 16:35:52.992590 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 16:35:52.992590 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 16:35:52.992590 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 16:35:52.992590 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 16:35:52.992590 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 16:35:52.992590 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 16:35:52.992590 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 16:35:52.992590 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 16:35:52.993488 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 16:35:52.993488 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 16:35:52.993488 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 16:35:52.993488 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 16:35:52.993488 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 16:35:52.993488 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 16:35:52.993488 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 16:35:52.993488 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 16:35:52.993488 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 16:35:52.993488 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 16:35:52.993488 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 16:35:52.993488 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application with Lock( Aug 01 16:35:52.993488 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 16:35:52.993488 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self._enter() Aug 01 16:35:52.993488 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 16:35:52.993488 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 16:35:52.994365 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 16:35:52.994365 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self.creator() Aug 01 16:35:52.994365 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 16:35:52.994365 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application created_value = creator( Aug 01 16:35:52.994365 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 16:35:52.994365 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 16:35:52.994365 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 16:35:52.994365 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 16:35:52.994365 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 16:35:52.994365 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 16:35:52.994365 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 01 16:35:52.994365 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application Aug 01 16:35:52.994365 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 61/120] 10.222.0.20 () {56 vars in 1114 bytes} [Mon Aug 1 16:35:52 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 16:35:52.994998 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3297733-981e-4ed7-b715-28b544d35f97 None admin] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:35:53.030484 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07dc2fb0-8ea6-400d-a2ed-0b6a5e23f1e3 None None] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:35:53.065701 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c90e14eb-dc63-4efb-b098-3c3025b33f3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:35:53.066371 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90e14eb-dc63-4efb-b098-3c3025b33f3c None admin] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:35:53.066655 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90e14eb-dc63-4efb-b098-3c3025b33f3c None admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:35:53.066922 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90e14eb-dc63-4efb-b098-3c3025b33f3c None admin] PATH_INFO: `/v3/users/d96b664ac2df42d2aa31b69bc44ebf19` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:35:53.068088 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c90e14eb-dc63-4efb-b098-3c3025b33f3c None admin] RBAC: Authorizing `identity:get_user(user_id=d96b664ac2df42d2aa31b69bc44ebf19)` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:35:53.069526 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c90e14eb-dc63-4efb-b098-3c3025b33f3c None admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:35:53.071696 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 60/121] 10.222.0.20 () {56 vars in 1177 bytes} [Mon Aug 1 16:35:52 2022] GET /identity/v3/users/d96b664ac2df42d2aa31b69bc44ebf19 => generated 275 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:35:53.072455 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c90e14eb-dc63-4efb-b098-3c3025b33f3c None admin] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:35:53.091473 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG dbcounter [-] [87058] Writing DB stats keystone:SELECT=100 {{(pid=87058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:35:53.109496 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG dbcounter [-] [87057] Writing DB stats keystone:SELECT=100 {{(pid=87057) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:35:53.111368 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3297733-981e-4ed7-b715-28b544d35f97 None admin] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:35:53.145484 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f139c5-6dff-4f59-b2d3-3baffb08ebe4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:35:53.146205 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f139c5-6dff-4f59-b2d3-3baffb08ebe4 None admin] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:35:53.146497 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f139c5-6dff-4f59-b2d3-3baffb08ebe4 None admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:35:53.146768 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f139c5-6dff-4f59-b2d3-3baffb08ebe4 None admin] PATH_INFO: `/v3/users/1007f8e379b74e169b98fb7c3c0ba5b4` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:35:53.147985 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30f139c5-6dff-4f59-b2d3-3baffb08ebe4 None admin] RBAC: Authorizing `identity:get_user(user_id=1007f8e379b74e169b98fb7c3c0ba5b4)` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:35:53.149560 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30f139c5-6dff-4f59-b2d3-3baffb08ebe4 None admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:35:53.151383 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 62/122] 10.222.0.20 () {56 vars in 1177 bytes} [Mon Aug 1 16:35:52 2022] GET /identity/v3/users/1007f8e379b74e169b98fb7c3c0ba5b4 => generated 247 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:35:53.152360 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec6efe2-6433-4931-940a-f326a134e619 None None] REQUEST_METHOD: `POST` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:35:53.152642 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec6efe2-6433-4931-940a-f326a134e619 None None] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:35:53.152915 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec6efe2-6433-4931-940a-f326a134e619 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:35:53.172452 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.auth.core [None req-cec6efe2-6433-4931-940a-f326a134e619 None None] MFA Rules not processed for user `1007f8e379b74e169b98fb7c3c0ba5b4`. Rule list: `[]` (Enabled: `True`). {{(pid=87058) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:35:53.173641 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.fernet_utils [None req-cec6efe2-6433-4931-940a-f326a134e619 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:35:53.182154 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c90e14eb-dc63-4efb-b098-3c3025b33f3c None admin] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:35:53.219815 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb6ec13-cabb-4eb4-8803-2e266e81080c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:35:53.220556 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb6ec13-cabb-4eb4-8803-2e266e81080c None admin] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:35:53.220839 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb6ec13-cabb-4eb4-8803-2e266e81080c None admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:35:53.221120 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb6ec13-cabb-4eb4-8803-2e266e81080c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:35:53.222384 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbb6ec13-cabb-4eb4-8803-2e266e81080c None admin] RBAC: Adding query filter params (role.id=02c40c90308547dd8ac5aaad0da8d68c, scope.project.id=64c2a2f6f91d4b9c8d5993d28fd19631, user.id=cce885e3eda2452eadebf5bff42d3e8d) {{(pid=87057) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:35:53.223009 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbb6ec13-cabb-4eb4-8803-2e266e81080c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:35:53.224482 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbb6ec13-cabb-4eb4-8803-2e266e81080c None admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:35:53.230565 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 61/123] 10.222.0.20 () {56 vars in 1374 bytes} [Mon Aug 1 16:35:53 2022] GET /identity/v3/role_assignments?user.id=cce885e3eda2452eadebf5bff42d3e8d&scope.project.id=64c2a2f6f91d4b9c8d5993d28fd19631&role.id=02c40c90308547dd8ac5aaad0da8d68c => generated 259 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:35:53.235344 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb6ec13-cabb-4eb4-8803-2e266e81080c None admin] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:35:53.254379 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 63/124] 10.222.0.20 () {58 vars in 1023 bytes} [Mon Aug 1 16:35:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 16:35:53.255699 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec6efe2-6433-4931-940a-f326a134e619 None None] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:35:53.354280 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG dbcounter [-] [87058] Writing DB stats keystone:SELECT=100 {{(pid=87058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:35:53.374305 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec6efe2-6433-4931-940a-f326a134e619 None None] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:35:53.387237 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG dbcounter [-] [87057] Writing DB stats keystone:SELECT=100 {{(pid=87057) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:35:53.392264 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb6ec13-cabb-4eb4-8803-2e266e81080c None admin] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:35:53.417239 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f0c53b-c35c-4aa0-bcdf-36332f6e7b8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:35:53.418270 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f0c53b-c35c-4aa0-bcdf-36332f6e7b8b None admin] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:35:53.418552 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f0c53b-c35c-4aa0-bcdf-36332f6e7b8b None admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:35:53.418827 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f0c53b-c35c-4aa0-bcdf-36332f6e7b8b None admin] PATH_INFO: `/v3/projects/64c2a2f6f91d4b9c8d5993d28fd19631` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:35:53.420031 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15f0c53b-c35c-4aa0-bcdf-36332f6e7b8b None admin] RBAC: Authorizing `identity:get_project(project_id=64c2a2f6f91d4b9c8d5993d28fd19631)` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:35:53.421483 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15f0c53b-c35c-4aa0-bcdf-36332f6e7b8b None admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:35:53.423190 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 64/125] 10.222.0.20 () {56 vars in 1186 bytes} [Mon Aug 1 16:35:53 2022] GET /identity/v3/projects/64c2a2f6f91d4b9c8d5993d28fd19631 => generated 298 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:35:53.423953 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f0c53b-c35c-4aa0-bcdf-36332f6e7b8b None admin] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:35:53.443566 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-553a7954-5bbd-4b2e-9deb-e809819dbd5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:35:53.444203 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-553a7954-5bbd-4b2e-9deb-e809819dbd5f None admin] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:35:53.444591 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-553a7954-5bbd-4b2e-9deb-e809819dbd5f None admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:35:53.444932 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-553a7954-5bbd-4b2e-9deb-e809819dbd5f None admin] PATH_INFO: `/v3/roles` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:35:53.446340 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-553a7954-5bbd-4b2e-9deb-e809819dbd5f None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=87057) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:35:53.446657 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-553a7954-5bbd-4b2e-9deb-e809819dbd5f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:35:53.448505 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-553a7954-5bbd-4b2e-9deb-e809819dbd5f None admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:35:53.465599 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 62/126] 10.222.0.20 () {56 vars in 1111 bytes} [Mon Aug 1 16:35:53 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:35:53.467112 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-553a7954-5bbd-4b2e-9deb-e809819dbd5f None admin] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:35:53.547591 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f0c53b-c35c-4aa0-bcdf-36332f6e7b8b None admin] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:35:53.580571 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-553a7954-5bbd-4b2e-9deb-e809819dbd5f None admin] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:35:53.595339 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b6e3e1d-cb41-4195-801d-2941948a77e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:35:53.596289 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6e3e1d-cb41-4195-801d-2941948a77e0 None admin] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:35:53.596576 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6e3e1d-cb41-4195-801d-2941948a77e0 None admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:35:53.597288 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6e3e1d-cb41-4195-801d-2941948a77e0 None admin] PATH_INFO: `/v3/projects/64c2a2f6f91d4b9c8d5993d28fd19631` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:35:53.598779 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b6e3e1d-cb41-4195-801d-2941948a77e0 None admin] RBAC: Authorizing `identity:get_project(project_id=64c2a2f6f91d4b9c8d5993d28fd19631)` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:35:53.600871 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b6e3e1d-cb41-4195-801d-2941948a77e0 None admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:35:53.603337 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 65/127] 10.222.0.20 () {56 vars in 1186 bytes} [Mon Aug 1 16:35:53 2022] GET /identity/v3/projects/64c2a2f6f91d4b9c8d5993d28fd19631 => generated 298 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:35:53.603921 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b6e3e1d-cb41-4195-801d-2941948a77e0 None admin] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:35:53.620404 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abbb0d5b-199c-4f8f-a82b-a1af4996864a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:35:53.621116 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abbb0d5b-199c-4f8f-a82b-a1af4996864a None admin] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:35:53.621497 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abbb0d5b-199c-4f8f-a82b-a1af4996864a None admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:35:53.621868 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abbb0d5b-199c-4f8f-a82b-a1af4996864a None admin] PATH_INFO: `/v3/roles/member` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:35:53.629675 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG dbcounter [-] [87058] Writing DB stats keystone:SELECT=100 {{(pid=87058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:35:53.633181 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abbb0d5b-199c-4f8f-a82b-a1af4996864a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 16:35:53.638778 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application [None req-abbb0d5b-199c-4f8f-a82b-a1af4996864a None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 01 16:35:53.638778 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 16:35:53.638778 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 16:35:53.638778 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 16:35:53.638778 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 16:35:53.638778 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 16:35:53.638778 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 16:35:53.638778 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 16:35:53.638778 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 16:35:53.638778 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 16:35:53.638778 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 16:35:53.638778 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 16:35:53.638778 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 16:35:53.638778 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 16:35:53.638778 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 16:35:53.638778 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 16:35:53.645723 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 16:35:53.645723 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 16:35:53.645723 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 16:35:53.645723 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 16:35:53.645723 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 16:35:53.645723 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 16:35:53.645723 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 16:35:53.645723 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 16:35:53.645723 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 16:35:53.645723 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 16:35:53.645723 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 16:35:53.645723 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application with Lock( Aug 01 16:35:53.645723 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 16:35:53.645723 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self._enter() Aug 01 16:35:53.645723 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 16:35:53.645723 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 16:35:53.646533 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 16:35:53.646533 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self.creator() Aug 01 16:35:53.646533 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 16:35:53.646533 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application created_value = creator( Aug 01 16:35:53.646533 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 16:35:53.646533 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 16:35:53.646533 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 16:35:53.646533 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 16:35:53.646533 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 16:35:53.646533 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 16:35:53.646533 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 01 16:35:53.646533 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application Aug 01 16:35:53.646533 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 63/128] 10.222.0.20 () {56 vars in 1099 bytes} [Mon Aug 1 16:35:53 2022] GET /identity/v3/roles/member => generated 84 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 16:35:53.646533 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abbb0d5b-199c-4f8f-a82b-a1af4996864a None admin] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:35:53.669762 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG dbcounter [-] [87057] Writing DB stats keystone:SELECT=100 {{(pid=87057) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:35:53.747158 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b6e3e1d-cb41-4195-801d-2941948a77e0 None admin] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:35:53.760325 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abbb0d5b-199c-4f8f-a82b-a1af4996864a None admin] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:35:53.786870 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-346f8cdc-13bb-40f8-80f5-760fbc958865 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:35:53.787884 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346f8cdc-13bb-40f8-80f5-760fbc958865 None admin] REQUEST_METHOD: `PUT` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:35:53.788359 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346f8cdc-13bb-40f8-80f5-760fbc958865 None admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:35:53.788871 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346f8cdc-13bb-40f8-80f5-760fbc958865 None admin] PATH_INFO: `/v3/projects/64c2a2f6f91d4b9c8d5993d28fd19631/users/d96b664ac2df42d2aa31b69bc44ebf19/roles/0999d3da63a545f18ff3252afdc90518` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:35:53.791795 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-346f8cdc-13bb-40f8-80f5-760fbc958865 None admin] RBAC: Authorizing `identity:create_grant(project_id=64c2a2f6f91d4b9c8d5993d28fd19631, user_id=d96b664ac2df42d2aa31b69bc44ebf19, role_id=0999d3da63a545f18ff3252afdc90518)` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:35:53.794274 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-346f8cdc-13bb-40f8-80f5-760fbc958865 None admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:35:53.805002 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb36b122-f521-48b8-bbc9-119893d13c38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:35:53.806361 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb36b122-f521-48b8-bbc9-119893d13c38 None admin] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:35:53.806646 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb36b122-f521-48b8-bbc9-119893d13c38 None admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:35:53.806980 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb36b122-f521-48b8-bbc9-119893d13c38 None admin] PATH_INFO: `/v3/users/d96b664ac2df42d2aa31b69bc44ebf19` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:35:53.808227 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb36b122-f521-48b8-bbc9-119893d13c38 None admin] RBAC: Authorizing `identity:get_user(user_id=d96b664ac2df42d2aa31b69bc44ebf19)` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:35:53.809787 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb36b122-f521-48b8-bbc9-119893d13c38 None admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:35:53.815935 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 64/129] 10.222.0.20 () {56 vars in 1177 bytes} [Mon Aug 1 16:35:53 2022] GET /identity/v3/users/d96b664ac2df42d2aa31b69bc44ebf19 => generated 275 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:35:53.815935 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb36b122-f521-48b8-bbc9-119893d13c38 None admin] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:35:53.865487 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 66/130] 10.222.0.20 () {58 vars in 1439 bytes} [Mon Aug 1 16:35:53 2022] PUT /identity/v3/projects/64c2a2f6f91d4b9c8d5993d28fd19631/users/d96b664ac2df42d2aa31b69bc44ebf19/roles/0999d3da63a545f18ff3252afdc90518 => generated 0 bytes in 262 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 16:35:53.867240 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-346f8cdc-13bb-40f8-80f5-760fbc958865 None admin] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:35:53.938583 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG dbcounter [-] [87057] Writing DB stats keystone:SELECT=100 {{(pid=87057) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:35:53.955853 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb36b122-f521-48b8-bbc9-119893d13c38 None admin] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:35:53.985721 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG dbcounter [-] [87058] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:35:54.013108 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-346f8cdc-13bb-40f8-80f5-760fbc958865 None admin] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:35:54.056027 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1729a4f3-b98c-4cec-bb1e-480c77e3000e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:35:54.056739 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1729a4f3-b98c-4cec-bb1e-480c77e3000e None admin] REQUEST_METHOD: `PUT` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:35:54.057104 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1729a4f3-b98c-4cec-bb1e-480c77e3000e None admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:35:54.057395 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1729a4f3-b98c-4cec-bb1e-480c77e3000e None admin] PATH_INFO: `/v3/projects/64c2a2f6f91d4b9c8d5993d28fd19631/users/1007f8e379b74e169b98fb7c3c0ba5b4/roles/440bb943dd5b46fea038b903b6eaaf81` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:35:54.064145 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b0377d-1f39-4dd9-a5f0-ff5b845f9159 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:35:54.065245 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1729a4f3-b98c-4cec-bb1e-480c77e3000e None admin] RBAC: Authorizing `identity:create_grant(project_id=64c2a2f6f91d4b9c8d5993d28fd19631, user_id=1007f8e379b74e169b98fb7c3c0ba5b4, role_id=440bb943dd5b46fea038b903b6eaaf81)` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:35:54.066122 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b0377d-1f39-4dd9-a5f0-ff5b845f9159 None admin] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:35:54.067878 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1729a4f3-b98c-4cec-bb1e-480c77e3000e None admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:35:54.068394 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b0377d-1f39-4dd9-a5f0-ff5b845f9159 None admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:35:54.068872 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b0377d-1f39-4dd9-a5f0-ff5b845f9159 None admin] PATH_INFO: `/v3/roles` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:35:54.069831 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98b0377d-1f39-4dd9-a5f0-ff5b845f9159 None admin] RBAC: Adding query filter params (name=member) {{(pid=87058) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:35:54.073617 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98b0377d-1f39-4dd9-a5f0-ff5b845f9159 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:35:54.075424 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98b0377d-1f39-4dd9-a5f0-ff5b845f9159 None admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:35:54.090355 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 67/131] 10.222.0.20 () {56 vars in 1101 bytes} [Mon Aug 1 16:35:53 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:35:54.091597 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b0377d-1f39-4dd9-a5f0-ff5b845f9159 None admin] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:35:54.185827 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 65/132] 10.222.0.20 () {58 vars in 1439 bytes} [Mon Aug 1 16:35:53 2022] PUT /identity/v3/projects/64c2a2f6f91d4b9c8d5993d28fd19631/users/1007f8e379b74e169b98fb7c3c0ba5b4/roles/440bb943dd5b46fea038b903b6eaaf81 => generated 0 bytes in 374 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 16:35:54.187366 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1729a4f3-b98c-4cec-bb1e-480c77e3000e None admin] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:35:54.214064 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b0377d-1f39-4dd9-a5f0-ff5b845f9159 None admin] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:35:54.251987 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd9488d-44c6-4d84-bfb1-8fad54fbe98f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:35:54.252647 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd9488d-44c6-4d84-bfb1-8fad54fbe98f None admin] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:35:54.252930 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd9488d-44c6-4d84-bfb1-8fad54fbe98f None admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:35:54.253204 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd9488d-44c6-4d84-bfb1-8fad54fbe98f None admin] PATH_INFO: `/v3/projects/64c2a2f6f91d4b9c8d5993d28fd19631` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:35:54.254822 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdd9488d-44c6-4d84-bfb1-8fad54fbe98f None admin] RBAC: Authorizing `identity:get_project(project_id=64c2a2f6f91d4b9c8d5993d28fd19631)` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:35:54.256311 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdd9488d-44c6-4d84-bfb1-8fad54fbe98f None admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:35:54.258293 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 68/133] 10.222.0.20 () {56 vars in 1186 bytes} [Mon Aug 1 16:35:54 2022] GET /identity/v3/projects/64c2a2f6f91d4b9c8d5993d28fd19631 => generated 298 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:35:54.266910 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd9488d-44c6-4d84-bfb1-8fad54fbe98f None admin] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:35:54.286056 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG dbcounter [-] [87058] Writing DB stats keystone:SELECT=100 {{(pid=87058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:35:54.398293 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd9488d-44c6-4d84-bfb1-8fad54fbe98f None admin] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:35:54.415529 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1729a4f3-b98c-4cec-bb1e-480c77e3000e None admin] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:35:54.464393 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4219105-16e7-48ba-ab09-b641fb66cb43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:35:54.465700 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4219105-16e7-48ba-ab09-b641fb66cb43 None admin] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:35:54.467110 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fe04584-8b05-4dd7-8879-cb1d4374afad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:35:54.467787 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe04584-8b05-4dd7-8879-cb1d4374afad None admin] REQUEST_METHOD: `PUT` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:35:54.468065 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe04584-8b05-4dd7-8879-cb1d4374afad None admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:35:54.468339 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe04584-8b05-4dd7-8879-cb1d4374afad None admin] PATH_INFO: `/v3/projects/64c2a2f6f91d4b9c8d5993d28fd19631/users/d96b664ac2df42d2aa31b69bc44ebf19/roles/3fac71a39f3f4ed198a3579064c30268` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:35:54.469144 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4219105-16e7-48ba-ab09-b641fb66cb43 None admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:35:54.469412 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4219105-16e7-48ba-ab09-b641fb66cb43 None admin] PATH_INFO: `/v3/users/d96b664ac2df42d2aa31b69bc44ebf19` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:35:54.471101 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4219105-16e7-48ba-ab09-b641fb66cb43 None admin] RBAC: Authorizing `identity:get_user(user_id=d96b664ac2df42d2aa31b69bc44ebf19)` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:35:54.472554 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4219105-16e7-48ba-ab09-b641fb66cb43 None admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:35:54.474363 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 66/134] 10.222.0.20 () {56 vars in 1177 bytes} [Mon Aug 1 16:35:54 2022] GET /identity/v3/users/d96b664ac2df42d2aa31b69bc44ebf19 => generated 275 bytes in 287 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:35:54.475343 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92eedbd-52cb-4691-ba9f-bdef6a79f395 None None] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:35:54.475686 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92eedbd-52cb-4691-ba9f-bdef6a79f395 None None] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:35:54.475888 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92eedbd-52cb-4691-ba9f-bdef6a79f395 None None] PATH_INFO: `/` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:35:54.477731 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 67/135] 10.222.0.20 () {54 vars in 926 bytes} [Mon Aug 1 16:35:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 16:35:54.480039 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fe04584-8b05-4dd7-8879-cb1d4374afad None admin] RBAC: Authorizing `identity:create_grant(project_id=64c2a2f6f91d4b9c8d5993d28fd19631, user_id=d96b664ac2df42d2aa31b69bc44ebf19, role_id=3fac71a39f3f4ed198a3579064c30268)` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:35:54.481519 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fe04584-8b05-4dd7-8879-cb1d4374afad None admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:35:54.485862 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b92eedbd-52cb-4691-ba9f-bdef6a79f395 None None] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:35:54.506899 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG dbcounter [-] [87057] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87057) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:35:54.549854 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 69/136] 10.222.0.20 () {58 vars in 1439 bytes} [Mon Aug 1 16:35:54 2022] PUT /identity/v3/projects/64c2a2f6f91d4b9c8d5993d28fd19631/users/d96b664ac2df42d2aa31b69bc44ebf19/roles/3fac71a39f3f4ed198a3579064c30268 => generated 0 bytes in 283 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 16:35:54.551146 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e3eda7-a8a6-4be4-91bf-84cf0c748a61 None None] REQUEST_METHOD: `POST` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:35:54.551411 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e3eda7-a8a6-4be4-91bf-84cf0c748a61 None None] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:35:54.551687 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e3eda7-a8a6-4be4-91bf-84cf0c748a61 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:35:54.589789 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.auth.core [None req-07e3eda7-a8a6-4be4-91bf-84cf0c748a61 None None] MFA Rules not processed for user `1007f8e379b74e169b98fb7c3c0ba5b4`. Rule list: `[]` (Enabled: `True`). {{(pid=87058) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:35:54.592300 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.fernet_utils [None req-07e3eda7-a8a6-4be4-91bf-84cf0c748a61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:35:54.633087 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b92eedbd-52cb-4691-ba9f-bdef6a79f395 None None] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:35:54.714517 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-341764a4-24b0-4efa-ab7c-19a9f6cf1824 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:35:54.715100 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341764a4-24b0-4efa-ab7c-19a9f6cf1824 None admin] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:35:54.715382 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341764a4-24b0-4efa-ab7c-19a9f6cf1824 None admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:35:54.715658 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341764a4-24b0-4efa-ab7c-19a9f6cf1824 None admin] PATH_INFO: `/v3/projects/0c14cf5caf744ecf88597ead2702ec70` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:35:54.717837 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-341764a4-24b0-4efa-ab7c-19a9f6cf1824 None admin] RBAC: Authorizing `identity:get_project(project_id=0c14cf5caf744ecf88597ead2702ec70)` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:35:54.719360 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-341764a4-24b0-4efa-ab7c-19a9f6cf1824 None admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:35:54.722411 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG dbcounter [-] [87058] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:35:54.723138 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 68/137] 10.222.0.20 () {56 vars in 1186 bytes} [Mon Aug 1 16:35:54 2022] GET /identity/v3/projects/0c14cf5caf744ecf88597ead2702ec70 => generated 312 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:35:54.729829 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-341764a4-24b0-4efa-ab7c-19a9f6cf1824 None admin] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:35:54.730907 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 70/138] 10.222.0.20 () {58 vars in 1023 bytes} [Mon Aug 1 16:35:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 16:35:54.779950 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e3eda7-a8a6-4be4-91bf-84cf0c748a61 None None] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:35:54.891640 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG dbcounter [-] [87057] Writing DB stats keystone:SELECT=100 {{(pid=87057) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:35:54.927735 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-341764a4-24b0-4efa-ab7c-19a9f6cf1824 None admin] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:35:54.968155 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e3eda7-a8a6-4be4-91bf-84cf0c748a61 None None] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:35:54.983854 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebfdd12e-50ec-45b7-a961-a588c01972c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:35:54.984142 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebfdd12e-50ec-45b7-a961-a588c01972c4 None admin] REQUEST_METHOD: `PUT` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:35:54.984422 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebfdd12e-50ec-45b7-a961-a588c01972c4 None admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:35:54.987789 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebfdd12e-50ec-45b7-a961-a588c01972c4 None admin] PATH_INFO: `/v3/projects/0c14cf5caf744ecf88597ead2702ec70/users/d96b664ac2df42d2aa31b69bc44ebf19/roles/0999d3da63a545f18ff3252afdc90518` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:35:54.987789 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebfdd12e-50ec-45b7-a961-a588c01972c4 None admin] RBAC: Authorizing `identity:create_grant(project_id=0c14cf5caf744ecf88597ead2702ec70, user_id=d96b664ac2df42d2aa31b69bc44ebf19, role_id=0999d3da63a545f18ff3252afdc90518)` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:35:54.988382 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebfdd12e-50ec-45b7-a961-a588c01972c4 None admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:35:55.034984 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98636b58-9ab2-424b-95b0-48b8b0da95d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:35:55.035983 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98636b58-9ab2-424b-95b0-48b8b0da95d8 None admin] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:35:55.036494 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98636b58-9ab2-424b-95b0-48b8b0da95d8 None admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:35:55.036954 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98636b58-9ab2-424b-95b0-48b8b0da95d8 None admin] PATH_INFO: `/v3/domains/default` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:35:55.042737 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98636b58-9ab2-424b-95b0-48b8b0da95d8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:35:55.044440 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98636b58-9ab2-424b-95b0-48b8b0da95d8 None admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:35:55.046283 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 71/139] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Aug 1 16:35:54 2022] GET /identity/v3/domains/default => generated 199 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 16:35:55.051875 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98636b58-9ab2-424b-95b0-48b8b0da95d8 None admin] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:35:55.065741 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 69/140] 10.222.0.20 () {58 vars in 1439 bytes} [Mon Aug 1 16:35:54 2022] PUT /identity/v3/projects/0c14cf5caf744ecf88597ead2702ec70/users/d96b664ac2df42d2aa31b69bc44ebf19/roles/0999d3da63a545f18ff3252afdc90518 => generated 0 bytes in 336 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 16:35:55.134455 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG dbcounter [-] [87058] Writing DB stats keystone:SELECT=100 {{(pid=87058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:35:55.170714 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98636b58-9ab2-424b-95b0-48b8b0da95d8 None admin] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:35:55.209185 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2815d07b-f6b2-4e6c-b72b-5f5935893bb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:35:55.210199 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2815d07b-f6b2-4e6c-b72b-5f5935893bb8 None admin] REQUEST_METHOD: `POST` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:35:55.210697 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2815d07b-f6b2-4e6c-b72b-5f5935893bb8 None admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:35:55.211157 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2815d07b-f6b2-4e6c-b72b-5f5935893bb8 None admin] PATH_INFO: `/v3/users` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:35:55.212139 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2815d07b-f6b2-4e6c-b72b-5f5935893bb8 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:35:55.213760 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2815d07b-f6b2-4e6c-b72b-5f5935893bb8 None admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:35:55.361083 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 72/141] 10.222.0.20 () {60 vars in 1132 bytes} [Mon Aug 1 16:35:55 2022] POST /identity/v3/users => generated 283 bytes in 310 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 16:35:56.046305 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea82f8bb-30d8-4c8a-a5f0-16aa82199a98 None None] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:35:56.046800 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea82f8bb-30d8-4c8a-a5f0-16aa82199a98 None None] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:35:56.046874 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea82f8bb-30d8-4c8a-a5f0-16aa82199a98 None None] PATH_INFO: `/` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:35:56.048225 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 70/142] 10.222.0.20 () {54 vars in 926 bytes} [Mon Aug 1 16:35:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 16:35:56.053641 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f51b6e5-ee7b-4cc9-8007-e0a1eaa0e309 None None] REQUEST_METHOD: `POST` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:35:56.054202 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f51b6e5-ee7b-4cc9-8007-e0a1eaa0e309 None None] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:35:56.054672 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f51b6e5-ee7b-4cc9-8007-e0a1eaa0e309 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:35:56.083008 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.auth.core [None req-1f51b6e5-ee7b-4cc9-8007-e0a1eaa0e309 None None] MFA Rules not processed for user `1007f8e379b74e169b98fb7c3c0ba5b4`. Rule list: `[]` (Enabled: `True`). {{(pid=87058) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:35:56.084571 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.fernet_utils [None req-1f51b6e5-ee7b-4cc9-8007-e0a1eaa0e309 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:35:56.168479 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 73/143] 10.222.0.20 () {58 vars in 1023 bytes} [Mon Aug 1 16:35:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 16:35:56.207102 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea82f8bb-30d8-4c8a-a5f0-16aa82199a98 None None] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:35:56.324426 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea82f8bb-30d8-4c8a-a5f0-16aa82199a98 None None] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:35:56.365213 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ebfa23d-01ec-4d91-9463-5b93ae8a179f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:35:56.365890 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ebfa23d-01ec-4d91-9463-5b93ae8a179f None admin] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:35:56.366211 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ebfa23d-01ec-4d91-9463-5b93ae8a179f None admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:35:56.366490 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ebfa23d-01ec-4d91-9463-5b93ae8a179f None admin] PATH_INFO: `/v3/roles/reader` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:35:56.383710 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ebfa23d-01ec-4d91-9463-5b93ae8a179f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 16:35:56.388704 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application [None req-8ebfa23d-01ec-4d91-9463-5b93ae8a179f None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 01 16:35:56.388704 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 16:35:56.388704 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 16:35:56.388704 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 16:35:56.388704 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 16:35:56.388704 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 16:35:56.388704 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 16:35:56.388704 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 16:35:56.388704 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 16:35:56.388704 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 16:35:56.388704 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 16:35:56.388704 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 16:35:56.388704 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 16:35:56.388704 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 16:35:56.388704 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 16:35:56.388704 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 16:35:56.389555 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 16:35:56.389555 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 16:35:56.389555 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 16:35:56.389555 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 16:35:56.389555 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 16:35:56.389555 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 16:35:56.389555 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 16:35:56.389555 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 16:35:56.389555 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 16:35:56.389555 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 16:35:56.389555 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 16:35:56.389555 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application with Lock( Aug 01 16:35:56.389555 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 16:35:56.389555 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self._enter() Aug 01 16:35:56.389555 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 16:35:56.389555 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 16:35:56.390324 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 16:35:56.390324 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self.creator() Aug 01 16:35:56.390324 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 16:35:56.390324 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application created_value = creator( Aug 01 16:35:56.390324 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 16:35:56.390324 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 16:35:56.390324 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 16:35:56.390324 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 16:35:56.390324 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 16:35:56.390324 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 16:35:56.390324 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 01 16:35:56.390324 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application Aug 01 16:35:56.390324 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 71/144] 10.222.0.20 () {56 vars in 1099 bytes} [Mon Aug 1 16:35:56 2022] GET /identity/v3/roles/reader => generated 84 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 16:35:56.396240 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f51b6e5-ee7b-4cc9-8007-e0a1eaa0e309 None None] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:35:56.473191 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG dbcounter [-] [87058] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:35:56.527990 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f51b6e5-ee7b-4cc9-8007-e0a1eaa0e309 None None] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:35:56.569959 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b73548b7-4c0f-4df5-a1e1-193cc0d2a9ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:35:56.570634 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73548b7-4c0f-4df5-a1e1-193cc0d2a9ec None admin] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:35:56.571502 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73548b7-4c0f-4df5-a1e1-193cc0d2a9ec None admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:35:56.571774 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73548b7-4c0f-4df5-a1e1-193cc0d2a9ec None admin] PATH_INFO: `/v3/roles` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:35:56.572584 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b73548b7-4c0f-4df5-a1e1-193cc0d2a9ec None admin] RBAC: Adding query filter params (name=reader) {{(pid=87058) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:35:56.572884 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b73548b7-4c0f-4df5-a1e1-193cc0d2a9ec None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:35:56.574300 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b73548b7-4c0f-4df5-a1e1-193cc0d2a9ec None admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:35:56.583611 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 74/145] 10.222.0.20 () {56 vars in 1101 bytes} [Mon Aug 1 16:35:56 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:35:56.589493 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ebfa23d-01ec-4d91-9463-5b93ae8a179f None admin] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:35:56.609166 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG dbcounter [-] [87057] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87057) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:35:56.727036 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ebfa23d-01ec-4d91-9463-5b93ae8a179f None admin] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:35:56.770539 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac060d5c-3bd3-4dc8-8dc3-3157bf474e5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:35:56.771201 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac060d5c-3bd3-4dc8-8dc3-3157bf474e5e None admin] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:35:56.771485 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac060d5c-3bd3-4dc8-8dc3-3157bf474e5e None admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:35:56.771765 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac060d5c-3bd3-4dc8-8dc3-3157bf474e5e None admin] PATH_INFO: `/v3/users/cce885e3eda2452eadebf5bff42d3e8d` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:35:56.772963 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac060d5c-3bd3-4dc8-8dc3-3157bf474e5e None admin] RBAC: Authorizing `identity:get_user(user_id=cce885e3eda2452eadebf5bff42d3e8d)` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:35:56.775100 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac060d5c-3bd3-4dc8-8dc3-3157bf474e5e None admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:35:56.776755 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 72/146] 10.222.0.20 () {56 vars in 1177 bytes} [Mon Aug 1 16:35:56 2022] GET /identity/v3/users/cce885e3eda2452eadebf5bff42d3e8d => generated 289 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:35:56.782483 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b73548b7-4c0f-4df5-a1e1-193cc0d2a9ec None admin] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:35:56.817873 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93cd67ab-6ae7-4bf1-be53-1a2b8b06406a None None] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:35:56.818190 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93cd67ab-6ae7-4bf1-be53-1a2b8b06406a None None] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:35:56.818472 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93cd67ab-6ae7-4bf1-be53-1a2b8b06406a None None] PATH_INFO: `/` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:35:56.820107 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 73/147] 10.222.0.20 () {54 vars in 926 bytes} [Mon Aug 1 16:35:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 16:35:56.825328 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ffeaad9-1e42-4c6e-a343-b2a89092d3f7 None None] REQUEST_METHOD: `POST` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:35:56.825622 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ffeaad9-1e42-4c6e-a343-b2a89092d3f7 None None] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:35:56.825908 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ffeaad9-1e42-4c6e-a343-b2a89092d3f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:35:56.855422 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.auth.core [None req-9ffeaad9-1e42-4c6e-a343-b2a89092d3f7 None None] MFA Rules not processed for user `1007f8e379b74e169b98fb7c3c0ba5b4`. Rule list: `[]` (Enabled: `True`). {{(pid=87057) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:35:56.856670 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.fernet_utils [None req-9ffeaad9-1e42-4c6e-a343-b2a89092d3f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87057) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:35:56.938944 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b73548b7-4c0f-4df5-a1e1-193cc0d2a9ec None admin] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:35:56.974239 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 74/148] 10.222.0.20 () {58 vars in 1023 bytes} [Mon Aug 1 16:35:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 01 16:35:56.974836 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG dbcounter [-] [87058] Writing DB stats keystone:SELECT=100 {{(pid=87058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:35:56.992113 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-601c2d07-60f0-46ad-9faf-8a00f63715e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:35:56.992113 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601c2d07-60f0-46ad-9faf-8a00f63715e2 None admin] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:35:56.992113 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601c2d07-60f0-46ad-9faf-8a00f63715e2 None admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:35:56.992113 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601c2d07-60f0-46ad-9faf-8a00f63715e2 None admin] PATH_INFO: `/v3/projects/64c2a2f6f91d4b9c8d5993d28fd19631` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:35:56.993134 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-601c2d07-60f0-46ad-9faf-8a00f63715e2 None admin] RBAC: Authorizing `identity:get_project(project_id=64c2a2f6f91d4b9c8d5993d28fd19631)` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:35:56.993134 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-601c2d07-60f0-46ad-9faf-8a00f63715e2 None admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:35:56.993134 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 75/149] 10.222.0.20 () {56 vars in 1186 bytes} [Mon Aug 1 16:35:56 2022] GET /identity/v3/projects/64c2a2f6f91d4b9c8d5993d28fd19631 => generated 298 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:35:56.998893 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ffeaad9-1e42-4c6e-a343-b2a89092d3f7 None None] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:35:57.002531 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG dbcounter [-] [87057] Writing DB stats keystone:SELECT=100 {{(pid=87057) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:35:57.009216 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-601c2d07-60f0-46ad-9faf-8a00f63715e2 None admin] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:35:57.179454 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ffeaad9-1e42-4c6e-a343-b2a89092d3f7 None None] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:35:57.221760 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-601c2d07-60f0-46ad-9faf-8a00f63715e2 None admin] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:35:57.249315 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-587d6699-90f2-47eb-bc7b-2c487edd5ce2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:35:57.249928 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587d6699-90f2-47eb-bc7b-2c487edd5ce2 None admin] REQUEST_METHOD: `PUT` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:35:57.250233 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587d6699-90f2-47eb-bc7b-2c487edd5ce2 None admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:35:57.250510 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587d6699-90f2-47eb-bc7b-2c487edd5ce2 None admin] PATH_INFO: `/v3/projects/64c2a2f6f91d4b9c8d5993d28fd19631/users/cce885e3eda2452eadebf5bff42d3e8d/roles/02c40c90308547dd8ac5aaad0da8d68c` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:35:57.253861 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-587d6699-90f2-47eb-bc7b-2c487edd5ce2 None admin] RBAC: Authorizing `identity:create_grant(project_id=64c2a2f6f91d4b9c8d5993d28fd19631, user_id=cce885e3eda2452eadebf5bff42d3e8d, role_id=02c40c90308547dd8ac5aaad0da8d68c)` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:35:57.256457 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-587d6699-90f2-47eb-bc7b-2c487edd5ce2 None admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:35:57.292220 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e5407a-963f-4c46-bd37-1f2e5688ccc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:35:57.293257 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e5407a-963f-4c46-bd37-1f2e5688ccc4 None admin] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:35:57.293738 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e5407a-963f-4c46-bd37-1f2e5688ccc4 None admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:35:57.294251 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e5407a-963f-4c46-bd37-1f2e5688ccc4 None admin] PATH_INFO: `/v3/roles/member` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:35:57.314922 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31e5407a-963f-4c46-bd37-1f2e5688ccc4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 16:35:57.321296 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application [None req-31e5407a-963f-4c46-bd37-1f2e5688ccc4 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 01 16:35:57.321296 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 16:35:57.321296 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 16:35:57.321296 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 16:35:57.321296 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 16:35:57.321296 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 16:35:57.321296 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 16:35:57.321296 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 16:35:57.321296 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 16:35:57.321296 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 16:35:57.321296 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 16:35:57.321296 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 16:35:57.321296 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 16:35:57.321296 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 16:35:57.321296 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 16:35:57.321296 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 16:35:57.322240 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 16:35:57.322240 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 16:35:57.322240 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 16:35:57.322240 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 16:35:57.322240 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 16:35:57.322240 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 16:35:57.322240 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 16:35:57.322240 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 16:35:57.322240 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 16:35:57.322240 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 16:35:57.322240 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 16:35:57.322240 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application with Lock( Aug 01 16:35:57.322240 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 16:35:57.322240 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self._enter() Aug 01 16:35:57.322240 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 16:35:57.322240 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 16:35:57.323444 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 16:35:57.323444 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self.creator() Aug 01 16:35:57.323444 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 16:35:57.323444 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application created_value = creator( Aug 01 16:35:57.323444 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 16:35:57.323444 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 16:35:57.323444 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 16:35:57.323444 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 16:35:57.323444 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 16:35:57.323444 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 16:35:57.323444 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 01 16:35:57.323444 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application Aug 01 16:35:57.323444 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 76/150] 10.222.0.20 () {56 vars in 1099 bytes} [Mon Aug 1 16:35:57 2022] GET /identity/v3/roles/member => generated 84 bytes in 314 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 01 16:35:57.324335 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3c87d2-ece4-4cd3-90b2-b2ca11999703 None None] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:35:57.324335 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3c87d2-ece4-4cd3-90b2-b2ca11999703 None None] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:35:57.324699 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3c87d2-ece4-4cd3-90b2-b2ca11999703 None None] PATH_INFO: `/` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:35:57.326019 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 77/151] 10.222.0.20 () {54 vars in 926 bytes} [Mon Aug 1 16:35:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 16:35:57.330873 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db3c87d2-ece4-4cd3-90b2-b2ca11999703 None None] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:35:57.349151 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 75/152] 10.222.0.20 () {58 vars in 1439 bytes} [Mon Aug 1 16:35:56 2022] PUT /identity/v3/projects/64c2a2f6f91d4b9c8d5993d28fd19631/users/cce885e3eda2452eadebf5bff42d3e8d/roles/02c40c90308547dd8ac5aaad0da8d68c => generated 0 bytes in 351 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 16:35:57.350456 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eedd62c-204e-4321-b270-185e87746380 None None] REQUEST_METHOD: `POST` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:35:57.350746 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eedd62c-204e-4321-b270-185e87746380 None None] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:35:57.351023 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eedd62c-204e-4321-b270-185e87746380 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:35:57.387906 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.auth.core [None req-9eedd62c-204e-4321-b270-185e87746380 None None] MFA Rules not processed for user `1007f8e379b74e169b98fb7c3c0ba5b4`. Rule list: `[]` (Enabled: `True`). {{(pid=87057) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:35:57.389146 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.fernet_utils [None req-9eedd62c-204e-4321-b270-185e87746380 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87057) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:35:57.422607 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG dbcounter [-] [87058] Writing DB stats keystone:SELECT=100 {{(pid=87058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:35:57.544013 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG dbcounter [-] [87057] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87057) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:35:57.560559 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 76/153] 10.222.0.20 () {58 vars in 1023 bytes} [Mon Aug 1 16:35:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 211 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 16:35:57.588581 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db3c87d2-ece4-4cd3-90b2-b2ca11999703 None None] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:35:57.613698 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eedd62c-204e-4321-b270-185e87746380 None None] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:35:57.643592 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccf41cf1-80b2-4592-953e-9644ac40791a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:35:57.644239 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf41cf1-80b2-4592-953e-9644ac40791a None admin] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:35:57.644522 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf41cf1-80b2-4592-953e-9644ac40791a None admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:35:57.644798 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf41cf1-80b2-4592-953e-9644ac40791a None admin] PATH_INFO: `/v3/roles` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:35:57.645544 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccf41cf1-80b2-4592-953e-9644ac40791a None admin] RBAC: Adding query filter params (name=member) {{(pid=87058) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:35:57.645841 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccf41cf1-80b2-4592-953e-9644ac40791a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:35:57.647274 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccf41cf1-80b2-4592-953e-9644ac40791a None admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:35:57.663705 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 78/154] 10.222.0.20 () {56 vars in 1101 bytes} [Mon Aug 1 16:35:57 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 333 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:35:57.671444 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccf41cf1-80b2-4592-953e-9644ac40791a None admin] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:35:57.842124 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eedd62c-204e-4321-b270-185e87746380 None None] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:35:57.869738 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccf41cf1-80b2-4592-953e-9644ac40791a None admin] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:35:57.875402 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG dbcounter [-] [87058] Writing DB stats keystone:SELECT=100 {{(pid=87058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:35:57.938937 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9abd818f-eb59-4030-a74c-52eab5dc2e48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:35:57.939603 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9abd818f-eb59-4030-a74c-52eab5dc2e48 None admin] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:35:57.939911 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9abd818f-eb59-4030-a74c-52eab5dc2e48 None admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:35:57.940145 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9abd818f-eb59-4030-a74c-52eab5dc2e48 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:35:57.954769 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d2bfe9d-5494-4d16-9f37-de72d259ddd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:35:57.955730 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2bfe9d-5494-4d16-9f37-de72d259ddd5 None admin] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:35:57.956188 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2bfe9d-5494-4d16-9f37-de72d259ddd5 None admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:35:57.956606 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2bfe9d-5494-4d16-9f37-de72d259ddd5 None admin] PATH_INFO: `/v3/users/d96b664ac2df42d2aa31b69bc44ebf19` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:35:57.960277 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9abd818f-eb59-4030-a74c-52eab5dc2e48 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 16:35:57.961098 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d2bfe9d-5494-4d16-9f37-de72d259ddd5 None admin] RBAC: Authorizing `identity:get_user(user_id=d96b664ac2df42d2aa31b69bc44ebf19)` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:35:57.963452 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d2bfe9d-5494-4d16-9f37-de72d259ddd5 None admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:35:57.965674 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 79/155] 10.222.0.20 () {56 vars in 1177 bytes} [Mon Aug 1 16:35:57 2022] GET /identity/v3/users/d96b664ac2df42d2aa31b69bc44ebf19 => generated 275 bytes in 294 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:35:57.965892 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application [None req-9abd818f-eb59-4030-a74c-52eab5dc2e48 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 16:35:57.965892 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 16:35:57.965892 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 16:35:57.965892 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 16:35:57.965892 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 16:35:57.965892 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 16:35:57.965892 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 16:35:57.965892 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 16:35:57.965892 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 16:35:57.965892 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 16:35:57.965892 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 16:35:57.965892 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 16:35:57.965892 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 16:35:57.965892 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 16:35:57.965892 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 16:35:57.965892 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 16:35:57.967010 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 16:35:57.967010 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 16:35:57.967010 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 16:35:57.967010 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 16:35:57.967010 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 16:35:57.967010 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 16:35:57.967010 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 16:35:57.967010 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 16:35:57.967010 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 16:35:57.967010 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 16:35:57.967010 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 16:35:57.967010 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application with Lock( Aug 01 16:35:57.967010 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 16:35:57.967010 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self._enter() Aug 01 16:35:57.967010 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 16:35:57.967010 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 16:35:57.967874 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 16:35:57.967874 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self.creator() Aug 01 16:35:57.967874 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 16:35:57.967874 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application created_value = creator( Aug 01 16:35:57.967874 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 16:35:57.967874 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 16:35:57.967874 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 16:35:57.967874 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 16:35:57.967874 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 16:35:57.967874 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 16:35:57.967874 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 16:35:57.967874 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application Aug 01 16:35:57.967874 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 77/156] 10.222.0.20 () {56 vars in 1096 bytes} [Mon Aug 1 16:35:57 2022] GET /identity/v3/roles/admin => generated 83 bytes in 354 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 01 16:35:57.972888 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d2bfe9d-5494-4d16-9f37-de72d259ddd5 None admin] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:35:57.973855 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9abd818f-eb59-4030-a74c-52eab5dc2e48 None admin] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:35:58.112674 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d2bfe9d-5494-4d16-9f37-de72d259ddd5 None admin] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:35:58.113639 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG dbcounter [-] [87057] Writing DB stats keystone:SELECT=100 {{(pid=87057) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:35:58.172844 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-748ef17e-f565-45ae-9638-eb16b3f22c3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:35:58.173744 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748ef17e-f565-45ae-9638-eb16b3f22c3e None admin] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:35:58.174200 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748ef17e-f565-45ae-9638-eb16b3f22c3e None admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:35:58.174627 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748ef17e-f565-45ae-9638-eb16b3f22c3e None admin] PATH_INFO: `/v3/roles` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:35:58.175491 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-748ef17e-f565-45ae-9638-eb16b3f22c3e None admin] RBAC: Adding query filter params (name=admin) {{(pid=87058) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:35:58.175941 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-748ef17e-f565-45ae-9638-eb16b3f22c3e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:35:58.177469 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-748ef17e-f565-45ae-9638-eb16b3f22c3e None admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:35:58.186085 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9abd818f-eb59-4030-a74c-52eab5dc2e48 None admin] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:35:58.197889 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 80/157] 10.222.0.20 () {56 vars in 1099 bytes} [Mon Aug 1 16:35:57 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:35:58.216500 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-748ef17e-f565-45ae-9638-eb16b3f22c3e None admin] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:35:58.235132 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33bb439b-d870-4017-9fe0-b7cde015b1ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:35:58.235464 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33bb439b-d870-4017-9fe0-b7cde015b1ea None admin] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:35:58.235750 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33bb439b-d870-4017-9fe0-b7cde015b1ea None admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:35:58.236028 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33bb439b-d870-4017-9fe0-b7cde015b1ea None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:35:58.286080 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG dbcounter [-] [87058] Writing DB stats keystone:SELECT=100 {{(pid=87058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:35:58.322323 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33bb439b-d870-4017-9fe0-b7cde015b1ea None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:35:58.323852 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33bb439b-d870-4017-9fe0-b7cde015b1ea None admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:35:58.368643 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-748ef17e-f565-45ae-9638-eb16b3f22c3e None admin] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:35:58.441877 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dee39af-a5ce-4eb2-9257-28092386c36e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:35:58.442828 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dee39af-a5ce-4eb2-9257-28092386c36e None admin] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:35:58.443261 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dee39af-a5ce-4eb2-9257-28092386c36e None admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:35:58.443675 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dee39af-a5ce-4eb2-9257-28092386c36e None admin] PATH_INFO: `/v3/users/1007f8e379b74e169b98fb7c3c0ba5b4` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:35:58.446162 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dee39af-a5ce-4eb2-9257-28092386c36e None admin] RBAC: Authorizing `identity:get_user(user_id=1007f8e379b74e169b98fb7c3c0ba5b4)` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:35:58.447859 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dee39af-a5ce-4eb2-9257-28092386c36e None admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:35:58.449592 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG dbcounter [-] [87057] Writing DB stats keystone:SELECT=100 {{(pid=87057) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:35:58.451002 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 81/158] 10.222.0.20 () {56 vars in 1177 bytes} [Mon Aug 1 16:35:58 2022] GET /identity/v3/users/1007f8e379b74e169b98fb7c3c0ba5b4 => generated 247 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 16:35:58.456678 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dee39af-a5ce-4eb2-9257-28092386c36e None admin] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:35:58.535395 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 78/159] 10.222.0.20 () {58 vars in 1282 bytes} [Mon Aug 1 16:35:57 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 562 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 01 16:35:58.541423 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33bb439b-d870-4017-9fe0-b7cde015b1ea None admin] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:35:58.591721 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dee39af-a5ce-4eb2-9257-28092386c36e None admin] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:35:58.593662 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG dbcounter [-] [87058] Writing DB stats keystone:SELECT=100 {{(pid=87058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:35:58.641758 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4827685-b4ab-4af3-a1c3-d0fdf4acb33b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:35:58.642337 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4827685-b4ab-4af3-a1c3-d0fdf4acb33b None admin] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:35:58.642619 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4827685-b4ab-4af3-a1c3-d0fdf4acb33b None admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:35:58.643157 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4827685-b4ab-4af3-a1c3-d0fdf4acb33b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:35:58.733106 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33bb439b-d870-4017-9fe0-b7cde015b1ea None admin] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:35:58.780629 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4827685-b4ab-4af3-a1c3-d0fdf4acb33b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:35:58.782130 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4827685-b4ab-4af3-a1c3-d0fdf4acb33b None admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:35:58.798788 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-524f6586-1e32-45d6-8bb8-85ee51adaeaf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:35:58.799443 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524f6586-1e32-45d6-8bb8-85ee51adaeaf None admin] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:35:58.799723 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524f6586-1e32-45d6-8bb8-85ee51adaeaf None admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:35:58.799994 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524f6586-1e32-45d6-8bb8-85ee51adaeaf None admin] PATH_INFO: `/v3/projects/64c2a2f6f91d4b9c8d5993d28fd19631` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:35:58.801252 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-524f6586-1e32-45d6-8bb8-85ee51adaeaf None admin] RBAC: Authorizing `identity:get_project(project_id=64c2a2f6f91d4b9c8d5993d28fd19631)` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:35:58.802731 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-524f6586-1e32-45d6-8bb8-85ee51adaeaf None admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:35:58.804411 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 79/160] 10.222.0.20 () {56 vars in 1186 bytes} [Mon Aug 1 16:35:58 2022] GET /identity/v3/projects/64c2a2f6f91d4b9c8d5993d28fd19631 => generated 298 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 16:35:58.806949 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e13035-01d8-4a21-839a-f5c34a4c0be5 None None] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:35:58.807725 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e13035-01d8-4a21-839a-f5c34a4c0be5 None None] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:35:58.808209 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e13035-01d8-4a21-839a-f5c34a4c0be5 None None] PATH_INFO: `/` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:35:58.809671 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 80/161] 10.222.0.20 () {54 vars in 926 bytes} [Mon Aug 1 16:35:58 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 16:35:58.816460 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4e13035-01d8-4a21-839a-f5c34a4c0be5 None None] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:35:58.859227 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG dbcounter [-] [87057] Writing DB stats keystone:SELECT=100 {{(pid=87057) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:35:58.996448 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 82/162] 10.222.0.20 () {58 vars in 1282 bytes} [Mon Aug 1 16:35:58 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 540 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 01 16:35:58.998154 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9826535d-b1bd-4527-bfc7-dc4240acc8d8 None None] REQUEST_METHOD: `POST` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:35:58.998739 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9826535d-b1bd-4527-bfc7-dc4240acc8d8 None None] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:35:58.999213 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4e13035-01d8-4a21-839a-f5c34a4c0be5 None None] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:35:58.999423 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9826535d-b1bd-4527-bfc7-dc4240acc8d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:35:59.019405 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG dbcounter [-] [87058] Writing DB stats keystone:SELECT=100 {{(pid=87058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:35:59.043750 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.auth.core [None req-9826535d-b1bd-4527-bfc7-dc4240acc8d8 None None] MFA Rules not processed for user `1007f8e379b74e169b98fb7c3c0ba5b4`. Rule list: `[]` (Enabled: `True`). {{(pid=87058) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:35:59.048078 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.fernet_utils [None req-9826535d-b1bd-4527-bfc7-dc4240acc8d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:35:59.058141 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc9791b-7d5e-4247-a009-2e35c2efc097 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:35:59.058771 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc9791b-7d5e-4247-a009-2e35c2efc097 None admin] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:35:59.059054 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc9791b-7d5e-4247-a009-2e35c2efc097 None admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:35:59.059332 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc9791b-7d5e-4247-a009-2e35c2efc097 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:35:59.060138 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffc9791b-7d5e-4247-a009-2e35c2efc097 None admin] RBAC: Adding query filter params (role.id=0999d3da63a545f18ff3252afdc90518, scope.project.id=64c2a2f6f91d4b9c8d5993d28fd19631, user.id=d96b664ac2df42d2aa31b69bc44ebf19) {{(pid=87057) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:35:59.060448 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffc9791b-7d5e-4247-a009-2e35c2efc097 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:35:59.062739 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffc9791b-7d5e-4247-a009-2e35c2efc097 None admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:35:59.069214 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 81/163] 10.222.0.20 () {56 vars in 1374 bytes} [Mon Aug 1 16:35:58 2022] GET /identity/v3/role_assignments?user.id=d96b664ac2df42d2aa31b69bc44ebf19&scope.project.id=64c2a2f6f91d4b9c8d5993d28fd19631&role.id=0999d3da63a545f18ff3252afdc90518 => generated 608 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:35:59.070023 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc9791b-7d5e-4247-a009-2e35c2efc097 None admin] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:35:59.200030 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG dbcounter [-] [87057] Writing DB stats keystone:SELECT=100 {{(pid=87057) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:35:59.215975 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 83/164] 10.222.0.20 () {58 vars in 1023 bytes} [Mon Aug 1 16:35:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 219 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 16:35:59.217896 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb2c7f3-98be-4480-a37e-642cde08ece7 None None] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:35:59.218477 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb2c7f3-98be-4480-a37e-642cde08ece7 None None] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:35:59.218960 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb2c7f3-98be-4480-a37e-642cde08ece7 None None] PATH_INFO: `/` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:35:59.220562 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 84/165] 10.222.0.20 () {54 vars in 926 bytes} [Mon Aug 1 16:35:59 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 16:35:59.222506 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc9791b-7d5e-4247-a009-2e35c2efc097 None admin] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:35:59.229280 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3638c326-4225-41a9-b3b5-371ab203d6b0 None None] REQUEST_METHOD: `POST` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:35:59.229947 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3638c326-4225-41a9-b3b5-371ab203d6b0 None None] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:35:59.230500 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3638c326-4225-41a9-b3b5-371ab203d6b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:35:59.262910 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.auth.core [None req-3638c326-4225-41a9-b3b5-371ab203d6b0 None None] MFA Rules not processed for user `1007f8e379b74e169b98fb7c3c0ba5b4`. Rule list: `[]` (Enabled: `True`). {{(pid=87058) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:35:59.264534 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.fernet_utils [None req-3638c326-4225-41a9-b3b5-371ab203d6b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:35:59.280199 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac9d27a-8abe-4e3f-b1b9-c48992395bdf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:35:59.280863 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac9d27a-8abe-4e3f-b1b9-c48992395bdf None admin] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:35:59.281148 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac9d27a-8abe-4e3f-b1b9-c48992395bdf None admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:35:59.281425 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac9d27a-8abe-4e3f-b1b9-c48992395bdf None admin] PATH_INFO: `/v3/projects/64c2a2f6f91d4b9c8d5993d28fd19631` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:35:59.286387 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ac9d27a-8abe-4e3f-b1b9-c48992395bdf None admin] RBAC: Authorizing `identity:get_project(project_id=64c2a2f6f91d4b9c8d5993d28fd19631)` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:35:59.288042 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ac9d27a-8abe-4e3f-b1b9-c48992395bdf None admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:35:59.290338 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 82/166] 10.222.0.20 () {56 vars in 1186 bytes} [Mon Aug 1 16:35:59 2022] GET /identity/v3/projects/64c2a2f6f91d4b9c8d5993d28fd19631 => generated 298 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:35:59.291343 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac9d27a-8abe-4e3f-b1b9-c48992395bdf None admin] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:35:59.394902 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 85/167] 10.222.0.20 () {58 vars in 1023 bytes} [Mon Aug 1 16:35:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 16:35:59.396506 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3638c326-4225-41a9-b3b5-371ab203d6b0 None None] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:35:59.467998 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac9d27a-8abe-4e3f-b1b9-c48992395bdf None admin] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:35:59.508446 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG dbcounter [-] [87058] Writing DB stats keystone:SELECT=100 {{(pid=87058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:35:59.521970 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7362ff1-65f1-4a72-a184-8debdc52b483 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:35:59.522656 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7362ff1-65f1-4a72-a184-8debdc52b483 None admin] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:35:59.522938 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7362ff1-65f1-4a72-a184-8debdc52b483 None admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:35:59.541033 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7362ff1-65f1-4a72-a184-8debdc52b483 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:35:59.543384 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7362ff1-65f1-4a72-a184-8debdc52b483 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 16:35:59.548613 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application [None req-b7362ff1-65f1-4a72-a184-8debdc52b483 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 01 16:35:59.548613 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 16:35:59.548613 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 16:35:59.548613 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 16:35:59.548613 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 16:35:59.548613 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 16:35:59.548613 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 16:35:59.548613 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 16:35:59.548613 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 16:35:59.548613 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 16:35:59.548613 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 16:35:59.548613 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 16:35:59.548613 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 16:35:59.548613 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 16:35:59.548613 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 16:35:59.548613 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 16:35:59.549676 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 16:35:59.549676 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 16:35:59.549676 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 16:35:59.549676 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 16:35:59.549676 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 16:35:59.549676 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 16:35:59.549676 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 16:35:59.549676 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 16:35:59.549676 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 16:35:59.549676 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 16:35:59.549676 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 16:35:59.549676 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application with Lock( Aug 01 16:35:59.549676 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 16:35:59.549676 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self._enter() Aug 01 16:35:59.549676 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 16:35:59.549676 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 16:35:59.550556 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 16:35:59.550556 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self.creator() Aug 01 16:35:59.550556 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 16:35:59.550556 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application created_value = creator( Aug 01 16:35:59.550556 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 16:35:59.550556 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 16:35:59.550556 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 16:35:59.550556 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 16:35:59.550556 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 16:35:59.550556 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 16:35:59.550556 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 01 16:35:59.550556 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application Aug 01 16:35:59.550556 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 83/168] 10.222.0.20 () {56 vars in 1114 bytes} [Mon Aug 1 16:35:59 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 259 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 16:35:59.551220 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7362ff1-65f1-4a72-a184-8debdc52b483 None admin] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:35:59.585711 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG dbcounter [-] [87057] Writing DB stats keystone:SELECT=100 {{(pid=87057) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:35:59.589918 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3638c326-4225-41a9-b3b5-371ab203d6b0 None None] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:35:59.656626 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f22cf9-af00-418c-b930-0e20d0e0d2d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:35:59.657259 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f22cf9-af00-418c-b930-0e20d0e0d2d5 None admin] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:35:59.657537 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f22cf9-af00-418c-b930-0e20d0e0d2d5 None admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:35:59.657810 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f22cf9-af00-418c-b930-0e20d0e0d2d5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:35:59.658646 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9f22cf9-af00-418c-b930-0e20d0e0d2d5 None admin] RBAC: Adding query filter params (role.id=440bb943dd5b46fea038b903b6eaaf81, scope.project.id=64c2a2f6f91d4b9c8d5993d28fd19631, user.id=1007f8e379b74e169b98fb7c3c0ba5b4) {{(pid=87058) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:35:59.658954 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9f22cf9-af00-418c-b930-0e20d0e0d2d5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:35:59.660534 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9f22cf9-af00-418c-b930-0e20d0e0d2d5 None admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:35:59.671681 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 86/169] 10.222.0.20 () {56 vars in 1374 bytes} [Mon Aug 1 16:35:59 2022] GET /identity/v3/role_assignments?user.id=1007f8e379b74e169b98fb7c3c0ba5b4&scope.project.id=64c2a2f6f91d4b9c8d5993d28fd19631&role.id=440bb943dd5b46fea038b903b6eaaf81 => generated 608 bytes in 276 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:35:59.672599 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f22cf9-af00-418c-b930-0e20d0e0d2d5 None admin] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:35:59.700174 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7362ff1-65f1-4a72-a184-8debdc52b483 None admin] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:35:59.753555 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcf0c108-c425-4e7e-812c-d67e8e2eff06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:35:59.754243 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf0c108-c425-4e7e-812c-d67e8e2eff06 None admin] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:35:59.754526 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf0c108-c425-4e7e-812c-d67e8e2eff06 None admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:35:59.754838 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf0c108-c425-4e7e-812c-d67e8e2eff06 None admin] PATH_INFO: `/v3/roles/member` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:35:59.771445 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcf0c108-c425-4e7e-812c-d67e8e2eff06 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 16:35:59.777485 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application [None req-fcf0c108-c425-4e7e-812c-d67e8e2eff06 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 01 16:35:59.777485 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 16:35:59.777485 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 16:35:59.777485 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 16:35:59.777485 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 16:35:59.777485 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 16:35:59.777485 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 16:35:59.777485 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 16:35:59.777485 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 16:35:59.777485 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 16:35:59.777485 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 16:35:59.777485 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 16:35:59.777485 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 16:35:59.777485 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 16:35:59.777485 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 16:35:59.777485 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 16:35:59.778554 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 16:35:59.778554 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 16:35:59.778554 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 16:35:59.778554 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 16:35:59.778554 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 16:35:59.778554 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 16:35:59.778554 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 16:35:59.778554 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 16:35:59.778554 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 16:35:59.778554 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 16:35:59.778554 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 16:35:59.778554 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application with Lock( Aug 01 16:35:59.778554 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 16:35:59.778554 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self._enter() Aug 01 16:35:59.778554 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 16:35:59.778554 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 16:35:59.779349 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 16:35:59.779349 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self.creator() Aug 01 16:35:59.779349 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 16:35:59.779349 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application created_value = creator( Aug 01 16:35:59.779349 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 16:35:59.779349 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 16:35:59.779349 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 16:35:59.779349 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 16:35:59.779349 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 16:35:59.779349 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 16:35:59.779349 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 01 16:35:59.779349 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application Aug 01 16:35:59.783024 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 84/170] 10.222.0.20 () {56 vars in 1099 bytes} [Mon Aug 1 16:35:59 2022] GET /identity/v3/roles/member => generated 84 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 16:35:59.788982 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcf0c108-c425-4e7e-812c-d67e8e2eff06 None admin] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:35:59.877074 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f22cf9-af00-418c-b930-0e20d0e0d2d5 None admin] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:35:59.931075 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG dbcounter [-] [87058] Writing DB stats keystone:SELECT=100 {{(pid=87058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:35:59.953385 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2f4cac8-d623-4e6e-a8cb-bbde8eac94d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:35:59.954023 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f4cac8-d623-4e6e-a8cb-bbde8eac94d5 None admin] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:35:59.954331 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f4cac8-d623-4e6e-a8cb-bbde8eac94d5 None admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:35:59.954605 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f4cac8-d623-4e6e-a8cb-bbde8eac94d5 None admin] PATH_INFO: `/v3/roles` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:35:59.955344 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2f4cac8-d623-4e6e-a8cb-bbde8eac94d5 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=87058) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:35:59.955806 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2f4cac8-d623-4e6e-a8cb-bbde8eac94d5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:35:59.957228 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2f4cac8-d623-4e6e-a8cb-bbde8eac94d5 None admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:35:59.970927 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 87/171] 10.222.0.20 () {56 vars in 1111 bytes} [Mon Aug 1 16:35:59 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 299 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:35:59.974720 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG dbcounter [-] [87057] Writing DB stats keystone:SELECT=100 {{(pid=87057) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:35:59.978688 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2f4cac8-d623-4e6e-a8cb-bbde8eac94d5 None admin] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:36:00.039906 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcf0c108-c425-4e7e-812c-d67e8e2eff06 None admin] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:36:00.110840 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d9b2f18-3545-4c40-b507-bd42d7f466c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:36:00.111491 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9b2f18-3545-4c40-b507-bd42d7f466c1 None admin] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:36:00.111777 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9b2f18-3545-4c40-b507-bd42d7f466c1 None admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:36:00.112057 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9b2f18-3545-4c40-b507-bd42d7f466c1 None admin] PATH_INFO: `/v3/roles` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:36:00.112803 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d9b2f18-3545-4c40-b507-bd42d7f466c1 None admin] RBAC: Adding query filter params (name=member) {{(pid=87057) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:36:00.113115 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d9b2f18-3545-4c40-b507-bd42d7f466c1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:36:00.114590 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d9b2f18-3545-4c40-b507-bd42d7f466c1 None admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:36:00.128734 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 85/172] 10.222.0.20 () {56 vars in 1101 bytes} [Mon Aug 1 16:35:59 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 340 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:36:00.134448 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d9b2f18-3545-4c40-b507-bd42d7f466c1 None admin] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:36:00.162779 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2f4cac8-d623-4e6e-a8cb-bbde8eac94d5 None admin] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:36:00.239316 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25fa75cf-fa7e-43f4-a3ca-c712d28dbb6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:36:00.239989 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25fa75cf-fa7e-43f4-a3ca-c712d28dbb6a None admin] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:36:00.240328 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25fa75cf-fa7e-43f4-a3ca-c712d28dbb6a None admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:36:00.240613 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25fa75cf-fa7e-43f4-a3ca-c712d28dbb6a None admin] PATH_INFO: `/v3/users/d96b664ac2df42d2aa31b69bc44ebf19` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:36:00.243949 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25fa75cf-fa7e-43f4-a3ca-c712d28dbb6a None admin] RBAC: Authorizing `identity:get_user(user_id=d96b664ac2df42d2aa31b69bc44ebf19)` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:36:00.245450 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25fa75cf-fa7e-43f4-a3ca-c712d28dbb6a None admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:36:00.249587 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 88/173] 10.222.0.20 () {56 vars in 1177 bytes} [Mon Aug 1 16:35:59 2022] GET /identity/v3/users/d96b664ac2df42d2aa31b69bc44ebf19 => generated 275 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:36:00.253977 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25fa75cf-fa7e-43f4-a3ca-c712d28dbb6a None admin] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:36:00.317345 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d9b2f18-3545-4c40-b507-bd42d7f466c1 None admin] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:36:00.339096 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG dbcounter [-] [87058] Writing DB stats keystone:SELECT=100 {{(pid=87058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:36:00.356639 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b4e36e8-2e70-4c0a-a734-6655bdc772eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:36:00.357550 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4e36e8-2e70-4c0a-a734-6655bdc772eb None admin] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:36:00.357977 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4e36e8-2e70-4c0a-a734-6655bdc772eb None admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:36:00.358897 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4e36e8-2e70-4c0a-a734-6655bdc772eb None admin] PATH_INFO: `/v3/users/d96b664ac2df42d2aa31b69bc44ebf19` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:36:00.360344 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b4e36e8-2e70-4c0a-a734-6655bdc772eb None admin] RBAC: Authorizing `identity:get_user(user_id=d96b664ac2df42d2aa31b69bc44ebf19)` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:36:00.362115 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b4e36e8-2e70-4c0a-a734-6655bdc772eb None admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:36:00.364131 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 86/174] 10.222.0.20 () {56 vars in 1177 bytes} [Mon Aug 1 16:36:00 2022] GET /identity/v3/users/d96b664ac2df42d2aa31b69bc44ebf19 => generated 275 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 16:36:00.366077 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c4d6bb-c6af-44a9-a25e-ba59143df77e None None] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:36:00.370707 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c4d6bb-c6af-44a9-a25e-ba59143df77e None None] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:36:00.370707 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c4d6bb-c6af-44a9-a25e-ba59143df77e None None] PATH_INFO: `/` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:36:00.370707 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 87/175] 10.222.0.20 () {54 vars in 926 bytes} [Mon Aug 1 16:36:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 16:36:00.372555 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4fbab4-24e5-41a2-9f6b-c1c73d6ffca1 None None] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:36:00.373043 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4fbab4-24e5-41a2-9f6b-c1c73d6ffca1 None None] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:36:00.373526 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4fbab4-24e5-41a2-9f6b-c1c73d6ffca1 None None] PATH_INFO: `/` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:36:00.375145 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 88/176] 10.222.0.20 () {54 vars in 926 bytes} [Mon Aug 1 16:36:00 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 16:36:00.376156 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e4fbab4-24e5-41a2-9f6b-c1c73d6ffca1 None None] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:36:00.394411 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG dbcounter [-] [87057] Writing DB stats keystone:SELECT=100 {{(pid=87057) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:36:00.398428 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25fa75cf-fa7e-43f4-a3ca-c712d28dbb6a None admin] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:36:00.441859 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ec3337-632d-40bc-99b3-9ca6b55bbeb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:36:00.442520 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ec3337-632d-40bc-99b3-9ca6b55bbeb9 None admin] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:36:00.442814 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ec3337-632d-40bc-99b3-9ca6b55bbeb9 None admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:36:00.445359 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ec3337-632d-40bc-99b3-9ca6b55bbeb9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:36:00.507040 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e4fbab4-24e5-41a2-9f6b-c1c73d6ffca1 None None] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:36:00.529328 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73ec3337-632d-40bc-99b3-9ca6b55bbeb9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:36:00.531179 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73ec3337-632d-40bc-99b3-9ca6b55bbeb9 None admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:36:00.551695 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d1fbb4-7c2a-448f-84ea-3604023e76e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:36:00.552320 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d1fbb4-7c2a-448f-84ea-3604023e76e5 None admin] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:36:00.552601 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d1fbb4-7c2a-448f-84ea-3604023e76e5 None admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:36:00.552872 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d1fbb4-7c2a-448f-84ea-3604023e76e5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:36:00.592302 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG dbcounter [-] [87058] Writing DB stats keystone:SELECT=100 {{(pid=87058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:36:00.640720 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29d1fbb4-7c2a-448f-84ea-3604023e76e5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:36:00.642236 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29d1fbb4-7c2a-448f-84ea-3604023e76e5 None admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:36:00.643238 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 89/177] 10.222.0.20 () {58 vars in 1282 bytes} [Mon Aug 1 16:36:00 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 390 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 01 16:36:00.644966 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e4c3e6-45f9-4d68-a172-a581a295eaf8 None None] REQUEST_METHOD: `POST` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:36:00.645472 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e4c3e6-45f9-4d68-a172-a581a295eaf8 None None] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:36:00.645970 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e4c3e6-45f9-4d68-a172-a581a295eaf8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:36:00.646349 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG dbcounter [-] [87057] Writing DB stats keystone:SELECT=100 {{(pid=87057) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:36:00.666855 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.auth.core [None req-93e4c3e6-45f9-4d68-a172-a581a295eaf8 None None] MFA Rules not processed for user `1007f8e379b74e169b98fb7c3c0ba5b4`. Rule list: `[]` (Enabled: `True`). {{(pid=87058) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:36:00.668487 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.fernet_utils [None req-93e4c3e6-45f9-4d68-a172-a581a295eaf8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:36:00.744002 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 90/178] 10.222.0.20 () {58 vars in 1023 bytes} [Mon Aug 1 16:36:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 16:36:00.745745 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4908975d-cebd-4d1e-8327-71270995302f None None] REQUEST_METHOD: `POST` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:36:00.746268 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4908975d-cebd-4d1e-8327-71270995302f None None] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:36:00.746788 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4908975d-cebd-4d1e-8327-71270995302f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:36:00.766656 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.auth.core [None req-4908975d-cebd-4d1e-8327-71270995302f None None] MFA Rules not processed for user `1007f8e379b74e169b98fb7c3c0ba5b4`. Rule list: `[]` (Enabled: `True`). {{(pid=87058) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:36:00.768437 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.fernet_utils [None req-4908975d-cebd-4d1e-8327-71270995302f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:36:00.770387 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 89/179] 10.222.0.20 () {58 vars in 1282 bytes} [Mon Aug 1 16:36:00 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 394 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 01 16:36:00.771245 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d1fbb4-7c2a-448f-84ea-3604023e76e5 None admin] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:36:00.842041 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 91/180] 10.222.0.20 () {58 vars in 1023 bytes} [Mon Aug 1 16:36:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 16:36:00.850261 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00404aeb-565a-4e48-b17a-5856aaff7b6f None None] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:36:00.850261 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00404aeb-565a-4e48-b17a-5856aaff7b6f None None] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:36:00.850261 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00404aeb-565a-4e48-b17a-5856aaff7b6f None None] PATH_INFO: `/` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:36:00.850261 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 92/181] 10.222.0.20 () {54 vars in 926 bytes} [Mon Aug 1 16:36:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 16:36:00.850261 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00404aeb-565a-4e48-b17a-5856aaff7b6f None None] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:36:00.871537 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG dbcounter [-] [87058] Writing DB stats keystone:SELECT=100 {{(pid=87058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:36:00.894931 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d1fbb4-7c2a-448f-84ea-3604023e76e5 None admin] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:36:00.905010 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG dbcounter [-] [87057] Writing DB stats keystone:SELECT=100 {{(pid=87057) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:36:00.930478 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9624f28f-ca92-4ec4-a606-11cd91ec86fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:36:00.931239 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9624f28f-ca92-4ec4-a606-11cd91ec86fd None admin] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:36:00.931562 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9624f28f-ca92-4ec4-a606-11cd91ec86fd None admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:36:00.931898 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9624f28f-ca92-4ec4-a606-11cd91ec86fd None admin] PATH_INFO: `/v3/projects/64c2a2f6f91d4b9c8d5993d28fd19631` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:36:00.933120 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9624f28f-ca92-4ec4-a606-11cd91ec86fd None admin] RBAC: Authorizing `identity:get_project(project_id=64c2a2f6f91d4b9c8d5993d28fd19631)` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:36:00.934722 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9624f28f-ca92-4ec4-a606-11cd91ec86fd None admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:36:00.936556 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 90/182] 10.222.0.20 () {56 vars in 1186 bytes} [Mon Aug 1 16:36:00 2022] GET /identity/v3/projects/64c2a2f6f91d4b9c8d5993d28fd19631 => generated 298 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:36:00.937393 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9624f28f-ca92-4ec4-a606-11cd91ec86fd None admin] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:36:00.970402 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00404aeb-565a-4e48-b17a-5856aaff7b6f None None] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:36:01.007189 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe3dc07-edea-4899-8a67-c4818223a9ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:36:01.007843 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe3dc07-edea-4899-8a67-c4818223a9ac None admin] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:36:01.008179 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe3dc07-edea-4899-8a67-c4818223a9ac None admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:36:01.008415 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe3dc07-edea-4899-8a67-c4818223a9ac None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:36:01.019157 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffe3dc07-edea-4899-8a67-c4818223a9ac None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 16:36:01.024651 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application [None req-ffe3dc07-edea-4899-8a67-c4818223a9ac None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 01 16:36:01.024651 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 16:36:01.024651 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 16:36:01.024651 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 16:36:01.024651 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 16:36:01.024651 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 16:36:01.024651 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 16:36:01.024651 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 16:36:01.024651 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 16:36:01.024651 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 16:36:01.024651 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 16:36:01.024651 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 16:36:01.024651 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 16:36:01.024651 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 16:36:01.024651 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 16:36:01.024651 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 16:36:01.025539 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 16:36:01.025539 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 16:36:01.025539 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 16:36:01.025539 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 16:36:01.025539 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 16:36:01.025539 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 16:36:01.025539 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 16:36:01.025539 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 16:36:01.025539 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 16:36:01.025539 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 16:36:01.025539 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 16:36:01.025539 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application with Lock( Aug 01 16:36:01.025539 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 16:36:01.025539 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self._enter() Aug 01 16:36:01.025539 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 16:36:01.025539 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 16:36:01.027442 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 16:36:01.027442 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self.creator() Aug 01 16:36:01.027442 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 16:36:01.027442 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application created_value = creator( Aug 01 16:36:01.027442 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 16:36:01.027442 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 16:36:01.027442 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 16:36:01.027442 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 16:36:01.027442 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 16:36:01.027442 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 16:36:01.027442 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 01 16:36:01.027442 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application Aug 01 16:36:01.027442 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 93/183] 10.222.0.20 () {56 vars in 1114 bytes} [Mon Aug 1 16:36:00 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 16:36:01.027442 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78817dbb-2a31-4e51-a02f-fc0df0f6dd92 None None] REQUEST_METHOD: `POST` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:36:01.028180 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78817dbb-2a31-4e51-a02f-fc0df0f6dd92 None None] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:36:01.028180 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78817dbb-2a31-4e51-a02f-fc0df0f6dd92 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:36:01.045707 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.auth.core [None req-78817dbb-2a31-4e51-a02f-fc0df0f6dd92 None None] MFA Rules not processed for user `1007f8e379b74e169b98fb7c3c0ba5b4`. Rule list: `[]` (Enabled: `True`). {{(pid=87058) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:36:01.046913 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.fernet_utils [None req-78817dbb-2a31-4e51-a02f-fc0df0f6dd92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:36:01.055672 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9624f28f-ca92-4ec4-a606-11cd91ec86fd None admin] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:36:01.093253 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4509702a-0098-4193-b683-b36c0f7c16be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:36:01.093900 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4509702a-0098-4193-b683-b36c0f7c16be None admin] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:36:01.094214 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4509702a-0098-4193-b683-b36c0f7c16be None admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:36:01.094491 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4509702a-0098-4193-b683-b36c0f7c16be None admin] PATH_INFO: `/v3/projects/0c14cf5caf744ecf88597ead2702ec70` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:36:01.095667 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4509702a-0098-4193-b683-b36c0f7c16be None admin] RBAC: Authorizing `identity:get_project(project_id=0c14cf5caf744ecf88597ead2702ec70)` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:36:01.097132 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4509702a-0098-4193-b683-b36c0f7c16be None admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:36:01.098918 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 91/184] 10.222.0.20 () {56 vars in 1186 bytes} [Mon Aug 1 16:36:00 2022] GET /identity/v3/projects/0c14cf5caf744ecf88597ead2702ec70 => generated 312 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:36:01.099601 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4509702a-0098-4193-b683-b36c0f7c16be None admin] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:36:01.119145 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 94/185] 10.222.0.20 () {58 vars in 1023 bytes} [Mon Aug 1 16:36:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 16:36:01.120008 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78817dbb-2a31-4e51-a02f-fc0df0f6dd92 None None] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:36:01.128035 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG dbcounter [-] [87058] Writing DB stats keystone:SELECT=100 {{(pid=87058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:36:01.155573 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG dbcounter [-] [87057] Writing DB stats keystone:SELECT=100 {{(pid=87057) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:36:01.211984 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4509702a-0098-4193-b683-b36c0f7c16be None admin] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:36:01.218821 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78817dbb-2a31-4e51-a02f-fc0df0f6dd92 None None] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:36:01.243856 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34e016e8-0ce5-4fb9-b092-ec94bdb9ec4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:36:01.244485 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e016e8-0ce5-4fb9-b092-ec94bdb9ec4b None admin] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:36:01.244758 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e016e8-0ce5-4fb9-b092-ec94bdb9ec4b None admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:36:01.245026 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e016e8-0ce5-4fb9-b092-ec94bdb9ec4b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:36:01.252032 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b73e13c4-789e-4fc1-9623-7fa076cb0331 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:36:01.252651 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73e13c4-789e-4fc1-9623-7fa076cb0331 None admin] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:36:01.252927 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73e13c4-789e-4fc1-9623-7fa076cb0331 None admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:36:01.253196 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73e13c4-789e-4fc1-9623-7fa076cb0331 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:36:01.253985 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b73e13c4-789e-4fc1-9623-7fa076cb0331 None admin] RBAC: Adding query filter params (role.id=3fac71a39f3f4ed198a3579064c30268, scope.project.id=64c2a2f6f91d4b9c8d5993d28fd19631, user.id=d96b664ac2df42d2aa31b69bc44ebf19) {{(pid=87058) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:36:01.254328 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b73e13c4-789e-4fc1-9623-7fa076cb0331 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:36:01.254976 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34e016e8-0ce5-4fb9-b092-ec94bdb9ec4b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 16:36:01.255765 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b73e13c4-789e-4fc1-9623-7fa076cb0331 None admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:36:01.260918 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application [None req-34e016e8-0ce5-4fb9-b092-ec94bdb9ec4b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 16:36:01.260918 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 16:36:01.260918 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 16:36:01.260918 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 16:36:01.260918 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 16:36:01.260918 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 16:36:01.260918 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 16:36:01.260918 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 16:36:01.260918 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 16:36:01.260918 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 16:36:01.260918 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 16:36:01.260918 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 16:36:01.260918 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 16:36:01.260918 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 16:36:01.260918 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 16:36:01.260918 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 16:36:01.261851 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 16:36:01.261851 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 16:36:01.261851 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 16:36:01.261851 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 16:36:01.261851 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 16:36:01.261851 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 16:36:01.261851 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 16:36:01.261851 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 16:36:01.261851 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 16:36:01.261851 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 16:36:01.261851 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 16:36:01.261851 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application with Lock( Aug 01 16:36:01.261851 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 16:36:01.261851 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self._enter() Aug 01 16:36:01.261851 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 16:36:01.261851 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 16:36:01.262688 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 16:36:01.262688 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self.creator() Aug 01 16:36:01.262688 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 16:36:01.262688 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application created_value = creator( Aug 01 16:36:01.262688 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 16:36:01.262688 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 16:36:01.262688 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 16:36:01.262688 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 16:36:01.262688 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 16:36:01.262688 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 16:36:01.262688 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 16:36:01.262688 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application Aug 01 16:36:01.262688 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 92/186] 10.222.0.20 () {56 vars in 1096 bytes} [Mon Aug 1 16:36:01 2022] GET /identity/v3/roles/admin => generated 83 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 16:36:01.263345 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34e016e8-0ce5-4fb9-b092-ec94bdb9ec4b None admin] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:36:01.266197 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 95/187] 10.222.0.20 () {56 vars in 1374 bytes} [Mon Aug 1 16:36:01 2022] GET /identity/v3/role_assignments?user.id=d96b664ac2df42d2aa31b69bc44ebf19&scope.project.id=64c2a2f6f91d4b9c8d5993d28fd19631&role.id=3fac71a39f3f4ed198a3579064c30268 => generated 608 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:36:01.268094 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e817db6-8e8d-4df7-9425-3a6493c4ea18 None None] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:36:01.268443 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e817db6-8e8d-4df7-9425-3a6493c4ea18 None None] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:36:01.268654 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e817db6-8e8d-4df7-9425-3a6493c4ea18 None None] PATH_INFO: `/` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:36:01.274312 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 96/188] 10.222.0.20 () {54 vars in 926 bytes} [Mon Aug 1 16:36:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 16:36:01.274312 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e817db6-8e8d-4df7-9425-3a6493c4ea18 None None] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:36:01.351271 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG dbcounter [-] [87058] Writing DB stats keystone:SELECT=100 {{(pid=87058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:36:01.387400 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e817db6-8e8d-4df7-9425-3a6493c4ea18 None None] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:36:01.387684 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG dbcounter [-] [87057] Writing DB stats keystone:SELECT=100 {{(pid=87057) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:36:01.390765 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34e016e8-0ce5-4fb9-b092-ec94bdb9ec4b None admin] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:36:01.429505 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64c041d8-045b-445e-9399-7910e66924d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:36:01.430455 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c041d8-045b-445e-9399-7910e66924d1 None admin] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:36:01.430657 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c041d8-045b-445e-9399-7910e66924d1 None admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:36:01.430936 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c041d8-045b-445e-9399-7910e66924d1 None admin] PATH_INFO: `/v3/roles` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:36:01.431672 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64c041d8-045b-445e-9399-7910e66924d1 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=87057) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:36:01.431978 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64c041d8-045b-445e-9399-7910e66924d1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:36:01.433378 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64c041d8-045b-445e-9399-7910e66924d1 None admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:36:01.443499 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 93/189] 10.222.0.20 () {56 vars in 1111 bytes} [Mon Aug 1 16:36:01 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:36:01.444418 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64c041d8-045b-445e-9399-7910e66924d1 None admin] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:36:01.454976 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c271655-ac38-404a-a871-9c639dbc2b7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:36:01.456146 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c271655-ac38-404a-a871-9c639dbc2b7d None admin] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:36:01.456428 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c271655-ac38-404a-a871-9c639dbc2b7d None admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:36:01.456712 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c271655-ac38-404a-a871-9c639dbc2b7d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:36:01.457501 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c271655-ac38-404a-a871-9c639dbc2b7d None admin] RBAC: Adding query filter params (role.id=0999d3da63a545f18ff3252afdc90518, scope.project.id=0c14cf5caf744ecf88597ead2702ec70, user.id=d96b664ac2df42d2aa31b69bc44ebf19) {{(pid=87058) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:36:01.457940 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c271655-ac38-404a-a871-9c639dbc2b7d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:36:01.460689 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c271655-ac38-404a-a871-9c639dbc2b7d None admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:36:01.469954 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 97/190] 10.222.0.20 () {56 vars in 1374 bytes} [Mon Aug 1 16:36:01 2022] GET /identity/v3/role_assignments?user.id=d96b664ac2df42d2aa31b69bc44ebf19&scope.project.id=0c14cf5caf744ecf88597ead2702ec70&role.id=0999d3da63a545f18ff3252afdc90518 => generated 608 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:36:01.470818 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c271655-ac38-404a-a871-9c639dbc2b7d None admin] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:36:01.565702 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64c041d8-045b-445e-9399-7910e66924d1 None admin] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:36:01.593917 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c271655-ac38-404a-a871-9c639dbc2b7d None admin] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:36:01.605409 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f6ea02c-6c1c-4447-ae35-0ef68fdf0277 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:36:01.606080 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6ea02c-6c1c-4447-ae35-0ef68fdf0277 None admin] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:36:01.606360 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6ea02c-6c1c-4447-ae35-0ef68fdf0277 None admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:36:01.606632 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6ea02c-6c1c-4447-ae35-0ef68fdf0277 None admin] PATH_INFO: `/v3/domains/default` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:36:01.607797 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f6ea02c-6c1c-4447-ae35-0ef68fdf0277 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:36:01.609262 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f6ea02c-6c1c-4447-ae35-0ef68fdf0277 None admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:36:01.615530 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 94/191] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Aug 1 16:36:01 2022] GET /identity/v3/domains/default => generated 199 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:36:01.615530 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42956edf-c749-41b9-9f93-2490c3507618 None None] REQUEST_METHOD: `POST` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:36:01.615530 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42956edf-c749-41b9-9f93-2490c3507618 None None] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:36:01.615530 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42956edf-c749-41b9-9f93-2490c3507618 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:36:01.631901 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.auth.core [None req-42956edf-c749-41b9-9f93-2490c3507618 None None] MFA Rules not processed for user `1007f8e379b74e169b98fb7c3c0ba5b4`. Rule list: `[]` (Enabled: `True`). {{(pid=87057) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:36:01.633716 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.fernet_utils [None req-42956edf-c749-41b9-9f93-2490c3507618 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87057) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:36:01.641837 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d1a5bc-e34c-43ab-9950-0783e54f2164 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:36:01.642482 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d1a5bc-e34c-43ab-9950-0783e54f2164 None admin] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:36:01.643020 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d1a5bc-e34c-43ab-9950-0783e54f2164 None admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:36:01.643288 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d1a5bc-e34c-43ab-9950-0783e54f2164 None admin] PATH_INFO: `/v3/roles` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:36:01.644007 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43d1a5bc-e34c-43ab-9950-0783e54f2164 None admin] RBAC: Adding query filter params (name=admin) {{(pid=87058) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:36:01.644310 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43d1a5bc-e34c-43ab-9950-0783e54f2164 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:36:01.646562 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43d1a5bc-e34c-43ab-9950-0783e54f2164 None admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:36:01.648121 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG dbcounter [-] [87058] Writing DB stats keystone:SELECT=100 {{(pid=87058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:36:01.657448 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 98/192] 10.222.0.20 () {56 vars in 1099 bytes} [Mon Aug 1 16:36:01 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:36:01.658646 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d1a5bc-e34c-43ab-9950-0783e54f2164 None admin] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:36:01.665985 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG dbcounter [-] [87057] Writing DB stats keystone:SELECT=100 {{(pid=87057) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:36:01.719097 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 95/193] 10.222.0.20 () {58 vars in 1023 bytes} [Mon Aug 1 16:36:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 16:36:01.719942 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42956edf-c749-41b9-9f93-2490c3507618 None None] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:36:01.771714 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d1a5bc-e34c-43ab-9950-0783e54f2164 None admin] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:36:01.806142 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f39b14d-586a-41fd-8d0e-5ed6e523f62c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:36:01.806782 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f39b14d-586a-41fd-8d0e-5ed6e523f62c None admin] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:36:01.807053 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f39b14d-586a-41fd-8d0e-5ed6e523f62c None admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:36:01.807317 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f39b14d-586a-41fd-8d0e-5ed6e523f62c None admin] PATH_INFO: `/v3/users/bccb7b2b19304928888200525af9748a` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:36:01.819834 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f39b14d-586a-41fd-8d0e-5ed6e523f62c None admin] RBAC: Authorizing `identity:get_user(user_id=bccb7b2b19304928888200525af9748a)` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:36:01.822424 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f39b14d-586a-41fd-8d0e-5ed6e523f62c None admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:36:01.824065 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 99/194] 10.222.0.20 () {56 vars in 1177 bytes} [Mon Aug 1 16:36:01 2022] GET /identity/v3/users/bccb7b2b19304928888200525af9748a => generated 283 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:36:01.824824 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f39b14d-586a-41fd-8d0e-5ed6e523f62c None admin] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:36:01.829790 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42956edf-c749-41b9-9f93-2490c3507618 None None] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:36:01.863536 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04e08478-d355-47f2-8ab2-f87a5200155a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:36:01.864152 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e08478-d355-47f2-8ab2-f87a5200155a None admin] REQUEST_METHOD: `POST` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:36:01.864437 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e08478-d355-47f2-8ab2-f87a5200155a None admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:36:01.864710 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e08478-d355-47f2-8ab2-f87a5200155a None admin] PATH_INFO: `/v3/users` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:36:01.865391 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04e08478-d355-47f2-8ab2-f87a5200155a None admin] RBAC: Authorizing `identity:create_user()` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:36:01.866884 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04e08478-d355-47f2-8ab2-f87a5200155a None admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:36:01.882019 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG dbcounter [-] [87058] Writing DB stats keystone:SELECT=100 {{(pid=87058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:36:01.926093 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f39b14d-586a-41fd-8d0e-5ed6e523f62c None admin] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:36:01.959445 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2615fbad-6686-48d9-9862-d9efe58b5797 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:36:01.960594 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2615fbad-6686-48d9-9862-d9efe58b5797 None admin] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:36:01.960594 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2615fbad-6686-48d9-9862-d9efe58b5797 None admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:36:01.960594 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2615fbad-6686-48d9-9862-d9efe58b5797 None admin] PATH_INFO: `/v3/users/bccb7b2b19304928888200525af9748a` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:36:01.962408 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2615fbad-6686-48d9-9862-d9efe58b5797 None admin] RBAC: Authorizing `identity:get_user(user_id=bccb7b2b19304928888200525af9748a)` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:36:01.963858 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2615fbad-6686-48d9-9862-d9efe58b5797 None admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:36:01.965484 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 100/195] 10.222.0.20 () {56 vars in 1177 bytes} [Mon Aug 1 16:36:01 2022] GET /identity/v3/users/bccb7b2b19304928888200525af9748a => generated 283 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:36:01.966281 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2615fbad-6686-48d9-9862-d9efe58b5797 None admin] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:36:01.972007 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 96/196] 10.222.0.20 () {60 vars in 1132 bytes} [Mon Aug 1 16:36:01 2022] POST /identity/v3/users => generated 297 bytes in 252 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 16:36:01.973169 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04e08478-d355-47f2-8ab2-f87a5200155a None admin] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:36:02.013966 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG dbcounter [-] [87057] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87057) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:36:02.080499 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2615fbad-6686-48d9-9862-d9efe58b5797 None admin] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:36:02.106339 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG dbcounter [-] [87058] Writing DB stats keystone:SELECT=100 {{(pid=87058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:36:02.108892 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04e08478-d355-47f2-8ab2-f87a5200155a None admin] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:36:02.118314 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f565db7e-a1fa-4c96-a996-61cee761c941 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:36:02.118976 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f565db7e-a1fa-4c96-a996-61cee761c941 None admin] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:36:02.119258 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f565db7e-a1fa-4c96-a996-61cee761c941 None admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:36:02.119534 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f565db7e-a1fa-4c96-a996-61cee761c941 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:36:02.130815 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f565db7e-a1fa-4c96-a996-61cee761c941 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 16:36:02.136940 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application [None req-f565db7e-a1fa-4c96-a996-61cee761c941 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 01 16:36:02.136940 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 16:36:02.136940 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 16:36:02.136940 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 16:36:02.136940 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 16:36:02.136940 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 16:36:02.136940 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 16:36:02.136940 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 16:36:02.136940 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 16:36:02.136940 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 16:36:02.136940 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 16:36:02.136940 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 16:36:02.136940 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 16:36:02.136940 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 16:36:02.136940 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 16:36:02.136940 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 16:36:02.137857 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 16:36:02.137857 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 16:36:02.137857 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 16:36:02.137857 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 16:36:02.137857 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 16:36:02.137857 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 16:36:02.137857 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 16:36:02.137857 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 16:36:02.137857 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 16:36:02.137857 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 16:36:02.137857 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 16:36:02.137857 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application with Lock( Aug 01 16:36:02.137857 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 16:36:02.137857 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self._enter() Aug 01 16:36:02.137857 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 16:36:02.137857 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 16:36:02.138700 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 16:36:02.138700 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self.creator() Aug 01 16:36:02.138700 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 16:36:02.138700 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application created_value = creator( Aug 01 16:36:02.138700 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 16:36:02.138700 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 16:36:02.138700 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 16:36:02.138700 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 16:36:02.138700 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 16:36:02.138700 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 16:36:02.138700 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 01 16:36:02.138700 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application Aug 01 16:36:02.138700 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 101/197] 10.222.0.20 () {56 vars in 1099 bytes} [Mon Aug 1 16:36:01 2022] GET /identity/v3/roles/reader => generated 84 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 16:36:02.139389 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f565db7e-a1fa-4c96-a996-61cee761c941 None admin] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:36:02.154736 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03773995-1590-44de-a5a9-c965b19f310c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:36:02.155768 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03773995-1590-44de-a5a9-c965b19f310c None admin] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:36:02.156225 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03773995-1590-44de-a5a9-c965b19f310c None admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:36:02.156508 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03773995-1590-44de-a5a9-c965b19f310c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:36:02.242096 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03773995-1590-44de-a5a9-c965b19f310c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:36:02.244749 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03773995-1590-44de-a5a9-c965b19f310c None admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:36:02.270062 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG dbcounter [-] [87057] Writing DB stats keystone:SELECT=100 {{(pid=87057) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:36:02.277000 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f565db7e-a1fa-4c96-a996-61cee761c941 None admin] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:36:02.319496 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-263b8dc3-eb85-43bd-8e87-f057780a2439 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:36:02.320449 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263b8dc3-eb85-43bd-8e87-f057780a2439 None admin] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:36:02.321052 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263b8dc3-eb85-43bd-8e87-f057780a2439 None admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:36:02.321516 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263b8dc3-eb85-43bd-8e87-f057780a2439 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:36:02.385271 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG dbcounter [-] [87058] Writing DB stats keystone:SELECT=100 {{(pid=87058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:36:02.398412 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 97/198] 10.222.0.20 () {58 vars in 1282 bytes} [Mon Aug 1 16:36:01 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 425 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 01 16:36:02.400431 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03773995-1590-44de-a5a9-c965b19f310c None admin] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:36:02.416482 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-263b8dc3-eb85-43bd-8e87-f057780a2439 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:36:02.418396 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-263b8dc3-eb85-43bd-8e87-f057780a2439 None admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:36:02.541170 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03773995-1590-44de-a5a9-c965b19f310c None admin] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:36:02.561302 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 102/199] 10.222.0.20 () {58 vars in 1282 bytes} [Mon Aug 1 16:36:02 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 422 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 01 16:36:02.562213 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-263b8dc3-eb85-43bd-8e87-f057780a2439 None admin] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:36:02.563031 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG dbcounter [-] [87057] Writing DB stats keystone:SELECT=100 {{(pid=87057) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:36:02.588097 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41516823-bbc6-47c6-8669-307ac9968d87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:36:02.589231 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41516823-bbc6-47c6-8669-307ac9968d87 None admin] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:36:02.589721 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41516823-bbc6-47c6-8669-307ac9968d87 None admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:36:02.590366 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41516823-bbc6-47c6-8669-307ac9968d87 None admin] PATH_INFO: `/v3/roles` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:36:02.591592 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41516823-bbc6-47c6-8669-307ac9968d87 None admin] RBAC: Adding query filter params (name=reader) {{(pid=87057) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:36:02.592319 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41516823-bbc6-47c6-8669-307ac9968d87 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:36:02.594129 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41516823-bbc6-47c6-8669-307ac9968d87 None admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:36:02.606506 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 98/200] 10.222.0.20 () {56 vars in 1101 bytes} [Mon Aug 1 16:36:02 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:36:02.607556 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41516823-bbc6-47c6-8669-307ac9968d87 None admin] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:36:02.663225 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG dbcounter [-] [87058] Writing DB stats keystone:SELECT=100 {{(pid=87058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:36:02.684537 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-263b8dc3-eb85-43bd-8e87-f057780a2439 None admin] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:36:02.723647 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a61cf11-dca6-4ba1-8ed4-8d172679476c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:36:02.724536 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a61cf11-dca6-4ba1-8ed4-8d172679476c None admin] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:36:02.724990 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a61cf11-dca6-4ba1-8ed4-8d172679476c None admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:36:02.725412 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a61cf11-dca6-4ba1-8ed4-8d172679476c None admin] PATH_INFO: `/v3/projects/d9a43129dd714d05bd409ffb0875f7e6` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:36:02.727340 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a61cf11-dca6-4ba1-8ed4-8d172679476c None admin] RBAC: Authorizing `identity:get_project(project_id=d9a43129dd714d05bd409ffb0875f7e6)` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:36:02.728945 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a61cf11-dca6-4ba1-8ed4-8d172679476c None admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:36:02.730854 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 103/201] 10.222.0.20 () {56 vars in 1186 bytes} [Mon Aug 1 16:36:02 2022] GET /identity/v3/projects/d9a43129dd714d05bd409ffb0875f7e6 => generated 302 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:36:02.731871 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a61cf11-dca6-4ba1-8ed4-8d172679476c None admin] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:36:02.733063 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41516823-bbc6-47c6-8669-307ac9968d87 None admin] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:36:02.779689 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8083fbd-f253-4859-a064-4dae332f5e75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:36:02.780664 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8083fbd-f253-4859-a064-4dae332f5e75 None admin] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:36:02.781048 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8083fbd-f253-4859-a064-4dae332f5e75 None admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:36:02.781325 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8083fbd-f253-4859-a064-4dae332f5e75 None admin] PATH_INFO: `/v3/projects/d9a43129dd714d05bd409ffb0875f7e6` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:36:02.782834 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8083fbd-f253-4859-a064-4dae332f5e75 None admin] RBAC: Authorizing `identity:get_project(project_id=d9a43129dd714d05bd409ffb0875f7e6)` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:36:02.785157 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8083fbd-f253-4859-a064-4dae332f5e75 None admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:36:02.787778 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 99/202] 10.222.0.20 () {56 vars in 1186 bytes} [Mon Aug 1 16:36:02 2022] GET /identity/v3/projects/d9a43129dd714d05bd409ffb0875f7e6 => generated 302 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:36:02.788668 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8083fbd-f253-4859-a064-4dae332f5e75 None admin] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:36:02.847003 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a61cf11-dca6-4ba1-8ed4-8d172679476c None admin] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:36:02.862453 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG dbcounter [-] [87057] Writing DB stats keystone:SELECT=100 {{(pid=87057) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:36:02.886530 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0fe3677-d4c1-4e4d-878c-c9fdafbbc95d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:36:02.887503 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0fe3677-d4c1-4e4d-878c-c9fdafbbc95d None admin] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:36:02.887950 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0fe3677-d4c1-4e4d-878c-c9fdafbbc95d None admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:36:02.892823 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0fe3677-d4c1-4e4d-878c-c9fdafbbc95d None admin] PATH_INFO: `/v3/users/cce885e3eda2452eadebf5bff42d3e8d` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:36:02.892823 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0fe3677-d4c1-4e4d-878c-c9fdafbbc95d None admin] RBAC: Authorizing `identity:get_user(user_id=cce885e3eda2452eadebf5bff42d3e8d)` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:36:02.892823 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0fe3677-d4c1-4e4d-878c-c9fdafbbc95d None admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:36:02.893547 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 104/203] 10.222.0.20 () {56 vars in 1177 bytes} [Mon Aug 1 16:36:02 2022] GET /identity/v3/users/cce885e3eda2452eadebf5bff42d3e8d => generated 289 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:36:02.894389 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0fe3677-d4c1-4e4d-878c-c9fdafbbc95d None admin] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:36:02.915457 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG dbcounter [-] [87058] Writing DB stats keystone:SELECT=100 {{(pid=87058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:36:02.918303 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8083fbd-f253-4859-a064-4dae332f5e75 None admin] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:36:02.954001 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e2cf41-51c9-484c-824b-11df1b48e9dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:36:02.954870 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e2cf41-51c9-484c-824b-11df1b48e9dd None admin] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:36:02.955294 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e2cf41-51c9-484c-824b-11df1b48e9dd None admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:36:02.955737 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e2cf41-51c9-484c-824b-11df1b48e9dd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:36:02.956669 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45e2cf41-51c9-484c-824b-11df1b48e9dd None admin] RBAC: Adding query filter params (role.id=3fac71a39f3f4ed198a3579064c30268, scope.project.id=d9a43129dd714d05bd409ffb0875f7e6, user.id=bccb7b2b19304928888200525af9748a) {{(pid=87057) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:36:02.958117 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45e2cf41-51c9-484c-824b-11df1b48e9dd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:36:02.959741 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45e2cf41-51c9-484c-824b-11df1b48e9dd None admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:36:02.966737 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 100/204] 10.222.0.20 () {56 vars in 1374 bytes} [Mon Aug 1 16:36:02 2022] GET /identity/v3/role_assignments?user.id=bccb7b2b19304928888200525af9748a&scope.project.id=d9a43129dd714d05bd409ffb0875f7e6&role.id=3fac71a39f3f4ed198a3579064c30268 => generated 259 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:36:02.967747 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e2cf41-51c9-484c-824b-11df1b48e9dd None admin] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:36:03.007381 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0fe3677-d4c1-4e4d-878c-c9fdafbbc95d None admin] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:36:03.045708 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfcbc377-b3b5-405e-afea-8fb3381f3535 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:36:03.046545 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfcbc377-b3b5-405e-afea-8fb3381f3535 None admin] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:36:03.047592 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfcbc377-b3b5-405e-afea-8fb3381f3535 None admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:36:03.047592 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfcbc377-b3b5-405e-afea-8fb3381f3535 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:36:03.048386 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfcbc377-b3b5-405e-afea-8fb3381f3535 None admin] RBAC: Adding query filter params (role.id=440bb943dd5b46fea038b903b6eaaf81, scope.project.id=d9a43129dd714d05bd409ffb0875f7e6, user.id=bccb7b2b19304928888200525af9748a) {{(pid=87058) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:36:03.048840 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfcbc377-b3b5-405e-afea-8fb3381f3535 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:36:03.050520 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfcbc377-b3b5-405e-afea-8fb3381f3535 None admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:36:03.058144 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 105/205] 10.222.0.20 () {56 vars in 1374 bytes} [Mon Aug 1 16:36:02 2022] GET /identity/v3/role_assignments?user.id=bccb7b2b19304928888200525af9748a&scope.project.id=d9a43129dd714d05bd409ffb0875f7e6&role.id=440bb943dd5b46fea038b903b6eaaf81 => generated 259 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:36:03.083351 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e2cf41-51c9-484c-824b-11df1b48e9dd None admin] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:36:03.110501 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG dbcounter [-] [87057] Writing DB stats keystone:SELECT=100 {{(pid=87057) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:36:03.122825 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c5f0aa-dc4c-4b8c-a4b8-63da52902455 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:36:03.123755 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c5f0aa-dc4c-4b8c-a4b8-63da52902455 None admin] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:36:03.124181 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c5f0aa-dc4c-4b8c-a4b8-63da52902455 None admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:36:03.124610 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c5f0aa-dc4c-4b8c-a4b8-63da52902455 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:36:03.201472 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8c5f0aa-dc4c-4b8c-a4b8-63da52902455 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:36:03.203257 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8c5f0aa-dc4c-4b8c-a4b8-63da52902455 None admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:36:03.313029 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 101/206] 10.222.0.20 () {58 vars in 1282 bytes} [Mon Aug 1 16:36:02 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 346 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 01 16:36:03.318759 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfcbc377-b3b5-405e-afea-8fb3381f3535 None admin] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:36:03.444676 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG dbcounter [-] [87058] Writing DB stats keystone:SELECT=100 {{(pid=87058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:36:03.463819 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfcbc377-b3b5-405e-afea-8fb3381f3535 None admin] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:36:03.500495 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f3efea6-4f7d-478e-8901-5206d164fa44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:36:03.501209 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3efea6-4f7d-478e-8901-5206d164fa44 None admin] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:36:03.501846 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3efea6-4f7d-478e-8901-5206d164fa44 None admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:36:03.502490 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3efea6-4f7d-478e-8901-5206d164fa44 None admin] PATH_INFO: `/v3/projects/64c2a2f6f91d4b9c8d5993d28fd19631` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:36:03.504150 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f3efea6-4f7d-478e-8901-5206d164fa44 None admin] RBAC: Authorizing `identity:get_project(project_id=64c2a2f6f91d4b9c8d5993d28fd19631)` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:36:03.505618 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f3efea6-4f7d-478e-8901-5206d164fa44 None admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:36:03.507518 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 106/207] 10.222.0.20 () {56 vars in 1186 bytes} [Mon Aug 1 16:36:03 2022] GET /identity/v3/projects/64c2a2f6f91d4b9c8d5993d28fd19631 => generated 298 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:36:03.513970 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c5f0aa-dc4c-4b8c-a4b8-63da52902455 None admin] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:36:03.529475 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG dbcounter [-] [87057] Writing DB stats keystone:SELECT=100 {{(pid=87057) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:36:03.644591 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c5f0aa-dc4c-4b8c-a4b8-63da52902455 None admin] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:36:03.693168 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc10dfe6-7223-4046-89ff-0cf598456175 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:36:03.693789 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc10dfe6-7223-4046-89ff-0cf598456175 None admin] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:36:03.694089 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc10dfe6-7223-4046-89ff-0cf598456175 None admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:36:03.694370 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc10dfe6-7223-4046-89ff-0cf598456175 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:36:03.695169 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc10dfe6-7223-4046-89ff-0cf598456175 None admin] RBAC: Adding query filter params (role.id=02c40c90308547dd8ac5aaad0da8d68c, scope.project.id=64c2a2f6f91d4b9c8d5993d28fd19631, user.id=cce885e3eda2452eadebf5bff42d3e8d) {{(pid=87057) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:36:03.695539 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc10dfe6-7223-4046-89ff-0cf598456175 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:36:03.697007 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc10dfe6-7223-4046-89ff-0cf598456175 None admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:36:03.705250 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 102/208] 10.222.0.20 () {56 vars in 1374 bytes} [Mon Aug 1 16:36:03 2022] GET /identity/v3/role_assignments?user.id=cce885e3eda2452eadebf5bff42d3e8d&scope.project.id=64c2a2f6f91d4b9c8d5993d28fd19631&role.id=02c40c90308547dd8ac5aaad0da8d68c => generated 608 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:36:04.373137 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76780740-31ae-4cf6-87f4-5c8260d009e8 None None] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:36:04.373644 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76780740-31ae-4cf6-87f4-5c8260d009e8 None None] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:36:04.373720 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76780740-31ae-4cf6-87f4-5c8260d009e8 None None] PATH_INFO: `/` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:36:04.375231 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 107/209] 10.222.0.20 () {54 vars in 926 bytes} [Mon Aug 1 16:36:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 16:36:04.375741 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce67159e-70b2-418a-baa7-c16dca53b58d None None] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:36:04.376035 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce67159e-70b2-418a-baa7-c16dca53b58d None None] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:36:04.376274 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce67159e-70b2-418a-baa7-c16dca53b58d None None] PATH_INFO: `/` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:36:04.377720 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 103/210] 10.222.0.20 () {54 vars in 926 bytes} [Mon Aug 1 16:36:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 16:36:04.383611 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db690284-b571-45e9-b1bc-33f9a44395a9 None None] REQUEST_METHOD: `POST` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:36:04.383762 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbad9892-106a-4044-8a3f-8926ab5021fb None None] REQUEST_METHOD: `POST` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:36:04.385951 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbad9892-106a-4044-8a3f-8926ab5021fb None None] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:36:04.386088 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db690284-b571-45e9-b1bc-33f9a44395a9 None None] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:36:04.386170 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbad9892-106a-4044-8a3f-8926ab5021fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:36:04.386241 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db690284-b571-45e9-b1bc-33f9a44395a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:36:04.426956 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.auth.core [None req-db690284-b571-45e9-b1bc-33f9a44395a9 None None] MFA Rules not processed for user `1007f8e379b74e169b98fb7c3c0ba5b4`. Rule list: `[]` (Enabled: `True`). {{(pid=87057) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:36:04.427820 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.auth.core [None req-bbad9892-106a-4044-8a3f-8926ab5021fb None None] MFA Rules not processed for user `1007f8e379b74e169b98fb7c3c0ba5b4`. Rule list: `[]` (Enabled: `True`). {{(pid=87058) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:36:04.428375 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.fernet_utils [None req-db690284-b571-45e9-b1bc-33f9a44395a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87057) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:36:04.429039 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.fernet_utils [None req-bbad9892-106a-4044-8a3f-8926ab5021fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:36:04.501970 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 104/211] 10.222.0.20 () {58 vars in 1023 bytes} [Mon Aug 1 16:36:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 16:36:04.517868 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 108/212] 10.222.0.20 () {58 vars in 1023 bytes} [Mon Aug 1 16:36:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 01 16:36:04.527355 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db690284-b571-45e9-b1bc-33f9a44395a9 None None] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:36:04.530346 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG dbcounter [-] [87057] Writing DB stats keystone:SELECT=100 {{(pid=87057) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:36:04.543280 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbad9892-106a-4044-8a3f-8926ab5021fb None None] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:36:04.633116 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG dbcounter [-] [87058] Writing DB stats keystone:SELECT=100 {{(pid=87058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:36:04.654476 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db690284-b571-45e9-b1bc-33f9a44395a9 None None] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:36:04.662903 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbad9892-106a-4044-8a3f-8926ab5021fb None None] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:36:04.692544 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a754002-a280-427a-9da8-41e880544e8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:36:04.693306 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a754002-a280-427a-9da8-41e880544e8f None admin] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:36:04.693585 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a754002-a280-427a-9da8-41e880544e8f None admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:36:04.694005 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a754002-a280-427a-9da8-41e880544e8f None admin] PATH_INFO: `/v3/domains/default` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:36:04.695418 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a754002-a280-427a-9da8-41e880544e8f None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:36:04.696871 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a754002-a280-427a-9da8-41e880544e8f None admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:36:04.702152 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 105/213] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Aug 1 16:36:04 2022] GET /identity/v3/domains/default => generated 199 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:36:04.704489 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a754002-a280-427a-9da8-41e880544e8f None admin] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:36:04.709753 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7836c76-8252-4cd4-b891-6acdaf3da276 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:36:04.710677 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7836c76-8252-4cd4-b891-6acdaf3da276 None admin] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:36:04.711694 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7836c76-8252-4cd4-b891-6acdaf3da276 None admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:36:04.712146 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7836c76-8252-4cd4-b891-6acdaf3da276 None admin] PATH_INFO: `/v3/roles/member` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:36:04.726702 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7836c76-8252-4cd4-b891-6acdaf3da276 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 16:36:04.734374 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application [None req-d7836c76-8252-4cd4-b891-6acdaf3da276 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 01 16:36:04.734374 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 16:36:04.734374 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 16:36:04.734374 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 16:36:04.734374 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 16:36:04.734374 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 16:36:04.734374 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 16:36:04.734374 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 16:36:04.734374 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 16:36:04.734374 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 16:36:04.734374 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 16:36:04.734374 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 16:36:04.734374 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 16:36:04.734374 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 16:36:04.734374 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 16:36:04.734374 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 16:36:04.735281 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 16:36:04.735281 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 16:36:04.735281 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 16:36:04.735281 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 16:36:04.735281 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 16:36:04.735281 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 16:36:04.735281 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 16:36:04.735281 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 16:36:04.735281 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 16:36:04.735281 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 16:36:04.735281 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 16:36:04.735281 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application with Lock( Aug 01 16:36:04.735281 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 16:36:04.735281 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self._enter() Aug 01 16:36:04.735281 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 16:36:04.735281 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 16:36:04.736078 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 16:36:04.736078 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self.creator() Aug 01 16:36:04.736078 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 16:36:04.736078 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application created_value = creator( Aug 01 16:36:04.736078 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 16:36:04.736078 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 16:36:04.736078 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 16:36:04.736078 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 16:36:04.736078 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 16:36:04.736078 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 16:36:04.736078 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 01 16:36:04.736078 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application Aug 01 16:36:04.738737 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 109/214] 10.222.0.20 () {56 vars in 1099 bytes} [Mon Aug 1 16:36:04 2022] GET /identity/v3/roles/member => generated 84 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 01 16:36:04.744906 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7836c76-8252-4cd4-b891-6acdaf3da276 None admin] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:36:04.807788 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG dbcounter [-] [87057] Writing DB stats keystone:SELECT=100 {{(pid=87057) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:36:04.850944 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a754002-a280-427a-9da8-41e880544e8f None admin] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:36:04.882347 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7836c76-8252-4cd4-b891-6acdaf3da276 None admin] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:36:04.893343 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5512e21e-9970-4dab-892a-e4e0d01c2228 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:36:04.893981 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5512e21e-9970-4dab-892a-e4e0d01c2228 None admin] REQUEST_METHOD: `POST` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:36:04.894624 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5512e21e-9970-4dab-892a-e4e0d01c2228 None admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:36:04.894624 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5512e21e-9970-4dab-892a-e4e0d01c2228 None admin] PATH_INFO: `/v3/users` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:36:04.895475 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5512e21e-9970-4dab-892a-e4e0d01c2228 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:36:04.897111 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5512e21e-9970-4dab-892a-e4e0d01c2228 None admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:36:04.927621 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c0841a-b45a-4c2b-8688-80cf3df84684 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:36:04.928247 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c0841a-b45a-4c2b-8688-80cf3df84684 None admin] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:36:04.928529 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c0841a-b45a-4c2b-8688-80cf3df84684 None admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:36:04.938299 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c0841a-b45a-4c2b-8688-80cf3df84684 None admin] PATH_INFO: `/v3/roles` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:36:04.938299 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06c0841a-b45a-4c2b-8688-80cf3df84684 None admin] RBAC: Adding query filter params (name=member) {{(pid=87058) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:36:04.938299 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06c0841a-b45a-4c2b-8688-80cf3df84684 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:36:04.938299 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06c0841a-b45a-4c2b-8688-80cf3df84684 None admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:36:04.941004 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 110/215] 10.222.0.20 () {56 vars in 1101 bytes} [Mon Aug 1 16:36:04 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:36:04.946678 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c0841a-b45a-4c2b-8688-80cf3df84684 None admin] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:36:04.949629 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG dbcounter [-] [87058] Writing DB stats keystone:SELECT=100 {{(pid=87058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:36:05.014064 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 106/216] 10.222.0.20 () {60 vars in 1132 bytes} [Mon Aug 1 16:36:04 2022] POST /identity/v3/users => generated 297 bytes in 301 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 16:36:05.073786 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c0841a-b45a-4c2b-8688-80cf3df84684 None admin] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:36:05.113560 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e867d4e-a785-4cec-94af-4df4f1848d12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:36:05.114251 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e867d4e-a785-4cec-94af-4df4f1848d12 None admin] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:36:05.114534 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e867d4e-a785-4cec-94af-4df4f1848d12 None admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:36:05.114809 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e867d4e-a785-4cec-94af-4df4f1848d12 None admin] PATH_INFO: `/v3/users/ac697afa751948bf87a605cf4aab7c65` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:36:05.130834 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e867d4e-a785-4cec-94af-4df4f1848d12 None admin] RBAC: Authorizing `identity:get_user(user_id=ac697afa751948bf87a605cf4aab7c65)` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:36:05.132432 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e867d4e-a785-4cec-94af-4df4f1848d12 None admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:36:05.134988 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 111/217] 10.222.0.20 () {56 vars in 1177 bytes} [Mon Aug 1 16:36:04 2022] GET /identity/v3/users/ac697afa751948bf87a605cf4aab7c65 => generated 297 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:36:05.140767 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5512e21e-9970-4dab-892a-e4e0d01c2228 None admin] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:36:05.257198 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5512e21e-9970-4dab-892a-e4e0d01c2228 None admin] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:36:05.286604 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG dbcounter [-] [87057] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87057) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:36:05.295200 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd028893-c1be-47fc-b55d-27fc569cf5bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:36:05.295837 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd028893-c1be-47fc-b55d-27fc569cf5bb None admin] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:36:05.296118 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd028893-c1be-47fc-b55d-27fc569cf5bb None admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:36:05.296385 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd028893-c1be-47fc-b55d-27fc569cf5bb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:36:05.315387 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f48391-0857-448f-aa1c-4e086bf6ddbb None None] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:36:05.315787 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f48391-0857-448f-aa1c-4e086bf6ddbb None None] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:36:05.316110 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f48391-0857-448f-aa1c-4e086bf6ddbb None None] PATH_INFO: `/` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:36:05.317628 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 112/218] 10.222.0.20 () {54 vars in 926 bytes} [Mon Aug 1 16:36:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 16:36:05.326024 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf5269d-7837-42d1-b3f3-1d0d248b30bb None None] REQUEST_METHOD: `POST` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:36:05.326213 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf5269d-7837-42d1-b3f3-1d0d248b30bb None None] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:36:05.326464 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf5269d-7837-42d1-b3f3-1d0d248b30bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:36:05.348414 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.auth.core [None req-7cf5269d-7837-42d1-b3f3-1d0d248b30bb None None] MFA Rules not processed for user `1007f8e379b74e169b98fb7c3c0ba5b4`. Rule list: `[]` (Enabled: `True`). {{(pid=87058) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:36:05.349623 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.fernet_utils [None req-7cf5269d-7837-42d1-b3f3-1d0d248b30bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:36:05.385532 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd028893-c1be-47fc-b55d-27fc569cf5bb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:36:05.387056 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd028893-c1be-47fc-b55d-27fc569cf5bb None admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:36:05.434947 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG dbcounter [-] [87058] Writing DB stats keystone:SELECT=100 {{(pid=87058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:36:05.453057 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 113/219] 10.222.0.20 () {58 vars in 1023 bytes} [Mon Aug 1 16:36:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 01 16:36:05.482765 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf5269d-7837-42d1-b3f3-1d0d248b30bb None None] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:36:05.538810 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 107/220] 10.222.0.20 () {58 vars in 1282 bytes} [Mon Aug 1 16:36:05 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 399 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 01 16:36:05.546549 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd028893-c1be-47fc-b55d-27fc569cf5bb None admin] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:36:05.582159 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG dbcounter [-] [87057] Writing DB stats keystone:SELECT=100 {{(pid=87057) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:36:05.608192 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf5269d-7837-42d1-b3f3-1d0d248b30bb None None] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:36:05.646877 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-548443c0-a95e-46cb-b9fb-67d3f635aafc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:36:05.647823 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548443c0-a95e-46cb-b9fb-67d3f635aafc None admin] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:36:05.648144 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548443c0-a95e-46cb-b9fb-67d3f635aafc None admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:36:05.649075 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548443c0-a95e-46cb-b9fb-67d3f635aafc None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:36:05.663703 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-548443c0-a95e-46cb-b9fb-67d3f635aafc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 16:36:05.668945 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd028893-c1be-47fc-b55d-27fc569cf5bb None admin] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:36:05.670752 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application [None req-548443c0-a95e-46cb-b9fb-67d3f635aafc None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 01 16:36:05.670752 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 16:36:05.670752 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 16:36:05.670752 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 16:36:05.670752 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 16:36:05.670752 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 16:36:05.670752 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 16:36:05.670752 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 16:36:05.670752 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 16:36:05.670752 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 16:36:05.670752 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 16:36:05.670752 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 16:36:05.670752 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 16:36:05.670752 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 16:36:05.670752 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 16:36:05.670752 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 16:36:05.671620 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 16:36:05.671620 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 16:36:05.671620 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 16:36:05.671620 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 16:36:05.671620 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 16:36:05.671620 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 16:36:05.671620 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 16:36:05.671620 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 16:36:05.671620 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 16:36:05.671620 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 16:36:05.671620 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 16:36:05.671620 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application with Lock( Aug 01 16:36:05.671620 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 16:36:05.671620 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self._enter() Aug 01 16:36:05.671620 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 16:36:05.671620 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 16:36:05.672440 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 16:36:05.672440 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self.creator() Aug 01 16:36:05.672440 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 16:36:05.672440 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application created_value = creator( Aug 01 16:36:05.672440 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 16:36:05.672440 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 16:36:05.672440 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 16:36:05.672440 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 16:36:05.672440 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 16:36:05.672440 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 16:36:05.672440 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 01 16:36:05.672440 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application Aug 01 16:36:05.673124 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 114/221] 10.222.0.20 () {56 vars in 1114 bytes} [Mon Aug 1 16:36:05 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 01 16:36:05.675962 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7344a121-f9e1-4432-a07d-30acd27b8af7 None None] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:36:05.675962 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7344a121-f9e1-4432-a07d-30acd27b8af7 None None] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:36:05.675962 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7344a121-f9e1-4432-a07d-30acd27b8af7 None None] PATH_INFO: `/` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:36:05.676507 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 115/222] 10.222.0.20 () {54 vars in 926 bytes} [Mon Aug 1 16:36:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 16:36:05.679121 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7344a121-f9e1-4432-a07d-30acd27b8af7 None None] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:36:05.721041 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a49053a6-b33d-47d6-81fe-d6853ac58d9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:36:05.722098 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49053a6-b33d-47d6-81fe-d6853ac58d9f None admin] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:36:05.722098 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49053a6-b33d-47d6-81fe-d6853ac58d9f None admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:36:05.722309 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49053a6-b33d-47d6-81fe-d6853ac58d9f None admin] PATH_INFO: `/v3/projects/d9a43129dd714d05bd409ffb0875f7e6` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:36:05.724091 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a49053a6-b33d-47d6-81fe-d6853ac58d9f None admin] RBAC: Authorizing `identity:get_project(project_id=d9a43129dd714d05bd409ffb0875f7e6)` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:36:05.725075 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a49053a6-b33d-47d6-81fe-d6853ac58d9f None admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:36:05.731770 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 108/223] 10.222.0.20 () {56 vars in 1186 bytes} [Mon Aug 1 16:36:05 2022] GET /identity/v3/projects/d9a43129dd714d05bd409ffb0875f7e6 => generated 302 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:36:05.731770 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c05434-5c2c-4d66-8eed-d95c08263f06 None None] REQUEST_METHOD: `POST` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:36:05.731770 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c05434-5c2c-4d66-8eed-d95c08263f06 None None] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:36:05.731770 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c05434-5c2c-4d66-8eed-d95c08263f06 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:36:05.747078 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.auth.core [None req-e7c05434-5c2c-4d66-8eed-d95c08263f06 None None] MFA Rules not processed for user `1007f8e379b74e169b98fb7c3c0ba5b4`. Rule list: `[]` (Enabled: `True`). {{(pid=87057) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:36:05.748459 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG dbcounter [-] [87058] Writing DB stats keystone:SELECT=100 {{(pid=87058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:36:05.748749 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.fernet_utils [None req-e7c05434-5c2c-4d66-8eed-d95c08263f06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87057) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:36:05.817257 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7344a121-f9e1-4432-a07d-30acd27b8af7 None None] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:36:05.832895 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 109/224] 10.222.0.20 () {58 vars in 1023 bytes} [Mon Aug 1 16:36:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 16:36:05.834383 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c05434-5c2c-4d66-8eed-d95c08263f06 None None] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:36:05.857567 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-849bcbe8-7b85-44aa-a079-1b268ae1b492 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:36:05.858585 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849bcbe8-7b85-44aa-a079-1b268ae1b492 None admin] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:36:05.858871 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849bcbe8-7b85-44aa-a079-1b268ae1b492 None admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:36:05.859345 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849bcbe8-7b85-44aa-a079-1b268ae1b492 None admin] PATH_INFO: `/v3/roles` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:36:05.860071 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-849bcbe8-7b85-44aa-a079-1b268ae1b492 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=87058) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:36:05.860378 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-849bcbe8-7b85-44aa-a079-1b268ae1b492 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:36:05.861876 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-849bcbe8-7b85-44aa-a079-1b268ae1b492 None admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:36:05.871084 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG dbcounter [-] [87057] Writing DB stats keystone:SELECT=100 {{(pid=87057) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:36:05.872870 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 116/225] 10.222.0.20 () {56 vars in 1111 bytes} [Mon Aug 1 16:36:05 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:36:05.873839 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-849bcbe8-7b85-44aa-a079-1b268ae1b492 None admin] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:36:05.978473 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c05434-5c2c-4d66-8eed-d95c08263f06 None None] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:36:06.003348 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-849bcbe8-7b85-44aa-a079-1b268ae1b492 None admin] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:36:06.016717 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-126ba46a-6750-486b-8c3c-22896453652f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:36:06.017349 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126ba46a-6750-486b-8c3c-22896453652f None admin] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:36:06.017983 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126ba46a-6750-486b-8c3c-22896453652f None admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:36:06.018287 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126ba46a-6750-486b-8c3c-22896453652f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:36:06.019087 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-126ba46a-6750-486b-8c3c-22896453652f None admin] RBAC: Adding query filter params (role.id=0999d3da63a545f18ff3252afdc90518, scope.project.id=d9a43129dd714d05bd409ffb0875f7e6, user.id=ac697afa751948bf87a605cf4aab7c65) {{(pid=87057) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:36:06.019405 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-126ba46a-6750-486b-8c3c-22896453652f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:36:06.020490 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG dbcounter [-] [87058] Writing DB stats keystone:SELECT=100 {{(pid=87058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:36:06.020856 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-126ba46a-6750-486b-8c3c-22896453652f None admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:36:06.031630 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 110/226] 10.222.0.20 () {56 vars in 1374 bytes} [Mon Aug 1 16:36:05 2022] GET /identity/v3/role_assignments?user.id=ac697afa751948bf87a605cf4aab7c65&scope.project.id=d9a43129dd714d05bd409ffb0875f7e6&role.id=0999d3da63a545f18ff3252afdc90518 => generated 259 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:36:06.032601 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-126ba46a-6750-486b-8c3c-22896453652f None admin] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:36:06.057034 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b0588b-385d-4c8d-86a4-c288f851ec5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:36:06.057913 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b0588b-385d-4c8d-86a4-c288f851ec5b None admin] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:36:06.058655 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b0588b-385d-4c8d-86a4-c288f851ec5b None admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:36:06.059022 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b0588b-385d-4c8d-86a4-c288f851ec5b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:36:06.084889 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31b0588b-385d-4c8d-86a4-c288f851ec5b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 16:36:06.096369 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application [None req-31b0588b-385d-4c8d-86a4-c288f851ec5b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 16:36:06.096369 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 16:36:06.096369 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 16:36:06.096369 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 16:36:06.096369 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 16:36:06.096369 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 16:36:06.096369 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 16:36:06.096369 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 16:36:06.096369 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 16:36:06.096369 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 16:36:06.096369 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 16:36:06.096369 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 16:36:06.096369 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 16:36:06.096369 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 16:36:06.096369 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 16:36:06.096369 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 16:36:06.097323 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 16:36:06.097323 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 16:36:06.097323 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 16:36:06.097323 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 16:36:06.097323 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 16:36:06.097323 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 16:36:06.097323 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 16:36:06.097323 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 16:36:06.097323 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 16:36:06.097323 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 16:36:06.097323 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 16:36:06.097323 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application with Lock( Aug 01 16:36:06.097323 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 16:36:06.097323 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self._enter() Aug 01 16:36:06.097323 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 16:36:06.097323 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 16:36:06.098167 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 16:36:06.098167 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self.creator() Aug 01 16:36:06.098167 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 16:36:06.098167 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application created_value = creator( Aug 01 16:36:06.098167 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 16:36:06.098167 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 16:36:06.098167 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 16:36:06.098167 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 16:36:06.098167 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 16:36:06.098167 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 16:36:06.098167 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 16:36:06.098167 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application Aug 01 16:36:06.098167 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 117/227] 10.222.0.20 () {56 vars in 1096 bytes} [Mon Aug 1 16:36:05 2022] GET /identity/v3/roles/admin => generated 83 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 16:36:06.106174 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b0588b-385d-4c8d-86a4-c288f851ec5b None admin] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:36:06.219768 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG dbcounter [-] [87057] Writing DB stats keystone:SELECT=100 {{(pid=87057) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:36:06.232933 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-126ba46a-6750-486b-8c3c-22896453652f None admin] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:36:06.311269 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b0588b-385d-4c8d-86a4-c288f851ec5b None admin] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:36:06.322707 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c98a3d03-7367-4bc4-a874-d74da5bd019e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:36:06.323469 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98a3d03-7367-4bc4-a874-d74da5bd019e None admin] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:36:06.323750 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98a3d03-7367-4bc4-a874-d74da5bd019e None admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:36:06.324021 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98a3d03-7367-4bc4-a874-d74da5bd019e None admin] PATH_INFO: `/v3/users/bccb7b2b19304928888200525af9748a` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:36:06.325339 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c98a3d03-7367-4bc4-a874-d74da5bd019e None admin] RBAC: Authorizing `identity:get_user(user_id=bccb7b2b19304928888200525af9748a)` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:36:06.327132 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c98a3d03-7367-4bc4-a874-d74da5bd019e None admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:36:06.328866 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 111/228] 10.222.0.20 () {56 vars in 1177 bytes} [Mon Aug 1 16:36:06 2022] GET /identity/v3/users/bccb7b2b19304928888200525af9748a => generated 283 bytes in 297 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:36:06.334562 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c98a3d03-7367-4bc4-a874-d74da5bd019e None admin] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:36:06.394862 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65858b03-db0e-48a8-a079-498a4d46d861 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:36:06.395443 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65858b03-db0e-48a8-a079-498a4d46d861 None admin] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:36:06.395713 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65858b03-db0e-48a8-a079-498a4d46d861 None admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:36:06.396152 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65858b03-db0e-48a8-a079-498a4d46d861 None admin] PATH_INFO: `/v3/roles` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:36:06.402057 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65858b03-db0e-48a8-a079-498a4d46d861 None admin] RBAC: Adding query filter params (name=admin) {{(pid=87058) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:36:06.403318 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65858b03-db0e-48a8-a079-498a4d46d861 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:36:06.406214 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65858b03-db0e-48a8-a079-498a4d46d861 None admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:36:06.424672 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 118/229] 10.222.0.20 () {56 vars in 1099 bytes} [Mon Aug 1 16:36:06 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 318 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:36:06.429623 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65858b03-db0e-48a8-a079-498a4d46d861 None admin] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:36:06.495644 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c98a3d03-7367-4bc4-a874-d74da5bd019e None admin] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:36:06.502611 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG dbcounter [-] [87058] Writing DB stats keystone:SELECT=100 {{(pid=87058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:36:06.567634 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cafb66c6-9f4d-4887-8e6e-b51210b8a511 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:36:06.568290 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cafb66c6-9f4d-4887-8e6e-b51210b8a511 None admin] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:36:06.568735 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cafb66c6-9f4d-4887-8e6e-b51210b8a511 None admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:36:06.569153 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cafb66c6-9f4d-4887-8e6e-b51210b8a511 None admin] PATH_INFO: `/v3/projects/d9a43129dd714d05bd409ffb0875f7e6` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:36:06.571168 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cafb66c6-9f4d-4887-8e6e-b51210b8a511 None admin] RBAC: Authorizing `identity:get_project(project_id=d9a43129dd714d05bd409ffb0875f7e6)` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:36:06.572897 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cafb66c6-9f4d-4887-8e6e-b51210b8a511 None admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:36:06.575120 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 112/230] 10.222.0.20 () {56 vars in 1186 bytes} [Mon Aug 1 16:36:06 2022] GET /identity/v3/projects/d9a43129dd714d05bd409ffb0875f7e6 => generated 302 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:36:06.587650 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cafb66c6-9f4d-4887-8e6e-b51210b8a511 None admin] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:36:06.623160 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65858b03-db0e-48a8-a079-498a4d46d861 None admin] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:36:06.639303 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG dbcounter [-] [87057] Writing DB stats keystone:SELECT=100 {{(pid=87057) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:36:06.697928 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d038c3-3fa3-4767-a5c9-a487333ee101 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:36:06.698962 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d038c3-3fa3-4767-a5c9-a487333ee101 None admin] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:36:06.700067 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d038c3-3fa3-4767-a5c9-a487333ee101 None admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:36:06.700546 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d038c3-3fa3-4767-a5c9-a487333ee101 None admin] PATH_INFO: `/v3/users/bccb7b2b19304928888200525af9748a` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:36:06.703114 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02d038c3-3fa3-4767-a5c9-a487333ee101 None admin] RBAC: Authorizing `identity:get_user(user_id=bccb7b2b19304928888200525af9748a)` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:36:06.705479 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02d038c3-3fa3-4767-a5c9-a487333ee101 None admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:36:06.708108 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 119/231] 10.222.0.20 () {56 vars in 1177 bytes} [Mon Aug 1 16:36:06 2022] GET /identity/v3/users/bccb7b2b19304928888200525af9748a => generated 283 bytes in 279 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:36:06.722067 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d038c3-3fa3-4767-a5c9-a487333ee101 None admin] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:36:06.749112 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cafb66c6-9f4d-4887-8e6e-b51210b8a511 None admin] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:36:06.799357 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c18d3c91-f6ec-4af3-aac2-96819a73adbd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:36:06.800044 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c18d3c91-f6ec-4af3-aac2-96819a73adbd None admin] REQUEST_METHOD: `PUT` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:36:06.800318 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c18d3c91-f6ec-4af3-aac2-96819a73adbd None admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:36:06.800584 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c18d3c91-f6ec-4af3-aac2-96819a73adbd None admin] PATH_INFO: `/v3/projects/d9a43129dd714d05bd409ffb0875f7e6/users/bccb7b2b19304928888200525af9748a/roles/3fac71a39f3f4ed198a3579064c30268` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:36:06.802846 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c18d3c91-f6ec-4af3-aac2-96819a73adbd None admin] RBAC: Authorizing `identity:create_grant(project_id=d9a43129dd714d05bd409ffb0875f7e6, user_id=bccb7b2b19304928888200525af9748a, role_id=3fac71a39f3f4ed198a3579064c30268)` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:36:06.804300 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c18d3c91-f6ec-4af3-aac2-96819a73adbd None admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:36:06.864099 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d038c3-3fa3-4767-a5c9-a487333ee101 None admin] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:36:06.867066 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG dbcounter [-] [87058] Writing DB stats keystone:SELECT=100 {{(pid=87058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:36:06.891399 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 113/232] 10.222.0.20 () {58 vars in 1439 bytes} [Mon Aug 1 16:36:06 2022] PUT /identity/v3/projects/d9a43129dd714d05bd409ffb0875f7e6/users/bccb7b2b19304928888200525af9748a/roles/3fac71a39f3f4ed198a3579064c30268 => generated 0 bytes in 304 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 16:36:06.912451 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f940547-935e-4f8d-982c-08b0504153d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:36:06.913160 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f940547-935e-4f8d-982c-08b0504153d0 None admin] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:36:06.913484 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f940547-935e-4f8d-982c-08b0504153d0 None admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:36:06.913760 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f940547-935e-4f8d-982c-08b0504153d0 None admin] PATH_INFO: `/v3/projects/d9a43129dd714d05bd409ffb0875f7e6` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:36:06.915667 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f940547-935e-4f8d-982c-08b0504153d0 None admin] RBAC: Authorizing `identity:get_project(project_id=d9a43129dd714d05bd409ffb0875f7e6)` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:36:06.917222 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f940547-935e-4f8d-982c-08b0504153d0 None admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:36:06.919139 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 120/233] 10.222.0.20 () {56 vars in 1186 bytes} [Mon Aug 1 16:36:06 2022] GET /identity/v3/projects/d9a43129dd714d05bd409ffb0875f7e6 => generated 302 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 16:36:06.925527 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c18d3c91-f6ec-4af3-aac2-96819a73adbd None admin] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:36:07.057902 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG dbcounter [-] [87057] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87057) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:36:07.075907 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c18d3c91-f6ec-4af3-aac2-96819a73adbd None admin] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:36:07.112769 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d8ceba8-51d7-4ba7-b746-b9abbde55084 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:36:07.113502 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8ceba8-51d7-4ba7-b746-b9abbde55084 None admin] REQUEST_METHOD: `PUT` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:36:07.113780 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8ceba8-51d7-4ba7-b746-b9abbde55084 None admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:36:07.114072 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8ceba8-51d7-4ba7-b746-b9abbde55084 None admin] PATH_INFO: `/v3/projects/d9a43129dd714d05bd409ffb0875f7e6/users/bccb7b2b19304928888200525af9748a/roles/440bb943dd5b46fea038b903b6eaaf81` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:36:07.116114 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d8ceba8-51d7-4ba7-b746-b9abbde55084 None admin] RBAC: Authorizing `identity:create_grant(project_id=d9a43129dd714d05bd409ffb0875f7e6, user_id=bccb7b2b19304928888200525af9748a, role_id=440bb943dd5b46fea038b903b6eaaf81)` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:36:07.118503 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d8ceba8-51d7-4ba7-b746-b9abbde55084 None admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:36:07.240535 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 114/234] 10.222.0.20 () {58 vars in 1439 bytes} [Mon Aug 1 16:36:06 2022] PUT /identity/v3/projects/d9a43129dd714d05bd409ffb0875f7e6/users/bccb7b2b19304928888200525af9748a/roles/440bb943dd5b46fea038b903b6eaaf81 => generated 0 bytes in 316 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 16:36:07.488740 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8bd8104-2cd3-4768-8f5b-41151e54c547 None None] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:36:07.489412 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8bd8104-2cd3-4768-8f5b-41151e54c547 None None] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:36:07.489936 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8bd8104-2cd3-4768-8f5b-41151e54c547 None None] PATH_INFO: `/` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:36:07.491837 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 121/235] 10.222.0.20 () {54 vars in 926 bytes} [Mon Aug 1 16:36:07 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 16:36:07.499378 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ccc9c09-30a9-4986-a611-fbcc80899219 None None] REQUEST_METHOD: `POST` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:36:07.500036 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ccc9c09-30a9-4986-a611-fbcc80899219 None None] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:36:07.500515 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ccc9c09-30a9-4986-a611-fbcc80899219 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:36:07.520408 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.auth.core [None req-0ccc9c09-30a9-4986-a611-fbcc80899219 None None] MFA Rules not processed for user `1007f8e379b74e169b98fb7c3c0ba5b4`. Rule list: `[]` (Enabled: `True`). {{(pid=87057) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:36:07.521965 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.fernet_utils [None req-0ccc9c09-30a9-4986-a611-fbcc80899219 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87057) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:36:07.610687 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 115/236] 10.222.0.20 () {58 vars in 1023 bytes} [Mon Aug 1 16:36:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 16:36:07.635139 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8bd8104-2cd3-4768-8f5b-41151e54c547 None None] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:36:07.749676 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8bd8104-2cd3-4768-8f5b-41151e54c547 None None] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:36:07.789434 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130396f1-c84d-45ed-8cd1-25c242e47f12 None None] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:36:07.789852 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130396f1-c84d-45ed-8cd1-25c242e47f12 None None] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:36:07.790070 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130396f1-c84d-45ed-8cd1-25c242e47f12 None None] PATH_INFO: `/` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:36:07.791555 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 116/237] 10.222.0.20 () {54 vars in 926 bytes} [Mon Aug 1 16:36:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 01 16:36:07.796845 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3845660-ef1c-497b-81fc-322cf1199b9d None None] REQUEST_METHOD: `POST` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:36:07.797396 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3845660-ef1c-497b-81fc-322cf1199b9d None None] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:36:07.797875 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3845660-ef1c-497b-81fc-322cf1199b9d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:36:07.806739 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-715511da-38da-4191-9ec7-198591a59201 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:36:07.807446 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715511da-38da-4191-9ec7-198591a59201 None admin] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:36:07.807732 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715511da-38da-4191-9ec7-198591a59201 None admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:36:07.808010 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715511da-38da-4191-9ec7-198591a59201 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:36:07.818199 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.auth.core [None req-e3845660-ef1c-497b-81fc-322cf1199b9d None None] MFA Rules not processed for user `1007f8e379b74e169b98fb7c3c0ba5b4`. Rule list: `[]` (Enabled: `True`). {{(pid=87057) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:36:07.819856 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.fernet_utils [None req-e3845660-ef1c-497b-81fc-322cf1199b9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87057) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:36:07.824517 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-715511da-38da-4191-9ec7-198591a59201 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 16:36:07.829519 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application [None req-715511da-38da-4191-9ec7-198591a59201 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 01 16:36:07.829519 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 16:36:07.829519 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 16:36:07.829519 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 16:36:07.829519 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 16:36:07.829519 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 16:36:07.829519 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 16:36:07.829519 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 16:36:07.829519 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 16:36:07.829519 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 16:36:07.829519 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 16:36:07.829519 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 16:36:07.829519 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 16:36:07.829519 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 16:36:07.829519 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 16:36:07.829519 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 16:36:07.830455 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 16:36:07.830455 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 16:36:07.830455 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 16:36:07.830455 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 16:36:07.830455 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 16:36:07.830455 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 16:36:07.830455 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 16:36:07.830455 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 16:36:07.830455 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 16:36:07.830455 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 16:36:07.830455 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 16:36:07.830455 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application with Lock( Aug 01 16:36:07.830455 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 16:36:07.830455 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self._enter() Aug 01 16:36:07.830455 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 16:36:07.830455 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 16:36:07.831630 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 16:36:07.831630 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self.creator() Aug 01 16:36:07.831630 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 16:36:07.831630 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application created_value = creator( Aug 01 16:36:07.831630 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 16:36:07.831630 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 16:36:07.831630 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 16:36:07.831630 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 16:36:07.831630 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 16:36:07.831630 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 16:36:07.831630 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 01 16:36:07.831630 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application Aug 01 16:36:07.831630 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 122/238] 10.222.0.20 () {56 vars in 1099 bytes} [Mon Aug 1 16:36:07 2022] GET /identity/v3/roles/reader => generated 84 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 16:36:07.840387 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-715511da-38da-4191-9ec7-198591a59201 None admin] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:36:07.910672 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG dbcounter [-] [87058] Writing DB stats keystone:SELECT=100 {{(pid=87058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:36:07.937006 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 117/239] 10.222.0.20 () {58 vars in 1023 bytes} [Mon Aug 1 16:36:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 01 16:36:07.971003 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3845660-ef1c-497b-81fc-322cf1199b9d None None] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:36:07.987237 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG dbcounter [-] [87057] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87057) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:36:08.073973 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-715511da-38da-4191-9ec7-198591a59201 None admin] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:36:08.117580 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3845660-ef1c-497b-81fc-322cf1199b9d None None] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:36:08.141226 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb25a104-c3d9-40ee-adbf-29c055e7b294 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:36:08.142225 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb25a104-c3d9-40ee-adbf-29c055e7b294 None admin] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:36:08.142961 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb25a104-c3d9-40ee-adbf-29c055e7b294 None admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:36:08.143433 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb25a104-c3d9-40ee-adbf-29c055e7b294 None admin] PATH_INFO: `/v3/roles` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:36:08.144398 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb25a104-c3d9-40ee-adbf-29c055e7b294 None admin] RBAC: Adding query filter params (name=reader) {{(pid=87058) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:36:08.144903 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb25a104-c3d9-40ee-adbf-29c055e7b294 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:36:08.146524 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb25a104-c3d9-40ee-adbf-29c055e7b294 None admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:36:08.158626 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 123/240] 10.222.0.20 () {56 vars in 1101 bytes} [Mon Aug 1 16:36:07 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 322 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 16:36:08.164785 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb25a104-c3d9-40ee-adbf-29c055e7b294 None admin] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:36:08.184059 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ea9ef2-9700-4a16-824d-cb5cdee531cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:36:08.185076 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ea9ef2-9700-4a16-824d-cb5cdee531cc None admin] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:36:08.185558 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ea9ef2-9700-4a16-824d-cb5cdee531cc None admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:36:08.186842 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ea9ef2-9700-4a16-824d-cb5cdee531cc None admin] PATH_INFO: `/v3/domains/default` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:36:08.188881 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39ea9ef2-9700-4a16-824d-cb5cdee531cc None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:36:08.191344 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39ea9ef2-9700-4a16-824d-cb5cdee531cc None admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:36:08.193913 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 118/241] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Aug 1 16:36:07 2022] GET /identity/v3/domains/default => generated 199 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:36:08.203115 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ea9ef2-9700-4a16-824d-cb5cdee531cc None admin] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:36:08.295457 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG dbcounter [-] [87057] Writing DB stats keystone:SELECT=100 {{(pid=87057) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:36:08.315644 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG dbcounter [-] [87058] Writing DB stats keystone:SELECT=100 {{(pid=87058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:36:08.323680 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ea9ef2-9700-4a16-824d-cb5cdee531cc None admin] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:36:08.327529 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb25a104-c3d9-40ee-adbf-29c055e7b294 None admin] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:36:08.363647 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c74ee115-78b2-41c9-b3c7-232fb711f890 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:36:08.364654 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74ee115-78b2-41c9-b3c7-232fb711f890 None admin] REQUEST_METHOD: `POST` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:36:08.365143 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74ee115-78b2-41c9-b3c7-232fb711f890 None admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:36:08.365249 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98908969-8c68-4e4c-9a18-b86839142d33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:36:08.365806 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74ee115-78b2-41c9-b3c7-232fb711f890 None admin] PATH_INFO: `/v3/users` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:36:08.365909 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98908969-8c68-4e4c-9a18-b86839142d33 None admin] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:36:08.366123 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98908969-8c68-4e4c-9a18-b86839142d33 None admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:36:08.366405 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98908969-8c68-4e4c-9a18-b86839142d33 None admin] PATH_INFO: `/v3/users/e4ba5c4a5a5a456ea72804daff75daa1` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:36:08.367387 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c74ee115-78b2-41c9-b3c7-232fb711f890 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:36:08.369187 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c74ee115-78b2-41c9-b3c7-232fb711f890 None admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:36:08.381628 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98908969-8c68-4e4c-9a18-b86839142d33 None admin] RBAC: Authorizing `identity:get_user(user_id=e4ba5c4a5a5a456ea72804daff75daa1)` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:36:08.383174 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98908969-8c68-4e4c-9a18-b86839142d33 None admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:36:08.385091 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 124/242] 10.222.0.20 () {56 vars in 1177 bytes} [Mon Aug 1 16:36:08 2022] GET /identity/v3/users/e4ba5c4a5a5a456ea72804daff75daa1 => generated 297 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:36:08.390511 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2e81e1-129a-445b-aca1-994bdf9b17f1 None None] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:36:08.390808 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2e81e1-129a-445b-aca1-994bdf9b17f1 None None] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:36:08.391060 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2e81e1-129a-445b-aca1-994bdf9b17f1 None None] PATH_INFO: `/` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:36:08.392309 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 125/243] 10.222.0.20 () {54 vars in 926 bytes} [Mon Aug 1 16:36:08 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 16:36:08.393280 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd2e81e1-129a-445b-aca1-994bdf9b17f1 None None] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:36:08.470415 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 119/244] 10.222.0.20 () {60 vars in 1132 bytes} [Mon Aug 1 16:36:08 2022] POST /identity/v3/users => generated 293 bytes in 267 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 16:36:08.471804 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720cae7d-66a7-4cfc-bfe6-100222c921aa None None] REQUEST_METHOD: `POST` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:36:08.473653 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720cae7d-66a7-4cfc-bfe6-100222c921aa None None] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:36:08.473653 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720cae7d-66a7-4cfc-bfe6-100222c921aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:36:08.516873 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.auth.core [None req-720cae7d-66a7-4cfc-bfe6-100222c921aa None None] MFA Rules not processed for user `1007f8e379b74e169b98fb7c3c0ba5b4`. Rule list: `[]` (Enabled: `True`). {{(pid=87057) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:36:08.518450 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.fernet_utils [None req-720cae7d-66a7-4cfc-bfe6-100222c921aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87057) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:36:08.550723 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd2e81e1-129a-445b-aca1-994bdf9b17f1 None None] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:36:08.625375 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6daaacf8-2097-4739-af00-7087ba3978e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:36:08.625923 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6daaacf8-2097-4739-af00-7087ba3978e1 None admin] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:36:08.626227 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6daaacf8-2097-4739-af00-7087ba3978e1 None admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:36:08.626500 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6daaacf8-2097-4739-af00-7087ba3978e1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:36:08.660352 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 120/245] 10.222.0.20 () {58 vars in 1023 bytes} [Mon Aug 1 16:36:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 16:36:08.661624 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG dbcounter [-] [87058] Writing DB stats keystone:SELECT=100 {{(pid=87058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:36:08.690467 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-720cae7d-66a7-4cfc-bfe6-100222c921aa None None] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:36:08.728502 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6daaacf8-2097-4739-af00-7087ba3978e1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:36:08.730898 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6daaacf8-2097-4739-af00-7087ba3978e1 None admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:36:08.763698 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG dbcounter [-] [87057] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87057) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:36:08.821003 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-720cae7d-66a7-4cfc-bfe6-100222c921aa None None] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:36:08.861105 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-972b407f-88ed-4de6-8798-fc322366f57f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:36:08.861905 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972b407f-88ed-4de6-8798-fc322366f57f None admin] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:36:08.862232 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972b407f-88ed-4de6-8798-fc322366f57f None admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:36:08.862508 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972b407f-88ed-4de6-8798-fc322366f57f None admin] PATH_INFO: `/v3/roles/member` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:36:08.866244 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 126/246] 10.222.0.20 () {58 vars in 1282 bytes} [Mon Aug 1 16:36:08 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 473 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 01 16:36:08.872225 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6daaacf8-2097-4739-af00-7087ba3978e1 None admin] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:36:08.876208 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-972b407f-88ed-4de6-8798-fc322366f57f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 16:36:08.881052 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application [None req-972b407f-88ed-4de6-8798-fc322366f57f None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 01 16:36:08.881052 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 16:36:08.881052 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 16:36:08.881052 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 16:36:08.881052 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 16:36:08.881052 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 16:36:08.881052 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 16:36:08.881052 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 16:36:08.881052 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 16:36:08.881052 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 16:36:08.881052 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 16:36:08.881052 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 16:36:08.881052 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 16:36:08.881052 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 16:36:08.881052 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 16:36:08.881052 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 16:36:08.881949 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 16:36:08.881949 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 16:36:08.881949 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 16:36:08.881949 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 16:36:08.881949 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 16:36:08.881949 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 16:36:08.881949 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 16:36:08.881949 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 16:36:08.881949 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 16:36:08.881949 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 16:36:08.881949 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 16:36:08.881949 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application with Lock( Aug 01 16:36:08.881949 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 16:36:08.881949 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self._enter() Aug 01 16:36:08.881949 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 16:36:08.881949 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 16:36:08.883781 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 16:36:08.883781 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self.creator() Aug 01 16:36:08.883781 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 16:36:08.883781 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application created_value = creator( Aug 01 16:36:08.883781 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 16:36:08.883781 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 16:36:08.883781 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 16:36:08.883781 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 16:36:08.883781 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 16:36:08.883781 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 16:36:08.883781 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 01 16:36:08.883781 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application Aug 01 16:36:08.883781 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 121/247] 10.222.0.20 () {56 vars in 1099 bytes} [Mon Aug 1 16:36:08 2022] GET /identity/v3/roles/member => generated 84 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 16:36:08.889492 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-972b407f-88ed-4de6-8798-fc322366f57f None admin] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:36:08.932598 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG dbcounter [-] [87058] Writing DB stats keystone:SELECT=100 {{(pid=87058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:36:08.998837 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6daaacf8-2097-4739-af00-7087ba3978e1 None admin] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:36:09.013912 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-972b407f-88ed-4de6-8798-fc322366f57f None admin] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:36:09.038688 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG dbcounter [-] [87057] Writing DB stats keystone:SELECT=100 {{(pid=87057) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:36:09.044852 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-419fb242-da95-4950-a633-fe9b6f266380 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:36:09.046145 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419fb242-da95-4950-a633-fe9b6f266380 None admin] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:36:09.046430 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419fb242-da95-4950-a633-fe9b6f266380 None admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:36:09.046702 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419fb242-da95-4950-a633-fe9b6f266380 None admin] PATH_INFO: `/v3/projects/d9a43129dd714d05bd409ffb0875f7e6` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:36:09.052473 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-419fb242-da95-4950-a633-fe9b6f266380 None admin] RBAC: Authorizing `identity:get_project(project_id=d9a43129dd714d05bd409ffb0875f7e6)` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:36:09.052473 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-419fb242-da95-4950-a633-fe9b6f266380 None admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:36:09.052473 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 127/248] 10.222.0.20 () {56 vars in 1186 bytes} [Mon Aug 1 16:36:08 2022] GET /identity/v3/projects/d9a43129dd714d05bd409ffb0875f7e6 => generated 302 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:36:09.061766 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-419fb242-da95-4950-a633-fe9b6f266380 None admin] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:36:09.071359 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4102c208-d3b1-4837-b1f5-1a1a755caaf5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:36:09.072247 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4102c208-d3b1-4837-b1f5-1a1a755caaf5 None admin] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:36:09.072763 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4102c208-d3b1-4837-b1f5-1a1a755caaf5 None admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:36:09.073176 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4102c208-d3b1-4837-b1f5-1a1a755caaf5 None admin] PATH_INFO: `/v3/roles` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:36:09.074078 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4102c208-d3b1-4837-b1f5-1a1a755caaf5 None admin] RBAC: Adding query filter params (name=member) {{(pid=87057) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:36:09.074671 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4102c208-d3b1-4837-b1f5-1a1a755caaf5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:36:09.076314 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4102c208-d3b1-4837-b1f5-1a1a755caaf5 None admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:36:09.090361 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 122/249] 10.222.0.20 () {56 vars in 1101 bytes} [Mon Aug 1 16:36:08 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 16:36:09.096127 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4102c208-d3b1-4837-b1f5-1a1a755caaf5 None admin] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:36:09.180208 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-419fb242-da95-4950-a633-fe9b6f266380 None admin] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:36:09.209794 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG dbcounter [-] [87058] Writing DB stats keystone:SELECT=100 {{(pid=87058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:36:09.214920 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4102c208-d3b1-4837-b1f5-1a1a755caaf5 None admin] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:36:09.238296 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f9f684d-16f8-41ca-907e-143c2c778796 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:36:09.238934 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9f684d-16f8-41ca-907e-143c2c778796 None admin] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:36:09.239213 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9f684d-16f8-41ca-907e-143c2c778796 None admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:36:09.240606 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9f684d-16f8-41ca-907e-143c2c778796 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:36:09.241436 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f9f684d-16f8-41ca-907e-143c2c778796 None admin] RBAC: Adding query filter params (role.id=02c40c90308547dd8ac5aaad0da8d68c, scope.project.id=d9a43129dd714d05bd409ffb0875f7e6, user.id=e4ba5c4a5a5a456ea72804daff75daa1) {{(pid=87058) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:36:09.241833 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f9f684d-16f8-41ca-907e-143c2c778796 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:36:09.244026 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f9f684d-16f8-41ca-907e-143c2c778796 None admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:36:09.252242 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 128/250] 10.222.0.20 () {56 vars in 1374 bytes} [Mon Aug 1 16:36:09 2022] GET /identity/v3/role_assignments?user.id=e4ba5c4a5a5a456ea72804daff75daa1&scope.project.id=d9a43129dd714d05bd409ffb0875f7e6&role.id=02c40c90308547dd8ac5aaad0da8d68c => generated 259 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 16:36:09.255129 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46ab2743-0102-48cf-940c-f55ea75bd09c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:36:09.257043 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ab2743-0102-48cf-940c-f55ea75bd09c None admin] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:36:09.257043 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ab2743-0102-48cf-940c-f55ea75bd09c None admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:36:09.257043 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ab2743-0102-48cf-940c-f55ea75bd09c None admin] PATH_INFO: `/v3/users/ac697afa751948bf87a605cf4aab7c65` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:36:09.257638 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46ab2743-0102-48cf-940c-f55ea75bd09c None admin] RBAC: Authorizing `identity:get_user(user_id=ac697afa751948bf87a605cf4aab7c65)` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:36:09.260309 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46ab2743-0102-48cf-940c-f55ea75bd09c None admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:36:09.263644 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 123/251] 10.222.0.20 () {56 vars in 1177 bytes} [Mon Aug 1 16:36:09 2022] GET /identity/v3/users/ac697afa751948bf87a605cf4aab7c65 => generated 297 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 16:36:09.271702 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f9f684d-16f8-41ca-907e-143c2c778796 None admin] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:36:09.477388 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f9f684d-16f8-41ca-907e-143c2c778796 None admin] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:36:09.538786 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97d27522-0677-44bc-98d1-192c4e2380dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:36:09.540622 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d27522-0677-44bc-98d1-192c4e2380dc None admin] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:36:09.541101 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d27522-0677-44bc-98d1-192c4e2380dc None admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:36:09.541639 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d27522-0677-44bc-98d1-192c4e2380dc None admin] PATH_INFO: `/v3/projects/d9a43129dd714d05bd409ffb0875f7e6` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:36:09.545650 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97d27522-0677-44bc-98d1-192c4e2380dc None admin] RBAC: Authorizing `identity:get_project(project_id=d9a43129dd714d05bd409ffb0875f7e6)` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:36:09.547473 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97d27522-0677-44bc-98d1-192c4e2380dc None admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:36:09.549659 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 129/252] 10.222.0.20 () {56 vars in 1186 bytes} [Mon Aug 1 16:36:09 2022] GET /identity/v3/projects/d9a43129dd714d05bd409ffb0875f7e6 => generated 302 bytes in 281 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:36:09.556826 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46ab2743-0102-48cf-940c-f55ea75bd09c None admin] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:36:09.558529 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a47bb8-12f8-4258-9b32-3e370a4cae84 None None] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:36:09.559023 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a47bb8-12f8-4258-9b32-3e370a4cae84 None None] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:36:09.559491 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a47bb8-12f8-4258-9b32-3e370a4cae84 None None] PATH_INFO: `/` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:36:09.562595 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 130/253] 10.222.0.20 () {54 vars in 926 bytes} [Mon Aug 1 16:36:09 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 01 16:36:09.568528 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff99e38-93d4-4d7f-b687-8b4503f26a01 None None] REQUEST_METHOD: `POST` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:36:09.569092 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff99e38-93d4-4d7f-b687-8b4503f26a01 None None] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:36:09.569586 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff99e38-93d4-4d7f-b687-8b4503f26a01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:36:09.598892 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.auth.core [None req-7ff99e38-93d4-4d7f-b687-8b4503f26a01 None None] MFA Rules not processed for user `1007f8e379b74e169b98fb7c3c0ba5b4`. Rule list: `[]` (Enabled: `True`). {{(pid=87058) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:36:09.600481 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.fernet_utils [None req-7ff99e38-93d4-4d7f-b687-8b4503f26a01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:36:09.652327 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG dbcounter [-] [87057] Writing DB stats keystone:SELECT=100 {{(pid=87057) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:36:09.660927 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG dbcounter [-] [87058] Writing DB stats keystone:SELECT=100 {{(pid=87058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:36:09.687499 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 131/254] 10.222.0.20 () {58 vars in 1023 bytes} [Mon Aug 1 16:36:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 01 16:36:09.724519 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ff99e38-93d4-4d7f-b687-8b4503f26a01 None None] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:36:09.758511 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46ab2743-0102-48cf-940c-f55ea75bd09c None admin] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:36:09.852936 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1804081a-79c8-4b86-a23e-4753a237e790 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:36:09.853628 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1804081a-79c8-4b86-a23e-4753a237e790 None admin] REQUEST_METHOD: `PUT` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:36:09.853934 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1804081a-79c8-4b86-a23e-4753a237e790 None admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:36:09.854256 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1804081a-79c8-4b86-a23e-4753a237e790 None admin] PATH_INFO: `/v3/projects/d9a43129dd714d05bd409ffb0875f7e6/users/ac697afa751948bf87a605cf4aab7c65/roles/0999d3da63a545f18ff3252afdc90518` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:36:09.863700 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1804081a-79c8-4b86-a23e-4753a237e790 None admin] RBAC: Authorizing `identity:create_grant(project_id=d9a43129dd714d05bd409ffb0875f7e6, user_id=ac697afa751948bf87a605cf4aab7c65, role_id=0999d3da63a545f18ff3252afdc90518)` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:36:09.865212 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1804081a-79c8-4b86-a23e-4753a237e790 None admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:36:09.868539 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ff99e38-93d4-4d7f-b687-8b4503f26a01 None None] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:36:09.918093 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a24fc21-4332-497b-b4a0-5083c3f4e4de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:36:09.918820 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a24fc21-4332-497b-b4a0-5083c3f4e4de None admin] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:36:09.919305 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a24fc21-4332-497b-b4a0-5083c3f4e4de None admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:36:09.919819 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a24fc21-4332-497b-b4a0-5083c3f4e4de None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:36:09.938209 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a24fc21-4332-497b-b4a0-5083c3f4e4de None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 16:36:09.945066 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application [None req-5a24fc21-4332-497b-b4a0-5083c3f4e4de None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 01 16:36:09.945066 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 16:36:09.945066 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 16:36:09.945066 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 16:36:09.945066 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 16:36:09.945066 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 16:36:09.945066 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 16:36:09.945066 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 16:36:09.945066 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 16:36:09.945066 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 16:36:09.945066 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 16:36:09.945066 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 16:36:09.945066 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 16:36:09.945066 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 16:36:09.945066 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 16:36:09.945066 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 16:36:09.946065 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 16:36:09.946065 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 16:36:09.946065 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 16:36:09.946065 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 16:36:09.946065 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 16:36:09.946065 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 16:36:09.946065 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 16:36:09.946065 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 16:36:09.946065 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 16:36:09.946065 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 16:36:09.946065 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 16:36:09.946065 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application with Lock( Aug 01 16:36:09.946065 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 16:36:09.946065 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self._enter() Aug 01 16:36:09.946065 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 16:36:09.946065 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 16:36:09.947442 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 16:36:09.947442 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self.creator() Aug 01 16:36:09.947442 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 16:36:09.947442 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application created_value = creator( Aug 01 16:36:09.947442 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 16:36:09.947442 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 16:36:09.947442 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 16:36:09.947442 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 16:36:09.947442 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 16:36:09.947442 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 16:36:09.947442 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 01 16:36:09.947442 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application Aug 01 16:36:09.949293 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 132/255] 10.222.0.20 () {56 vars in 1114 bytes} [Mon Aug 1 16:36:09 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 01 16:36:09.956259 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a24fc21-4332-497b-b4a0-5083c3f4e4de None admin] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:36:09.963102 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 124/256] 10.222.0.20 () {58 vars in 1439 bytes} [Mon Aug 1 16:36:09 2022] PUT /identity/v3/projects/d9a43129dd714d05bd409ffb0875f7e6/users/ac697afa751948bf87a605cf4aab7c65/roles/0999d3da63a545f18ff3252afdc90518 => generated 0 bytes in 407 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 16:36:10.019752 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG dbcounter [-] [87058] Writing DB stats keystone:SELECT=100 {{(pid=87058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:36:10.050897 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d9e5e0-a8d0-4e65-9bde-c369db20ef6d None None] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:36:10.051198 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d9e5e0-a8d0-4e65-9bde-c369db20ef6d None None] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:36:10.051445 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d9e5e0-a8d0-4e65-9bde-c369db20ef6d None None] PATH_INFO: `/` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:36:10.053129 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 125/257] 10.222.0.20 () {54 vars in 926 bytes} [Mon Aug 1 16:36:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 01 16:36:10.059227 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f1c609-269c-4246-9835-74d1c7c0d2d5 None None] REQUEST_METHOD: `POST` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:36:10.059684 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f1c609-269c-4246-9835-74d1c7c0d2d5 None None] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:36:10.060151 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f1c609-269c-4246-9835-74d1c7c0d2d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:36:10.089675 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a24fc21-4332-497b-b4a0-5083c3f4e4de None admin] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:36:10.107127 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.auth.core [None req-64f1c609-269c-4246-9835-74d1c7c0d2d5 None None] MFA Rules not processed for user `1007f8e379b74e169b98fb7c3c0ba5b4`. Rule list: `[]` (Enabled: `True`). {{(pid=87057) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:36:10.108424 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.fernet_utils [None req-64f1c609-269c-4246-9835-74d1c7c0d2d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87057) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:36:10.128830 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab64c78c-c3b1-4796-8639-05fb853a73aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:36:10.129470 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab64c78c-c3b1-4796-8639-05fb853a73aa None admin] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:36:10.129753 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab64c78c-c3b1-4796-8639-05fb853a73aa None admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:36:10.130053 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab64c78c-c3b1-4796-8639-05fb853a73aa None admin] PATH_INFO: `/v3/roles` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:36:10.130798 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab64c78c-c3b1-4796-8639-05fb853a73aa None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=87058) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:36:10.131110 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab64c78c-c3b1-4796-8639-05fb853a73aa None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:36:10.132540 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab64c78c-c3b1-4796-8639-05fb853a73aa None admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:36:10.141908 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 133/258] 10.222.0.20 () {56 vars in 1111 bytes} [Mon Aug 1 16:36:09 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 16:36:10.148137 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab64c78c-c3b1-4796-8639-05fb853a73aa None admin] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:36:10.263039 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab64c78c-c3b1-4796-8639-05fb853a73aa None admin] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:36:10.264583 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG dbcounter [-] [87058] Writing DB stats keystone:SELECT=100 {{(pid=87058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:36:10.284871 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 126/259] 10.222.0.20 () {58 vars in 1023 bytes} [Mon Aug 1 16:36:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 226 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 01 16:36:10.301964 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5884f488-63db-4cbc-acee-854b5f4bf07e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:36:10.302650 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5884f488-63db-4cbc-acee-854b5f4bf07e None admin] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:36:10.302934 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5884f488-63db-4cbc-acee-854b5f4bf07e None admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:36:10.303209 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5884f488-63db-4cbc-acee-854b5f4bf07e None admin] PATH_INFO: `/v3/users/bccb7b2b19304928888200525af9748a` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:36:10.304409 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5884f488-63db-4cbc-acee-854b5f4bf07e None admin] RBAC: Authorizing `identity:get_user(user_id=bccb7b2b19304928888200525af9748a)` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:36:10.306815 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5884f488-63db-4cbc-acee-854b5f4bf07e None admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:36:10.308632 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 134/260] 10.222.0.20 () {56 vars in 1177 bytes} [Mon Aug 1 16:36:10 2022] GET /identity/v3/users/bccb7b2b19304928888200525af9748a => generated 283 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 16:36:10.314399 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f1c609-269c-4246-9835-74d1c7c0d2d5 None None] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:36:10.322666 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5884f488-63db-4cbc-acee-854b5f4bf07e None admin] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:36:10.404405 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG dbcounter [-] [87057] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87057) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:36:10.499037 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f1c609-269c-4246-9835-74d1c7c0d2d5 None None] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:36:10.536243 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5884f488-63db-4cbc-acee-854b5f4bf07e None admin] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:36:10.583645 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ebd1f63-cad5-40eb-b590-bfe1cd8b22f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:36:10.584431 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ebd1f63-cad5-40eb-b590-bfe1cd8b22f8 None admin] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:36:10.584800 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ebd1f63-cad5-40eb-b590-bfe1cd8b22f8 None admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:36:10.585164 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ebd1f63-cad5-40eb-b590-bfe1cd8b22f8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:36:10.634562 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d949737-24e0-459f-ba1b-b43c7dc525a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:36:10.635475 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d949737-24e0-459f-ba1b-b43c7dc525a3 None admin] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:36:10.635907 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d949737-24e0-459f-ba1b-b43c7dc525a3 None admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:36:10.636331 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d949737-24e0-459f-ba1b-b43c7dc525a3 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:36:10.666938 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ebd1f63-cad5-40eb-b590-bfe1cd8b22f8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:36:10.667668 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d949737-24e0-459f-ba1b-b43c7dc525a3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 16:36:10.668634 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ebd1f63-cad5-40eb-b590-bfe1cd8b22f8 None admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:36:10.672976 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application [None req-3d949737-24e0-459f-ba1b-b43c7dc525a3 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 16:36:10.672976 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 16:36:10.672976 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 16:36:10.672976 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 16:36:10.672976 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 16:36:10.672976 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 16:36:10.672976 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 16:36:10.672976 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 16:36:10.672976 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 16:36:10.672976 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 16:36:10.672976 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 16:36:10.672976 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 16:36:10.672976 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 16:36:10.672976 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 16:36:10.672976 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 16:36:10.672976 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 16:36:10.674174 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 16:36:10.674174 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 16:36:10.674174 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 16:36:10.674174 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 16:36:10.674174 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 16:36:10.674174 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 16:36:10.674174 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 16:36:10.674174 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 16:36:10.674174 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 16:36:10.674174 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 16:36:10.674174 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 16:36:10.674174 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application with Lock( Aug 01 16:36:10.674174 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 16:36:10.674174 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self._enter() Aug 01 16:36:10.674174 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 16:36:10.674174 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 16:36:10.675176 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 16:36:10.675176 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self.creator() Aug 01 16:36:10.675176 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 16:36:10.675176 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application created_value = creator( Aug 01 16:36:10.675176 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 16:36:10.675176 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 16:36:10.675176 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 16:36:10.675176 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 16:36:10.675176 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 16:36:10.675176 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 16:36:10.675176 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 16:36:10.675176 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application Aug 01 16:36:10.677494 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 135/261] 10.222.0.20 () {56 vars in 1096 bytes} [Mon Aug 1 16:36:10 2022] GET /identity/v3/roles/admin => generated 83 bytes in 355 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 16:36:10.683418 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d949737-24e0-459f-ba1b-b43c7dc525a3 None admin] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:36:10.716376 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG dbcounter [-] [87057] Writing DB stats keystone:SELECT=100 {{(pid=87057) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:36:10.770354 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG dbcounter [-] [87058] Writing DB stats keystone:SELECT=100 {{(pid=87058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:36:10.795854 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 127/262] 10.222.0.20 () {58 vars in 1282 bytes} [Mon Aug 1 16:36:10 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 482 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 01 16:36:10.802705 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ebd1f63-cad5-40eb-b590-bfe1cd8b22f8 None admin] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:36:10.938335 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d949737-24e0-459f-ba1b-b43c7dc525a3 None admin] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:36:11.001755 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d7c894d-05c9-438c-8570-c2c3f0f80330 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:36:11.002416 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7c894d-05c9-438c-8570-c2c3f0f80330 None admin] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:36:11.002696 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7c894d-05c9-438c-8570-c2c3f0f80330 None admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:36:11.002972 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7c894d-05c9-438c-8570-c2c3f0f80330 None admin] PATH_INFO: `/v3/roles` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:36:11.003701 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d7c894d-05c9-438c-8570-c2c3f0f80330 None admin] RBAC: Adding query filter params (name=admin) {{(pid=87058) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:36:11.004007 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d7c894d-05c9-438c-8570-c2c3f0f80330 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:36:11.005389 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d7c894d-05c9-438c-8570-c2c3f0f80330 None admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:36:11.015610 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 136/263] 10.222.0.20 () {56 vars in 1099 bytes} [Mon Aug 1 16:36:10 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 332 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 16:36:11.022451 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d7c894d-05c9-438c-8570-c2c3f0f80330 None admin] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:36:11.060945 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ebd1f63-cad5-40eb-b590-bfe1cd8b22f8 None admin] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:36:11.144501 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c7daf6-afe8-45f4-a012-0d0a1f551aea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:36:11.145245 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c7daf6-afe8-45f4-a012-0d0a1f551aea None admin] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:36:11.145533 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c7daf6-afe8-45f4-a012-0d0a1f551aea None admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:36:11.145801 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c7daf6-afe8-45f4-a012-0d0a1f551aea None admin] PATH_INFO: `/v3/projects/d9a43129dd714d05bd409ffb0875f7e6` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:36:11.150164 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5c7daf6-afe8-45f4-a012-0d0a1f551aea None admin] RBAC: Authorizing `identity:get_project(project_id=d9a43129dd714d05bd409ffb0875f7e6)` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:36:11.151734 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5c7daf6-afe8-45f4-a012-0d0a1f551aea None admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:36:11.153944 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 128/264] 10.222.0.20 () {56 vars in 1186 bytes} [Mon Aug 1 16:36:10 2022] GET /identity/v3/projects/d9a43129dd714d05bd409ffb0875f7e6 => generated 302 bytes in 352 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 16:36:11.175960 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c7daf6-afe8-45f4-a012-0d0a1f551aea None admin] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:36:11.187889 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG dbcounter [-] [87058] Writing DB stats keystone:SELECT=100 {{(pid=87058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:36:11.196742 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d7c894d-05c9-438c-8570-c2c3f0f80330 None admin] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:36:11.199762 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG dbcounter [-] [87057] Writing DB stats keystone:SELECT=100 {{(pid=87057) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:36:11.232620 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-457e42b1-093e-4f45-92c3-11f691c2878a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:36:11.233268 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457e42b1-093e-4f45-92c3-11f691c2878a None admin] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:36:11.233549 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457e42b1-093e-4f45-92c3-11f691c2878a None admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:36:11.233825 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457e42b1-093e-4f45-92c3-11f691c2878a None admin] PATH_INFO: `/v3/users/bccb7b2b19304928888200525af9748a` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:36:11.235104 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-457e42b1-093e-4f45-92c3-11f691c2878a None admin] RBAC: Authorizing `identity:get_user(user_id=bccb7b2b19304928888200525af9748a)` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:36:11.236624 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-457e42b1-093e-4f45-92c3-11f691c2878a None admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:36:11.238346 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 137/265] 10.222.0.20 () {56 vars in 1177 bytes} [Mon Aug 1 16:36:11 2022] GET /identity/v3/users/bccb7b2b19304928888200525af9748a => generated 283 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:36:11.248958 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-457e42b1-093e-4f45-92c3-11f691c2878a None admin] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:36:11.434074 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c7daf6-afe8-45f4-a012-0d0a1f551aea None admin] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:36:11.434436 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-457e42b1-093e-4f45-92c3-11f691c2878a None admin] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:36:11.513476 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-523f403c-f134-49db-9d00-64f8fa9a075c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:36:11.514134 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523f403c-f134-49db-9d00-64f8fa9a075c None admin] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:36:11.514411 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523f403c-f134-49db-9d00-64f8fa9a075c None admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:36:11.514679 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523f403c-f134-49db-9d00-64f8fa9a075c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:36:11.515472 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-523f403c-f134-49db-9d00-64f8fa9a075c None admin] RBAC: Adding query filter params (role.id=3fac71a39f3f4ed198a3579064c30268, scope.project.id=d9a43129dd714d05bd409ffb0875f7e6, user.id=bccb7b2b19304928888200525af9748a) {{(pid=87057) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:36:11.515775 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-523f403c-f134-49db-9d00-64f8fa9a075c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:36:11.516246 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d7d80ec-b1d9-4729-ba97-c41d07b60d59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:36:11.517059 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7d80ec-b1d9-4729-ba97-c41d07b60d59 None admin] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:36:11.517221 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-523f403c-f134-49db-9d00-64f8fa9a075c None admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:36:11.517920 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7d80ec-b1d9-4729-ba97-c41d07b60d59 None admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:36:11.518385 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7d80ec-b1d9-4729-ba97-c41d07b60d59 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:36:11.533276 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 129/266] 10.222.0.20 () {56 vars in 1374 bytes} [Mon Aug 1 16:36:11 2022] GET /identity/v3/role_assignments?user.id=bccb7b2b19304928888200525af9748a&scope.project.id=d9a43129dd714d05bd409ffb0875f7e6&role.id=3fac71a39f3f4ed198a3579064c30268 => generated 608 bytes in 358 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 16:36:11.574734 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG dbcounter [-] [87058] Writing DB stats keystone:SELECT=100 {{(pid=87058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:36:11.640440 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d7d80ec-b1d9-4729-ba97-c41d07b60d59 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:36:11.641901 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d7d80ec-b1d9-4729-ba97-c41d07b60d59 None admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:36:11.765350 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 138/267] 10.222.0.20 () {58 vars in 1282 bytes} [Mon Aug 1 16:36:11 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 517 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 01 16:36:11.771276 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-523f403c-f134-49db-9d00-64f8fa9a075c None admin] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:36:11.859086 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG dbcounter [-] [87057] Writing DB stats keystone:SELECT=100 {{(pid=87057) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:36:11.895191 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-523f403c-f134-49db-9d00-64f8fa9a075c None admin] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:36:11.948203 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9651b6c0-ece7-4fdc-9486-3fb779062e03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:36:11.948911 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9651b6c0-ece7-4fdc-9486-3fb779062e03 None admin] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:36:11.949193 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9651b6c0-ece7-4fdc-9486-3fb779062e03 None admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:36:11.949522 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9651b6c0-ece7-4fdc-9486-3fb779062e03 None admin] PATH_INFO: `/v3/projects/d9a43129dd714d05bd409ffb0875f7e6` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:36:11.954724 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9651b6c0-ece7-4fdc-9486-3fb779062e03 None admin] RBAC: Authorizing `identity:get_project(project_id=d9a43129dd714d05bd409ffb0875f7e6)` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:36:11.956177 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9651b6c0-ece7-4fdc-9486-3fb779062e03 None admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:36:11.958186 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 130/268] 10.222.0.20 () {56 vars in 1186 bytes} [Mon Aug 1 16:36:11 2022] GET /identity/v3/projects/d9a43129dd714d05bd409ffb0875f7e6 => generated 302 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:36:11.964736 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d7d80ec-b1d9-4729-ba97-c41d07b60d59 None admin] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:36:12.027249 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG dbcounter [-] [87058] Writing DB stats keystone:SELECT=100 {{(pid=87058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:36:12.050992 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c4c90f-2590-4d96-90fa-9af9daaae96f None None] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:36:12.051339 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c4c90f-2590-4d96-90fa-9af9daaae96f None None] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:36:12.051546 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c4c90f-2590-4d96-90fa-9af9daaae96f None None] PATH_INFO: `/` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:36:12.052904 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 131/269] 10.222.0.20 () {54 vars in 926 bytes} [Mon Aug 1 16:36:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 01 16:36:12.058400 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05afaa8-c0fc-48ff-8f09-122135021cd4 None None] REQUEST_METHOD: `POST` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:36:12.058672 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05afaa8-c0fc-48ff-8f09-122135021cd4 None None] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:36:12.059034 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05afaa8-c0fc-48ff-8f09-122135021cd4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:36:12.089613 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.auth.core [None req-a05afaa8-c0fc-48ff-8f09-122135021cd4 None None] MFA Rules not processed for user `1007f8e379b74e169b98fb7c3c0ba5b4`. Rule list: `[]` (Enabled: `True`). {{(pid=87057) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:36:12.090850 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.fernet_utils [None req-a05afaa8-c0fc-48ff-8f09-122135021cd4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87057) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:36:12.104218 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d7d80ec-b1d9-4729-ba97-c41d07b60d59 None admin] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:36:12.175390 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c481daf-8c02-461e-8a92-841436030355 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:36:12.176395 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c481daf-8c02-461e-8a92-841436030355 None admin] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:36:12.176868 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c481daf-8c02-461e-8a92-841436030355 None admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:36:12.177485 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c481daf-8c02-461e-8a92-841436030355 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:36:12.178516 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c481daf-8c02-461e-8a92-841436030355 None admin] RBAC: Adding query filter params (role.id=440bb943dd5b46fea038b903b6eaaf81, scope.project.id=d9a43129dd714d05bd409ffb0875f7e6, user.id=bccb7b2b19304928888200525af9748a) {{(pid=87058) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:36:12.179041 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c481daf-8c02-461e-8a92-841436030355 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:36:12.179629 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 132/270] 10.222.0.20 () {58 vars in 1023 bytes} [Mon Aug 1 16:36:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 01 16:36:12.181235 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c481daf-8c02-461e-8a92-841436030355 None admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:36:12.190824 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 139/271] 10.222.0.20 () {56 vars in 1374 bytes} [Mon Aug 1 16:36:11 2022] GET /identity/v3/role_assignments?user.id=bccb7b2b19304928888200525af9748a&scope.project.id=d9a43129dd714d05bd409ffb0875f7e6&role.id=440bb943dd5b46fea038b903b6eaaf81 => generated 608 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:36:12.216153 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a05afaa8-c0fc-48ff-8f09-122135021cd4 None None] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:36:12.281268 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e345b04d-2692-4f94-9000-0aa648896a20 None None] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:36:12.281461 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e345b04d-2692-4f94-9000-0aa648896a20 None None] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:36:12.281738 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e345b04d-2692-4f94-9000-0aa648896a20 None None] PATH_INFO: `/` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:36:12.283418 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 140/272] 10.222.0.20 () {54 vars in 926 bytes} [Mon Aug 1 16:36:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 16:36:12.289518 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ebb03a-71fc-4d0a-979d-fdc155f310ef None None] REQUEST_METHOD: `POST` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:36:12.289894 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ebb03a-71fc-4d0a-979d-fdc155f310ef None None] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:36:12.290205 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ebb03a-71fc-4d0a-979d-fdc155f310ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:36:12.315256 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.auth.core [None req-16ebb03a-71fc-4d0a-979d-fdc155f310ef None None] MFA Rules not processed for user `1007f8e379b74e169b98fb7c3c0ba5b4`. Rule list: `[]` (Enabled: `True`). {{(pid=87058) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:36:12.316913 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.fernet_utils [None req-16ebb03a-71fc-4d0a-979d-fdc155f310ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:36:12.341995 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG dbcounter [-] [87057] Writing DB stats keystone:SELECT=100 {{(pid=87057) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:36:12.375881 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a05afaa8-c0fc-48ff-8f09-122135021cd4 None None] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:36:12.403731 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 141/273] 10.222.0.20 () {58 vars in 1023 bytes} [Mon Aug 1 16:36:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 16:36:12.405571 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2412bb05-b25b-478b-bcba-76b36053474b None None] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:36:12.405859 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2412bb05-b25b-478b-bcba-76b36053474b None None] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:36:12.406155 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2412bb05-b25b-478b-bcba-76b36053474b None None] PATH_INFO: `/` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:36:12.407742 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 142/274] 10.222.0.20 () {54 vars in 926 bytes} [Mon Aug 1 16:36:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 16:36:12.413597 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c33b2b-1e7c-44b2-80ea-35eb296a598b None None] REQUEST_METHOD: `POST` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:36:12.414490 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c33b2b-1e7c-44b2-80ea-35eb296a598b None None] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:36:12.414718 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c33b2b-1e7c-44b2-80ea-35eb296a598b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:36:12.418391 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cb70421-57cf-4a52-83a5-0c7de8348662 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:36:12.419265 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb70421-57cf-4a52-83a5-0c7de8348662 None admin] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:36:12.419760 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb70421-57cf-4a52-83a5-0c7de8348662 None admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:36:12.420249 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb70421-57cf-4a52-83a5-0c7de8348662 None admin] PATH_INFO: `/v3/domains/default` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:36:12.421820 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cb70421-57cf-4a52-83a5-0c7de8348662 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:36:12.423688 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cb70421-57cf-4a52-83a5-0c7de8348662 None admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:36:12.425544 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 133/275] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Aug 1 16:36:12 2022] GET /identity/v3/domains/default => generated 199 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:36:12.430914 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cb70421-57cf-4a52-83a5-0c7de8348662 None admin] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:36:12.438981 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.auth.core [None req-42c33b2b-1e7c-44b2-80ea-35eb296a598b None None] MFA Rules not processed for user `1007f8e379b74e169b98fb7c3c0ba5b4`. Rule list: `[]` (Enabled: `True`). {{(pid=87058) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:36:12.440320 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.fernet_utils [None req-42c33b2b-1e7c-44b2-80ea-35eb296a598b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:36:12.504818 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG dbcounter [-] [87058] Writing DB stats keystone:SELECT=100 {{(pid=87058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:36:12.527240 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 143/276] 10.222.0.20 () {58 vars in 1023 bytes} [Mon Aug 1 16:36:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 16:36:12.528403 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42c33b2b-1e7c-44b2-80ea-35eb296a598b None None] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:36:12.565206 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cb70421-57cf-4a52-83a5-0c7de8348662 None admin] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:36:12.601342 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37006982-4d26-4e04-ba53-315e221395b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:36:12.601978 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37006982-4d26-4e04-ba53-315e221395b6 None admin] REQUEST_METHOD: `POST` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:36:12.602383 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37006982-4d26-4e04-ba53-315e221395b6 None admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:36:12.602668 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37006982-4d26-4e04-ba53-315e221395b6 None admin] PATH_INFO: `/v3/users` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:36:12.603359 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37006982-4d26-4e04-ba53-315e221395b6 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:36:12.604820 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37006982-4d26-4e04-ba53-315e221395b6 None admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:36:12.615029 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG dbcounter [-] [87057] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=87057) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:36:12.640322 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42c33b2b-1e7c-44b2-80ea-35eb296a598b None None] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:36:12.674455 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-534e1cc0-0045-4711-b32b-0950aa628170 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:36:12.675096 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-534e1cc0-0045-4711-b32b-0950aa628170 None admin] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:36:12.675370 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-534e1cc0-0045-4711-b32b-0950aa628170 None admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:36:12.675638 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-534e1cc0-0045-4711-b32b-0950aa628170 None admin] PATH_INFO: `/v3/roles/member` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:36:12.685721 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-534e1cc0-0045-4711-b32b-0950aa628170 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 16:36:12.690662 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application [None req-534e1cc0-0045-4711-b32b-0950aa628170 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 01 16:36:12.690662 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 16:36:12.690662 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 16:36:12.690662 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 16:36:12.690662 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 16:36:12.690662 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 16:36:12.690662 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 16:36:12.690662 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 16:36:12.690662 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 16:36:12.690662 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 16:36:12.690662 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 16:36:12.690662 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 16:36:12.690662 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 16:36:12.690662 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 16:36:12.690662 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 16:36:12.690662 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 16:36:12.691730 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 16:36:12.691730 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 16:36:12.691730 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 16:36:12.691730 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 16:36:12.691730 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 16:36:12.691730 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 16:36:12.691730 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 16:36:12.691730 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 16:36:12.691730 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 16:36:12.691730 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 16:36:12.691730 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 16:36:12.691730 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application with Lock( Aug 01 16:36:12.691730 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 16:36:12.691730 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self._enter() Aug 01 16:36:12.691730 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 16:36:12.691730 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 16:36:12.692622 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 16:36:12.692622 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self.creator() Aug 01 16:36:12.692622 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 16:36:12.692622 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application created_value = creator( Aug 01 16:36:12.692622 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 16:36:12.692622 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 16:36:12.692622 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 16:36:12.692622 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 16:36:12.692622 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 16:36:12.692622 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 16:36:12.692622 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 01 16:36:12.692622 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application Aug 01 16:36:12.692622 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 144/277] 10.222.0.20 () {56 vars in 1099 bytes} [Mon Aug 1 16:36:12 2022] GET /identity/v3/roles/member => generated 84 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 16:36:12.692622 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-534e1cc0-0045-4711-b32b-0950aa628170 None admin] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:36:12.725923 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 134/278] 10.222.0.20 () {60 vars in 1132 bytes} [Mon Aug 1 16:36:12 2022] POST /identity/v3/users => generated 293 bytes in 285 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 16:36:12.725923 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37006982-4d26-4e04-ba53-315e221395b6 None admin] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:36:12.770203 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG dbcounter [-] [87058] Writing DB stats keystone:SELECT=100 {{(pid=87058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:36:12.834582 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37006982-4d26-4e04-ba53-315e221395b6 None admin] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:36:12.855881 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-534e1cc0-0045-4711-b32b-0950aa628170 None admin] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:36:12.878282 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b322cc39-dc6d-4c65-9132-3730c172df35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:36:12.879585 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b322cc39-dc6d-4c65-9132-3730c172df35 None admin] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:36:12.880080 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b322cc39-dc6d-4c65-9132-3730c172df35 None admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:36:12.880581 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b322cc39-dc6d-4c65-9132-3730c172df35 None admin] PATH_INFO: `/v3/roles` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:36:12.881623 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b322cc39-dc6d-4c65-9132-3730c172df35 None admin] RBAC: Adding query filter params (name=member) {{(pid=87057) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:36:12.882169 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b322cc39-dc6d-4c65-9132-3730c172df35 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:36:12.883883 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b322cc39-dc6d-4c65-9132-3730c172df35 None admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:36:12.893551 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 135/279] 10.222.0.20 () {56 vars in 1101 bytes} [Mon Aug 1 16:36:12 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:36:12.901285 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b322cc39-dc6d-4c65-9132-3730c172df35 None admin] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:36:12.904827 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b8dc767-9a39-438a-bdd1-9858d489b9ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:36:12.905434 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8dc767-9a39-438a-bdd1-9858d489b9ef None admin] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:36:12.905715 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8dc767-9a39-438a-bdd1-9858d489b9ef None admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:36:12.905986 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8dc767-9a39-438a-bdd1-9858d489b9ef None admin] PATH_INFO: `/v3/roles/reader` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:36:12.922578 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b8dc767-9a39-438a-bdd1-9858d489b9ef None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 16:36:12.928521 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application [None req-4b8dc767-9a39-438a-bdd1-9858d489b9ef None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 01 16:36:12.928521 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 16:36:12.928521 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 16:36:12.928521 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 16:36:12.928521 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 16:36:12.928521 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 16:36:12.928521 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 16:36:12.928521 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 16:36:12.928521 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 16:36:12.928521 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 16:36:12.928521 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 16:36:12.928521 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 16:36:12.928521 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 16:36:12.928521 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 16:36:12.928521 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 16:36:12.928521 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 16:36:12.929401 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 16:36:12.929401 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 16:36:12.929401 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 16:36:12.929401 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 16:36:12.929401 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 16:36:12.929401 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 16:36:12.929401 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 16:36:12.929401 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 16:36:12.929401 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 16:36:12.929401 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 16:36:12.929401 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 16:36:12.929401 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application with Lock( Aug 01 16:36:12.929401 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 16:36:12.929401 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self._enter() Aug 01 16:36:12.929401 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 16:36:12.929401 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 16:36:12.931119 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 16:36:12.931119 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self.creator() Aug 01 16:36:12.931119 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 16:36:12.931119 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application created_value = creator( Aug 01 16:36:12.931119 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 16:36:12.931119 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 16:36:12.931119 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 16:36:12.931119 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 16:36:12.931119 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 16:36:12.931119 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 16:36:12.931119 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 01 16:36:12.931119 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application Aug 01 16:36:12.931119 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 145/280] 10.222.0.20 () {56 vars in 1099 bytes} [Mon Aug 1 16:36:12 2022] GET /identity/v3/roles/reader => generated 84 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 16:36:12.937381 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b8dc767-9a39-438a-bdd1-9858d489b9ef None admin] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:36:12.973528 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG dbcounter [-] [87057] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=87057) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:36:13.029875 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b322cc39-dc6d-4c65-9132-3730c172df35 None admin] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:36:13.068346 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b8dc767-9a39-438a-bdd1-9858d489b9ef None admin] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:36:13.073597 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG dbcounter [-] [87058] Writing DB stats keystone:SELECT=100 {{(pid=87058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:36:13.083047 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f376d95c-279e-405f-9646-d565de481f54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:36:13.086485 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f376d95c-279e-405f-9646-d565de481f54 None admin] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:36:13.086485 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f376d95c-279e-405f-9646-d565de481f54 None admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:36:13.086485 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f376d95c-279e-405f-9646-d565de481f54 None admin] PATH_INFO: `/v3/users/950e03b49834417cac6a2df952ce522e` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:36:13.105439 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f376d95c-279e-405f-9646-d565de481f54 None admin] RBAC: Authorizing `identity:get_user(user_id=950e03b49834417cac6a2df952ce522e)` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:36:13.107993 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f376d95c-279e-405f-9646-d565de481f54 None admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:36:13.110573 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 136/281] 10.222.0.20 () {56 vars in 1177 bytes} [Mon Aug 1 16:36:12 2022] GET /identity/v3/users/950e03b49834417cac6a2df952ce522e => generated 293 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:36:13.116659 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f376d95c-279e-405f-9646-d565de481f54 None admin] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:36:13.119698 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82761f6f-0205-4a22-b04f-be9498e00c04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:36:13.120331 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82761f6f-0205-4a22-b04f-be9498e00c04 None admin] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:36:13.120615 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82761f6f-0205-4a22-b04f-be9498e00c04 None admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:36:13.120888 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82761f6f-0205-4a22-b04f-be9498e00c04 None admin] PATH_INFO: `/v3/roles` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:36:13.121654 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82761f6f-0205-4a22-b04f-be9498e00c04 None admin] RBAC: Adding query filter params (name=reader) {{(pid=87058) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:36:13.121962 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82761f6f-0205-4a22-b04f-be9498e00c04 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:36:13.123487 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82761f6f-0205-4a22-b04f-be9498e00c04 None admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:36:13.136778 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 146/282] 10.222.0.20 () {56 vars in 1101 bytes} [Mon Aug 1 16:36:12 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 16:36:13.143679 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82761f6f-0205-4a22-b04f-be9498e00c04 None admin] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:36:13.249169 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f376d95c-279e-405f-9646-d565de481f54 None admin] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:36:13.269603 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG dbcounter [-] [87057] Writing DB stats keystone:SELECT=100 {{(pid=87057) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:36:13.280538 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82761f6f-0205-4a22-b04f-be9498e00c04 None admin] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:36:13.298363 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7553d7b4-13f7-4479-ab90-894f6a81a228 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:36:13.298994 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7553d7b4-13f7-4479-ab90-894f6a81a228 None admin] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:36:13.299278 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7553d7b4-13f7-4479-ab90-894f6a81a228 None admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:36:13.299806 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7553d7b4-13f7-4479-ab90-894f6a81a228 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:36:13.300640 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7553d7b4-13f7-4479-ab90-894f6a81a228 None admin] RBAC: Adding query filter params (role.id=0999d3da63a545f18ff3252afdc90518, user.id=950e03b49834417cac6a2df952ce522e, scope.system=all) {{(pid=87057) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:36:13.301387 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7553d7b4-13f7-4479-ab90-894f6a81a228 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:36:13.303441 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7553d7b4-13f7-4479-ab90-894f6a81a228 None admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:36:13.313289 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 137/283] 10.222.0.20 () {56 vars in 1308 bytes} [Mon Aug 1 16:36:13 2022] GET /identity/v3/role_assignments?user.id=950e03b49834417cac6a2df952ce522e&scope.system=all&role.id=0999d3da63a545f18ff3252afdc90518 => generated 226 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 16:36:13.314879 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5976f8d7-31d0-436e-ab78-f5dc2dd3473c None None] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:36:13.315333 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5976f8d7-31d0-436e-ab78-f5dc2dd3473c None None] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:36:13.315774 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5976f8d7-31d0-436e-ab78-f5dc2dd3473c None None] PATH_INFO: `/` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:36:13.317622 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 138/284] 10.222.0.20 () {54 vars in 926 bytes} [Mon Aug 1 16:36:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 16:36:13.330701 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d32986-e89b-45e7-851e-5ae0f8da9984 None None] REQUEST_METHOD: `POST` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:36:13.331272 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d32986-e89b-45e7-851e-5ae0f8da9984 None None] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:36:13.331752 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d32986-e89b-45e7-851e-5ae0f8da9984 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:36:13.340755 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e51aa263-2097-4855-972a-bcd453c6fdd3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:36:13.341706 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51aa263-2097-4855-972a-bcd453c6fdd3 None admin] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:36:13.341992 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51aa263-2097-4855-972a-bcd453c6fdd3 None admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:36:13.342295 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51aa263-2097-4855-972a-bcd453c6fdd3 None admin] PATH_INFO: `/v3/users/e4ba5c4a5a5a456ea72804daff75daa1` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:36:13.345876 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e51aa263-2097-4855-972a-bcd453c6fdd3 None admin] RBAC: Authorizing `identity:get_user(user_id=e4ba5c4a5a5a456ea72804daff75daa1)` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:36:13.348164 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e51aa263-2097-4855-972a-bcd453c6fdd3 None admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:36:13.350581 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 147/285] 10.222.0.20 () {56 vars in 1177 bytes} [Mon Aug 1 16:36:13 2022] GET /identity/v3/users/e4ba5c4a5a5a456ea72804daff75daa1 => generated 297 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 16:36:13.358949 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.auth.core [None req-d1d32986-e89b-45e7-851e-5ae0f8da9984 None None] MFA Rules not processed for user `1007f8e379b74e169b98fb7c3c0ba5b4`. Rule list: `[]` (Enabled: `True`). {{(pid=87057) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:36:13.359135 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e51aa263-2097-4855-972a-bcd453c6fdd3 None admin] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:36:13.360778 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.fernet_utils [None req-d1d32986-e89b-45e7-851e-5ae0f8da9984 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87057) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:36:13.421487 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG dbcounter [-] [87058] Writing DB stats keystone:SELECT=100 {{(pid=87058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:36:13.436655 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 139/286] 10.222.0.20 () {58 vars in 1023 bytes} [Mon Aug 1 16:36:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 16:36:13.474470 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1d32986-e89b-45e7-851e-5ae0f8da9984 None None] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:36:13.558093 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e51aa263-2097-4855-972a-bcd453c6fdd3 None admin] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:36:13.604876 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1d32986-e89b-45e7-851e-5ae0f8da9984 None None] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:36:13.606892 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9398872f-2536-43f2-971a-e9b1bc5c229b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:36:13.608624 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9398872f-2536-43f2-971a-e9b1bc5c229b None admin] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:36:13.609108 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9398872f-2536-43f2-971a-e9b1bc5c229b None admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:36:13.609591 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9398872f-2536-43f2-971a-e9b1bc5c229b None admin] PATH_INFO: `/v3/projects/d9a43129dd714d05bd409ffb0875f7e6` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:36:13.611014 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9398872f-2536-43f2-971a-e9b1bc5c229b None admin] RBAC: Authorizing `identity:get_project(project_id=d9a43129dd714d05bd409ffb0875f7e6)` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:36:13.612754 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9398872f-2536-43f2-971a-e9b1bc5c229b None admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:36:13.615034 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 148/287] 10.222.0.20 () {56 vars in 1186 bytes} [Mon Aug 1 16:36:13 2022] GET /identity/v3/projects/d9a43129dd714d05bd409ffb0875f7e6 => generated 302 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:36:13.621493 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9398872f-2536-43f2-971a-e9b1bc5c229b None admin] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:36:13.623230 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG dbcounter [-] [87057] Writing DB stats keystone:SELECT=100 {{(pid=87057) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:36:13.669541 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec87dee9-efc8-438d-b344-52a4d0842acb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:36:13.670210 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec87dee9-efc8-438d-b344-52a4d0842acb None admin] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:36:13.670493 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec87dee9-efc8-438d-b344-52a4d0842acb None admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:36:13.670767 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec87dee9-efc8-438d-b344-52a4d0842acb None admin] PATH_INFO: `/v3/roles/member` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:36:13.688802 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec87dee9-efc8-438d-b344-52a4d0842acb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 16:36:13.698967 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application [None req-ec87dee9-efc8-438d-b344-52a4d0842acb None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 01 16:36:13.698967 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 16:36:13.698967 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 16:36:13.698967 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 16:36:13.698967 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 16:36:13.698967 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 16:36:13.698967 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 16:36:13.698967 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 16:36:13.698967 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 16:36:13.698967 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 16:36:13.698967 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 16:36:13.698967 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 16:36:13.698967 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 16:36:13.698967 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 16:36:13.698967 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 16:36:13.698967 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 16:36:13.699890 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 16:36:13.699890 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 16:36:13.699890 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 16:36:13.699890 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 16:36:13.699890 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 16:36:13.699890 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 16:36:13.699890 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 16:36:13.699890 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 16:36:13.699890 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 16:36:13.699890 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 16:36:13.699890 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 16:36:13.699890 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application with Lock( Aug 01 16:36:13.699890 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 16:36:13.699890 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self._enter() Aug 01 16:36:13.699890 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 16:36:13.699890 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 16:36:13.700679 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 16:36:13.700679 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self.creator() Aug 01 16:36:13.700679 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 16:36:13.700679 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application created_value = creator( Aug 01 16:36:13.700679 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 16:36:13.700679 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 16:36:13.700679 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 16:36:13.700679 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 16:36:13.700679 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 16:36:13.700679 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 16:36:13.700679 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 01 16:36:13.700679 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application Aug 01 16:36:13.704351 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 140/288] 10.222.0.20 () {56 vars in 1099 bytes} [Mon Aug 1 16:36:13 2022] GET /identity/v3/roles/member => generated 84 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 01 16:36:13.710520 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec87dee9-efc8-438d-b344-52a4d0842acb None admin] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:36:13.800259 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9398872f-2536-43f2-971a-e9b1bc5c229b None admin] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:36:13.812516 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG dbcounter [-] [87058] Writing DB stats keystone:SELECT=100 {{(pid=87058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:36:13.856092 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec87dee9-efc8-438d-b344-52a4d0842acb None admin] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:36:13.893513 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a240e3cd-fcc4-4edb-a04a-7dba1740805b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:36:13.895130 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a240e3cd-fcc4-4edb-a04a-7dba1740805b None admin] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:36:13.895609 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a240e3cd-fcc4-4edb-a04a-7dba1740805b None admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:36:13.896078 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a240e3cd-fcc4-4edb-a04a-7dba1740805b None admin] PATH_INFO: `/v3/roles` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:36:13.897086 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a240e3cd-fcc4-4edb-a04a-7dba1740805b None admin] RBAC: Adding query filter params (name=member) {{(pid=87057) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:36:13.897584 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a240e3cd-fcc4-4edb-a04a-7dba1740805b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:36:13.900027 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a240e3cd-fcc4-4edb-a04a-7dba1740805b None admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:36:13.909564 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 141/289] 10.222.0.20 () {56 vars in 1101 bytes} [Mon Aug 1 16:36:13 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:36:13.914464 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec3b5bba-623b-483e-ba49-a694c72516cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:36:13.914464 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3b5bba-623b-483e-ba49-a694c72516cd None admin] REQUEST_METHOD: `PUT` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:36:13.914464 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3b5bba-623b-483e-ba49-a694c72516cd None admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:36:13.914464 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3b5bba-623b-483e-ba49-a694c72516cd None admin] PATH_INFO: `/v3/projects/d9a43129dd714d05bd409ffb0875f7e6/users/e4ba5c4a5a5a456ea72804daff75daa1/roles/02c40c90308547dd8ac5aaad0da8d68c` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:36:13.917674 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a240e3cd-fcc4-4edb-a04a-7dba1740805b None admin] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:36:13.917864 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec3b5bba-623b-483e-ba49-a694c72516cd None admin] RBAC: Authorizing `identity:create_grant(project_id=d9a43129dd714d05bd409ffb0875f7e6, user_id=e4ba5c4a5a5a456ea72804daff75daa1, role_id=02c40c90308547dd8ac5aaad0da8d68c)` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:36:13.919529 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec3b5bba-623b-483e-ba49-a694c72516cd None admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:36:13.970400 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG dbcounter [-] [87057] Writing DB stats keystone:SELECT=100 {{(pid=87057) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:36:14.028657 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a240e3cd-fcc4-4edb-a04a-7dba1740805b None admin] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:36:14.037936 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 149/290] 10.222.0.20 () {58 vars in 1439 bytes} [Mon Aug 1 16:36:13 2022] PUT /identity/v3/projects/d9a43129dd714d05bd409ffb0875f7e6/users/e4ba5c4a5a5a456ea72804daff75daa1/roles/02c40c90308547dd8ac5aaad0da8d68c => generated 0 bytes in 417 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 16:36:14.083269 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b39785d-c870-4e8a-b8a3-c73671e4d71d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:36:14.084303 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b39785d-c870-4e8a-b8a3-c73671e4d71d None admin] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:36:14.084812 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b39785d-c870-4e8a-b8a3-c73671e4d71d None admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:36:14.085282 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b39785d-c870-4e8a-b8a3-c73671e4d71d None admin] PATH_INFO: `/v3/users/ac697afa751948bf87a605cf4aab7c65` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:36:14.087474 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b39785d-c870-4e8a-b8a3-c73671e4d71d None admin] RBAC: Authorizing `identity:get_user(user_id=ac697afa751948bf87a605cf4aab7c65)` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:36:14.089177 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b39785d-c870-4e8a-b8a3-c73671e4d71d None admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:36:14.095617 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 142/291] 10.222.0.20 () {56 vars in 1177 bytes} [Mon Aug 1 16:36:13 2022] GET /identity/v3/users/ac697afa751948bf87a605cf4aab7c65 => generated 297 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 16:36:14.102431 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec3b5bba-623b-483e-ba49-a694c72516cd None admin] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:36:14.236403 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec3b5bba-623b-483e-ba49-a694c72516cd None admin] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:36:14.275051 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-777a652f-09c3-413c-928e-1406179022a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:36:14.275698 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777a652f-09c3-413c-928e-1406179022a7 None admin] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:36:14.275968 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777a652f-09c3-413c-928e-1406179022a7 None admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:36:14.276241 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777a652f-09c3-413c-928e-1406179022a7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:36:14.308917 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG dbcounter [-] [87058] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:36:14.349689 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-777a652f-09c3-413c-928e-1406179022a7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:36:14.351221 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-777a652f-09c3-413c-928e-1406179022a7 None admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:36:14.456815 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 150/292] 10.222.0.20 () {58 vars in 1282 bytes} [Mon Aug 1 16:36:14 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 356 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 01 16:36:14.463122 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b39785d-c870-4e8a-b8a3-c73671e4d71d None admin] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:36:14.574845 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG dbcounter [-] [87057] Writing DB stats keystone:SELECT=100 {{(pid=87057) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:36:14.581621 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b39785d-c870-4e8a-b8a3-c73671e4d71d None admin] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:36:14.622532 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28528e83-c87e-4ca5-8dc5-1409db0955f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:36:14.623179 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28528e83-c87e-4ca5-8dc5-1409db0955f8 None admin] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:36:14.623458 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28528e83-c87e-4ca5-8dc5-1409db0955f8 None admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:36:14.623731 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28528e83-c87e-4ca5-8dc5-1409db0955f8 None admin] PATH_INFO: `/v3/projects/d9a43129dd714d05bd409ffb0875f7e6` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:36:14.624900 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28528e83-c87e-4ca5-8dc5-1409db0955f8 None admin] RBAC: Authorizing `identity:get_project(project_id=d9a43129dd714d05bd409ffb0875f7e6)` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:36:14.626367 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28528e83-c87e-4ca5-8dc5-1409db0955f8 None admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:36:14.628041 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 143/293] 10.222.0.20 () {56 vars in 1186 bytes} [Mon Aug 1 16:36:14 2022] GET /identity/v3/projects/d9a43129dd714d05bd409ffb0875f7e6 => generated 302 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:36:14.633907 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-777a652f-09c3-413c-928e-1406179022a7 None admin] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:36:14.710517 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG dbcounter [-] [87058] Writing DB stats keystone:SELECT=100 {{(pid=87058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:36:14.750765 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-777a652f-09c3-413c-928e-1406179022a7 None admin] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:36:14.792686 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f425403-51ed-415f-bd08-e016d3b5fe11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:36:14.793309 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f425403-51ed-415f-bd08-e016d3b5fe11 None admin] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:36:14.793594 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f425403-51ed-415f-bd08-e016d3b5fe11 None admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:36:14.793869 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f425403-51ed-415f-bd08-e016d3b5fe11 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:36:14.798745 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f425403-51ed-415f-bd08-e016d3b5fe11 None admin] RBAC: Adding query filter params (role.id=0999d3da63a545f18ff3252afdc90518, scope.project.id=d9a43129dd714d05bd409ffb0875f7e6, user.id=ac697afa751948bf87a605cf4aab7c65) {{(pid=87058) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:36:14.799202 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f425403-51ed-415f-bd08-e016d3b5fe11 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:36:14.800650 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f425403-51ed-415f-bd08-e016d3b5fe11 None admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:36:14.806909 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 151/294] 10.222.0.20 () {56 vars in 1374 bytes} [Mon Aug 1 16:36:14 2022] GET /identity/v3/role_assignments?user.id=ac697afa751948bf87a605cf4aab7c65&scope.project.id=d9a43129dd714d05bd409ffb0875f7e6&role.id=0999d3da63a545f18ff3252afdc90518 => generated 608 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:36:15.201942 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b81b86-c266-436b-b4ff-34c8ef45e6c0 None None] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:36:15.202448 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b81b86-c266-436b-b4ff-34c8ef45e6c0 None None] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:36:15.202448 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b81b86-c266-436b-b4ff-34c8ef45e6c0 None None] PATH_INFO: `/` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:36:15.204063 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 144/295] 10.222.0.20 () {54 vars in 926 bytes} [Mon Aug 1 16:36:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 16:36:15.208910 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37dc008-f2dd-4d63-bf05-5f44dc8b124f None None] REQUEST_METHOD: `POST` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:36:15.209331 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37dc008-f2dd-4d63-bf05-5f44dc8b124f None None] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:36:15.209461 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37dc008-f2dd-4d63-bf05-5f44dc8b124f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:36:15.230608 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.auth.core [None req-f37dc008-f2dd-4d63-bf05-5f44dc8b124f None None] MFA Rules not processed for user `1007f8e379b74e169b98fb7c3c0ba5b4`. Rule list: `[]` (Enabled: `True`). {{(pid=87058) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:36:15.231799 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.fernet_utils [None req-f37dc008-f2dd-4d63-bf05-5f44dc8b124f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:36:15.310169 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 152/296] 10.222.0.20 () {58 vars in 1023 bytes} [Mon Aug 1 16:36:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 16:36:15.333379 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b81b86-c266-436b-b4ff-34c8ef45e6c0 None None] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:36:15.455031 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b81b86-c266-436b-b4ff-34c8ef45e6c0 None None] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:36:15.505306 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d3733d-a496-4b7f-9b1a-489c2b26c555 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:36:15.505946 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d3733d-a496-4b7f-9b1a-489c2b26c555 None admin] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:36:15.506210 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70189f4f-9a54-44af-ae3d-25dfdd2bfe6d None None] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:36:15.506313 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d3733d-a496-4b7f-9b1a-489c2b26c555 None admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:36:15.506495 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70189f4f-9a54-44af-ae3d-25dfdd2bfe6d None None] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:36:15.506591 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d3733d-a496-4b7f-9b1a-489c2b26c555 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:36:15.506764 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70189f4f-9a54-44af-ae3d-25dfdd2bfe6d None None] PATH_INFO: `/` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:36:15.509087 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 153/297] 10.222.0.20 () {54 vars in 926 bytes} [Mon Aug 1 16:36:15 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 16:36:15.513861 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4c79f6-81f0-4de8-b5d2-44977dbcabb8 None None] REQUEST_METHOD: `POST` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:36:15.514299 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4c79f6-81f0-4de8-b5d2-44977dbcabb8 None None] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:36:15.514661 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4c79f6-81f0-4de8-b5d2-44977dbcabb8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:36:15.520811 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7d3733d-a496-4b7f-9b1a-489c2b26c555 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 16:36:15.528494 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application [None req-c7d3733d-a496-4b7f-9b1a-489c2b26c555 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 01 16:36:15.528494 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 16:36:15.528494 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 16:36:15.528494 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 16:36:15.528494 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 16:36:15.528494 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 16:36:15.528494 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 16:36:15.528494 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 16:36:15.528494 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 16:36:15.528494 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 16:36:15.528494 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 16:36:15.528494 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 16:36:15.528494 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 16:36:15.528494 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 16:36:15.528494 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 16:36:15.528494 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 16:36:15.529607 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 16:36:15.529607 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 16:36:15.529607 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 16:36:15.529607 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 16:36:15.529607 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 16:36:15.529607 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 16:36:15.529607 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 16:36:15.529607 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 16:36:15.529607 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 16:36:15.529607 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 16:36:15.529607 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 16:36:15.529607 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application with Lock( Aug 01 16:36:15.529607 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 16:36:15.529607 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self._enter() Aug 01 16:36:15.529607 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 16:36:15.529607 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 16:36:15.530651 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 16:36:15.530651 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self.creator() Aug 01 16:36:15.530651 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 16:36:15.530651 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application created_value = creator( Aug 01 16:36:15.530651 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 16:36:15.530651 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 16:36:15.530651 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 16:36:15.530651 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 16:36:15.530651 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 16:36:15.530651 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 16:36:15.530651 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 01 16:36:15.530651 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application Aug 01 16:36:15.532827 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 145/298] 10.222.0.20 () {56 vars in 1099 bytes} [Mon Aug 1 16:36:15 2022] GET /identity/v3/roles/reader => generated 84 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 16:36:15.538176 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.auth.core [None req-cc4c79f6-81f0-4de8-b5d2-44977dbcabb8 None None] MFA Rules not processed for user `1007f8e379b74e169b98fb7c3c0ba5b4`. Rule list: `[]` (Enabled: `True`). {{(pid=87058) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:36:15.538419 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d3733d-a496-4b7f-9b1a-489c2b26c555 None admin] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:36:15.540437 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.fernet_utils [None req-cc4c79f6-81f0-4de8-b5d2-44977dbcabb8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:36:15.570383 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG dbcounter [-] [87057] Writing DB stats keystone:SELECT=100 {{(pid=87057) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:36:15.610923 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG dbcounter [-] [87058] Writing DB stats keystone:SELECT=100 {{(pid=87058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:36:15.634008 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 154/299] 10.222.0.20 () {58 vars in 1023 bytes} [Mon Aug 1 16:36:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 01 16:36:15.655140 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d3733d-a496-4b7f-9b1a-489c2b26c555 None admin] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:36:15.657961 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc4c79f6-81f0-4de8-b5d2-44977dbcabb8 None None] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:36:15.701940 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7693427-2ed3-4ad4-a317-852d5f604d7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:36:15.702605 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7693427-2ed3-4ad4-a317-852d5f604d7a None admin] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:36:15.702882 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7693427-2ed3-4ad4-a317-852d5f604d7a None admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:36:15.703150 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7693427-2ed3-4ad4-a317-852d5f604d7a None admin] PATH_INFO: `/v3/roles` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:36:15.703909 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7693427-2ed3-4ad4-a317-852d5f604d7a None admin] RBAC: Adding query filter params (name=reader) {{(pid=87057) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:36:15.704215 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7693427-2ed3-4ad4-a317-852d5f604d7a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:36:15.705643 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7693427-2ed3-4ad4-a317-852d5f604d7a None admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:36:15.715248 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 146/300] 10.222.0.20 () {56 vars in 1101 bytes} [Mon Aug 1 16:36:15 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 16:36:15.716290 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363095d1-24b2-4e4b-b4ff-7e391775f4a5 None None] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:36:15.716575 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363095d1-24b2-4e4b-b4ff-7e391775f4a5 None None] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:36:15.716849 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363095d1-24b2-4e4b-b4ff-7e391775f4a5 None None] PATH_INFO: `/` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:36:15.718205 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 147/301] 10.222.0.20 () {54 vars in 926 bytes} [Mon Aug 1 16:36:15 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 16:36:15.722877 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-363095d1-24b2-4e4b-b4ff-7e391775f4a5 None None] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:36:15.781155 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc4c79f6-81f0-4de8-b5d2-44977dbcabb8 None None] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:36:15.814327 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG dbcounter [-] [87057] Writing DB stats keystone:SELECT=100 {{(pid=87057) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:36:15.817429 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04325cbd-8b39-49ab-befb-0a7a194481f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:36:15.818210 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04325cbd-8b39-49ab-befb-0a7a194481f1 None admin] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:36:15.818408 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04325cbd-8b39-49ab-befb-0a7a194481f1 None admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:36:15.818642 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04325cbd-8b39-49ab-befb-0a7a194481f1 None admin] PATH_INFO: `/v3/domains/default` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:36:15.819785 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04325cbd-8b39-49ab-befb-0a7a194481f1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:36:15.821794 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04325cbd-8b39-49ab-befb-0a7a194481f1 None admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:36:15.823508 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 155/302] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Aug 1 16:36:15 2022] GET /identity/v3/domains/default => generated 199 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:36:15.828276 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-363095d1-24b2-4e4b-b4ff-7e391775f4a5 None None] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:36:15.828954 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e515e94-cede-4c84-9fd0-763a6144c8f1 None None] REQUEST_METHOD: `POST` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:36:15.829241 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e515e94-cede-4c84-9fd0-763a6144c8f1 None None] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:36:15.829539 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e515e94-cede-4c84-9fd0-763a6144c8f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:36:15.847288 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.auth.core [None req-7e515e94-cede-4c84-9fd0-763a6144c8f1 None None] MFA Rules not processed for user `1007f8e379b74e169b98fb7c3c0ba5b4`. Rule list: `[]` (Enabled: `True`). {{(pid=87058) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:36:15.848497 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.fernet_utils [None req-7e515e94-cede-4c84-9fd0-763a6144c8f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:36:15.866510 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e20cb5ac-88b0-4801-bc35-895c44428da2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:36:15.867365 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20cb5ac-88b0-4801-bc35-895c44428da2 None admin] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:36:15.867653 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20cb5ac-88b0-4801-bc35-895c44428da2 None admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:36:15.867927 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20cb5ac-88b0-4801-bc35-895c44428da2 None admin] PATH_INFO: `/v3/users/c173554995b744a6ba3a400282d04418` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:36:15.882066 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e20cb5ac-88b0-4801-bc35-895c44428da2 None admin] RBAC: Authorizing `identity:get_user(user_id=c173554995b744a6ba3a400282d04418)` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:36:15.883821 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e20cb5ac-88b0-4801-bc35-895c44428da2 None admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:36:15.885635 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 148/303] 10.222.0.20 () {56 vars in 1177 bytes} [Mon Aug 1 16:36:15 2022] GET /identity/v3/users/c173554995b744a6ba3a400282d04418 => generated 293 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:36:15.886984 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e20cb5ac-88b0-4801-bc35-895c44428da2 None admin] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:36:15.908700 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG dbcounter [-] [87058] Writing DB stats keystone:SELECT=100 {{(pid=87058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:36:15.927854 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 156/304] 10.222.0.20 () {58 vars in 1023 bytes} [Mon Aug 1 16:36:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 16:36:15.928952 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e515e94-cede-4c84-9fd0-763a6144c8f1 None None] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:36:16.002150 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e20cb5ac-88b0-4801-bc35-895c44428da2 None admin] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:36:16.036892 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e515e94-cede-4c84-9fd0-763a6144c8f1 None None] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:36:16.037104 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96792367-dd0f-4974-806a-8c51ed340ef6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:36:16.037673 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96792367-dd0f-4974-806a-8c51ed340ef6 None admin] REQUEST_METHOD: `POST` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:36:16.037960 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96792367-dd0f-4974-806a-8c51ed340ef6 None admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:36:16.038280 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96792367-dd0f-4974-806a-8c51ed340ef6 None admin] PATH_INFO: `/v3/groups` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:36:16.038992 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96792367-dd0f-4974-806a-8c51ed340ef6 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:36:16.040443 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96792367-dd0f-4974-806a-8c51ed340ef6 None admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:36:16.072003 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b93e4589-0b2e-442a-b1eb-a8582b511299 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:36:16.072623 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93e4589-0b2e-442a-b1eb-a8582b511299 None admin] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:36:16.072904 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93e4589-0b2e-442a-b1eb-a8582b511299 None admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:36:16.073176 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93e4589-0b2e-442a-b1eb-a8582b511299 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:36:16.073970 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b93e4589-0b2e-442a-b1eb-a8582b511299 None admin] RBAC: Adding query filter params (role.id=02c40c90308547dd8ac5aaad0da8d68c, user.id=c173554995b744a6ba3a400282d04418, scope.system=all) {{(pid=87058) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:36:16.074380 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b93e4589-0b2e-442a-b1eb-a8582b511299 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:36:16.075811 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b93e4589-0b2e-442a-b1eb-a8582b511299 None admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:36:16.080783 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 157/305] 10.222.0.20 () {56 vars in 1308 bytes} [Mon Aug 1 16:36:15 2022] GET /identity/v3/role_assignments?user.id=c173554995b744a6ba3a400282d04418&scope.system=all&role.id=02c40c90308547dd8ac5aaad0da8d68c => generated 226 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:36:16.081566 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b93e4589-0b2e-442a-b1eb-a8582b511299 None admin] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:36:16.127029 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 149/306] 10.222.0.20 () {60 vars in 1134 bytes} [Mon Aug 1 16:36:15 2022] POST /identity/v3/groups => generated 229 bytes in 241 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 16:36:16.148947 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG dbcounter [-] [87058] Writing DB stats keystone:SELECT=100 {{(pid=87058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:36:16.214019 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b93e4589-0b2e-442a-b1eb-a8582b511299 None admin] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:36:16.256217 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc9dedc2-0c3c-48a5-8c99-37926cd8ba43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:36:16.257243 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9dedc2-0c3c-48a5-8c99-37926cd8ba43 None admin] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:36:16.258402 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9dedc2-0c3c-48a5-8c99-37926cd8ba43 None admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:36:16.258954 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9dedc2-0c3c-48a5-8c99-37926cd8ba43 None admin] PATH_INFO: `/v3/roles/member` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:36:16.272211 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc9dedc2-0c3c-48a5-8c99-37926cd8ba43 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 16:36:16.278523 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application [None req-cc9dedc2-0c3c-48a5-8c99-37926cd8ba43 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 01 16:36:16.278523 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 16:36:16.278523 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 16:36:16.278523 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 16:36:16.278523 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 16:36:16.278523 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 16:36:16.278523 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 16:36:16.278523 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 16:36:16.278523 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 16:36:16.278523 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 16:36:16.278523 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 16:36:16.278523 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 16:36:16.278523 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 16:36:16.278523 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 16:36:16.278523 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 16:36:16.278523 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 16:36:16.279588 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 16:36:16.279588 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 16:36:16.279588 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 16:36:16.279588 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 16:36:16.279588 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 16:36:16.279588 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 16:36:16.279588 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 16:36:16.279588 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 16:36:16.279588 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 16:36:16.279588 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 16:36:16.279588 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 16:36:16.279588 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application with Lock( Aug 01 16:36:16.279588 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 16:36:16.279588 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self._enter() Aug 01 16:36:16.279588 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 16:36:16.279588 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 16:36:16.280372 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 16:36:16.280372 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self.creator() Aug 01 16:36:16.280372 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 16:36:16.280372 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application created_value = creator( Aug 01 16:36:16.280372 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 16:36:16.280372 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 16:36:16.280372 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 16:36:16.280372 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 16:36:16.280372 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 16:36:16.280372 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 16:36:16.280372 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 01 16:36:16.280372 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application Aug 01 16:36:16.282943 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 158/307] 10.222.0.20 () {56 vars in 1099 bytes} [Mon Aug 1 16:36:16 2022] GET /identity/v3/roles/member => generated 84 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 16:36:16.293647 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96792367-dd0f-4974-806a-8c51ed340ef6 None admin] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:36:16.305058 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG dbcounter [-] [87057] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87057) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:36:16.405002 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96792367-dd0f-4974-806a-8c51ed340ef6 None admin] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:36:16.524571 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0080020f-dff2-4f40-a04c-c3ea7b583950 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:36:16.525210 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0080020f-dff2-4f40-a04c-c3ea7b583950 None admin] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:36:16.525494 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0080020f-dff2-4f40-a04c-c3ea7b583950 None admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:36:16.525767 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0080020f-dff2-4f40-a04c-c3ea7b583950 None admin] PATH_INFO: `/v3/roles` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:36:16.534054 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0080020f-dff2-4f40-a04c-c3ea7b583950 None admin] RBAC: Adding query filter params (name=member) {{(pid=87057) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:36:16.534054 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0080020f-dff2-4f40-a04c-c3ea7b583950 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:36:16.534054 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0080020f-dff2-4f40-a04c-c3ea7b583950 None admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:36:16.536917 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 150/308] 10.222.0.20 () {56 vars in 1101 bytes} [Mon Aug 1 16:36:16 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:36:16.542755 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc9dedc2-0c3c-48a5-8c99-37926cd8ba43 None admin] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:36:16.628326 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1838c295-acfc-4697-9117-4679034b457b None None] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:36:16.628646 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1838c295-acfc-4697-9117-4679034b457b None None] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:36:16.628881 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1838c295-acfc-4697-9117-4679034b457b None None] PATH_INFO: `/` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:36:16.630558 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 151/309] 10.222.0.20 () {54 vars in 926 bytes} [Mon Aug 1 16:36:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 01 16:36:16.636848 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba646eef-458d-4a18-8316-9a89b0eb1fa0 None None] REQUEST_METHOD: `POST` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:36:16.637376 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba646eef-458d-4a18-8316-9a89b0eb1fa0 None None] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:36:16.637869 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba646eef-458d-4a18-8316-9a89b0eb1fa0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:36:16.656427 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.auth.core [None req-ba646eef-458d-4a18-8316-9a89b0eb1fa0 None None] MFA Rules not processed for user `1007f8e379b74e169b98fb7c3c0ba5b4`. Rule list: `[]` (Enabled: `True`). {{(pid=87057) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:36:16.657692 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.fernet_utils [None req-ba646eef-458d-4a18-8316-9a89b0eb1fa0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87057) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:36:16.660779 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc9dedc2-0c3c-48a5-8c99-37926cd8ba43 None admin] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:36:16.671849 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG dbcounter [-] [87058] Writing DB stats keystone:SELECT=100 {{(pid=87058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:36:16.701335 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d262f7ae-aa17-4f9c-9c3f-b478ad2de8dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:36:16.701671 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d262f7ae-aa17-4f9c-9c3f-b478ad2de8dd None admin] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:36:16.706055 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d262f7ae-aa17-4f9c-9c3f-b478ad2de8dd None admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:36:16.706055 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d262f7ae-aa17-4f9c-9c3f-b478ad2de8dd None admin] PATH_INFO: `/v3/users/950e03b49834417cac6a2df952ce522e` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:36:16.706055 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d262f7ae-aa17-4f9c-9c3f-b478ad2de8dd None admin] RBAC: Authorizing `identity:get_user(user_id=950e03b49834417cac6a2df952ce522e)` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:36:16.706055 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d262f7ae-aa17-4f9c-9c3f-b478ad2de8dd None admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:36:16.706898 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 159/310] 10.222.0.20 () {56 vars in 1177 bytes} [Mon Aug 1 16:36:16 2022] GET /identity/v3/users/950e03b49834417cac6a2df952ce522e => generated 293 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:36:16.712588 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d262f7ae-aa17-4f9c-9c3f-b478ad2de8dd None admin] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:36:16.733323 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG dbcounter [-] [87057] Writing DB stats keystone:SELECT=100 {{(pid=87057) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:36:16.736388 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 152/311] 10.222.0.20 () {58 vars in 1023 bytes} [Mon Aug 1 16:36:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 01 16:36:16.762765 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba646eef-458d-4a18-8316-9a89b0eb1fa0 None None] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:36:16.818726 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d262f7ae-aa17-4f9c-9c3f-b478ad2de8dd None admin] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:36:16.864386 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44205e14-0781-4c42-ab8b-772ef6fd636a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:36:16.865444 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44205e14-0781-4c42-ab8b-772ef6fd636a None admin] REQUEST_METHOD: `PUT` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:36:16.865940 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44205e14-0781-4c42-ab8b-772ef6fd636a None admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:36:16.866442 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44205e14-0781-4c42-ab8b-772ef6fd636a None admin] PATH_INFO: `/v3/system/users/950e03b49834417cac6a2df952ce522e/roles/0999d3da63a545f18ff3252afdc90518` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:36:16.868236 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44205e14-0781-4c42-ab8b-772ef6fd636a None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=950e03b49834417cac6a2df952ce522e, role_id=0999d3da63a545f18ff3252afdc90518)` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:36:16.869889 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44205e14-0781-4c42-ab8b-772ef6fd636a None admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:36:16.875199 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba646eef-458d-4a18-8316-9a89b0eb1fa0 None None] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:36:16.929103 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb1c640c-ad5c-4749-8291-1eb363cacfbd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:36:16.930056 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1c640c-ad5c-4749-8291-1eb363cacfbd None admin] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:36:16.930515 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1c640c-ad5c-4749-8291-1eb363cacfbd None admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:36:16.930940 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1c640c-ad5c-4749-8291-1eb363cacfbd None admin] PATH_INFO: `/v3/roles/reader` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:36:16.942250 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb1c640c-ad5c-4749-8291-1eb363cacfbd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 16:36:16.948039 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application [None req-cb1c640c-ad5c-4749-8291-1eb363cacfbd None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 01 16:36:16.948039 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 16:36:16.948039 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 16:36:16.948039 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 16:36:16.948039 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 16:36:16.948039 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 16:36:16.948039 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 16:36:16.948039 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 16:36:16.948039 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 16:36:16.948039 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 16:36:16.948039 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 16:36:16.948039 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 16:36:16.948039 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 16:36:16.948039 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 16:36:16.948039 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 16:36:16.948039 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 16:36:16.949110 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 16:36:16.949110 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 16:36:16.949110 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 16:36:16.949110 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 16:36:16.949110 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 16:36:16.949110 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 16:36:16.949110 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 16:36:16.949110 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 16:36:16.949110 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 16:36:16.949110 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 16:36:16.949110 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 16:36:16.949110 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application with Lock( Aug 01 16:36:16.949110 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 16:36:16.949110 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self._enter() Aug 01 16:36:16.949110 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 16:36:16.949110 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 16:36:16.950099 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 16:36:16.950099 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self.creator() Aug 01 16:36:16.950099 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 16:36:16.950099 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application created_value = creator( Aug 01 16:36:16.950099 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 16:36:16.950099 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 16:36:16.950099 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 16:36:16.950099 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 16:36:16.950099 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 16:36:16.950099 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 16:36:16.950099 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 01 16:36:16.950099 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application Aug 01 16:36:16.952585 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 153/312] 10.222.0.20 () {56 vars in 1099 bytes} [Mon Aug 1 16:36:16 2022] GET /identity/v3/roles/reader => generated 84 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 01 16:36:16.957404 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb1c640c-ad5c-4749-8291-1eb363cacfbd None admin] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:36:16.971731 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 160/313] 10.222.0.20 () {58 vars in 1334 bytes} [Mon Aug 1 16:36:16 2022] PUT /identity/v3/system/users/950e03b49834417cac6a2df952ce522e/roles/0999d3da63a545f18ff3252afdc90518 => generated 0 bytes in 259 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 16:36:17.033521 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG dbcounter [-] [87057] Writing DB stats keystone:SELECT=100 {{(pid=87057) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:36:17.081036 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb1c640c-ad5c-4749-8291-1eb363cacfbd None admin] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:36:17.119898 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd97d974-101f-42f0-9e36-ec7ba2807534 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:36:17.119898 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd97d974-101f-42f0-9e36-ec7ba2807534 None admin] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:36:17.120196 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd97d974-101f-42f0-9e36-ec7ba2807534 None admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:36:17.120436 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd97d974-101f-42f0-9e36-ec7ba2807534 None admin] PATH_INFO: `/v3/roles` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:36:17.121236 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd97d974-101f-42f0-9e36-ec7ba2807534 None admin] RBAC: Adding query filter params (name=reader) {{(pid=87057) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:36:17.122405 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd97d974-101f-42f0-9e36-ec7ba2807534 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:36:17.122960 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd97d974-101f-42f0-9e36-ec7ba2807534 None admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:36:17.131990 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 154/314] 10.222.0.20 () {56 vars in 1101 bytes} [Mon Aug 1 16:36:16 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:36:17.137471 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44205e14-0781-4c42-ab8b-772ef6fd636a None admin] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:36:17.184792 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG dbcounter [-] [87058] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:36:17.248349 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44205e14-0781-4c42-ab8b-772ef6fd636a None admin] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:36:17.286301 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f86af029-1e1b-42b0-8d9d-bafd7a886e0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:36:17.287302 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86af029-1e1b-42b0-8d9d-bafd7a886e0d None admin] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:36:17.287787 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86af029-1e1b-42b0-8d9d-bafd7a886e0d None admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:36:17.288262 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86af029-1e1b-42b0-8d9d-bafd7a886e0d None admin] PATH_INFO: `/v3/users/e4ba5c4a5a5a456ea72804daff75daa1` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:36:17.289810 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f86af029-1e1b-42b0-8d9d-bafd7a886e0d None admin] RBAC: Authorizing `identity:get_user(user_id=e4ba5c4a5a5a456ea72804daff75daa1)` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:36:17.291581 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f86af029-1e1b-42b0-8d9d-bafd7a886e0d None admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:36:17.293757 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 161/315] 10.222.0.20 () {56 vars in 1177 bytes} [Mon Aug 1 16:36:17 2022] GET /identity/v3/users/e4ba5c4a5a5a456ea72804daff75daa1 => generated 297 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:36:17.299048 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd97d974-101f-42f0-9e36-ec7ba2807534 None admin] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:36:17.413870 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd97d974-101f-42f0-9e36-ec7ba2807534 None admin] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:36:17.436210 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG dbcounter [-] [87057] Writing DB stats keystone:SELECT=100 {{(pid=87057) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:36:17.452045 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7adf5122-94ac-4b99-a066-7dd66de7958d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:36:17.452958 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7adf5122-94ac-4b99-a066-7dd66de7958d None admin] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:36:17.453500 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7adf5122-94ac-4b99-a066-7dd66de7958d None admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:36:17.453994 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7adf5122-94ac-4b99-a066-7dd66de7958d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:36:17.529430 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7adf5122-94ac-4b99-a066-7dd66de7958d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:36:17.530941 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7adf5122-94ac-4b99-a066-7dd66de7958d None admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:36:17.636745 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 155/316] 10.222.0.20 () {58 vars in 1282 bytes} [Mon Aug 1 16:36:17 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 338 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 01 16:36:17.643377 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f86af029-1e1b-42b0-8d9d-bafd7a886e0d None admin] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:36:17.747803 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f86af029-1e1b-42b0-8d9d-bafd7a886e0d None admin] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:36:17.756969 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG dbcounter [-] [87058] Writing DB stats keystone:SELECT=100 {{(pid=87058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:36:17.782290 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d220a60-b56a-406d-b806-adb86ca93900 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:36:17.782945 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d220a60-b56a-406d-b806-adb86ca93900 None admin] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:36:17.783230 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d220a60-b56a-406d-b806-adb86ca93900 None admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:36:17.783501 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d220a60-b56a-406d-b806-adb86ca93900 None admin] PATH_INFO: `/v3/projects/d9a43129dd714d05bd409ffb0875f7e6` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:36:17.784675 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d220a60-b56a-406d-b806-adb86ca93900 None admin] RBAC: Authorizing `identity:get_project(project_id=d9a43129dd714d05bd409ffb0875f7e6)` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:36:17.786162 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d220a60-b56a-406d-b806-adb86ca93900 None admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:36:17.788143 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 162/317] 10.222.0.20 () {56 vars in 1186 bytes} [Mon Aug 1 16:36:17 2022] GET /identity/v3/projects/d9a43129dd714d05bd409ffb0875f7e6 => generated 302 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:36:17.794597 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7adf5122-94ac-4b99-a066-7dd66de7958d None admin] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:36:17.836821 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG dbcounter [-] [87057] Writing DB stats keystone:SELECT=100 {{(pid=87057) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:36:17.899292 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7adf5122-94ac-4b99-a066-7dd66de7958d None admin] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:36:17.936657 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c948f96-1e19-494a-8234-680cd9de4043 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:36:17.937306 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c948f96-1e19-494a-8234-680cd9de4043 None admin] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:36:17.937725 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c948f96-1e19-494a-8234-680cd9de4043 None admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:36:17.938003 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c948f96-1e19-494a-8234-680cd9de4043 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:36:17.938831 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c948f96-1e19-494a-8234-680cd9de4043 None admin] RBAC: Adding query filter params (role.id=02c40c90308547dd8ac5aaad0da8d68c, scope.project.id=d9a43129dd714d05bd409ffb0875f7e6, user.id=e4ba5c4a5a5a456ea72804daff75daa1) {{(pid=87057) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:36:17.939140 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c948f96-1e19-494a-8234-680cd9de4043 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:36:17.940597 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c948f96-1e19-494a-8234-680cd9de4043 None admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:36:17.947953 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 156/318] 10.222.0.20 () {56 vars in 1374 bytes} [Mon Aug 1 16:36:17 2022] GET /identity/v3/role_assignments?user.id=e4ba5c4a5a5a456ea72804daff75daa1&scope.project.id=d9a43129dd714d05bd409ffb0875f7e6&role.id=02c40c90308547dd8ac5aaad0da8d68c => generated 608 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:36:18.130306 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b35c16c-890e-48e2-8e17-f04f6e0bd7db None None] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:36:18.130705 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b35c16c-890e-48e2-8e17-f04f6e0bd7db None None] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:36:18.130915 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b35c16c-890e-48e2-8e17-f04f6e0bd7db None None] PATH_INFO: `/` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:36:18.132532 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 163/319] 10.222.0.20 () {54 vars in 926 bytes} [Mon Aug 1 16:36:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 16:36:18.138489 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ceab103-953e-4fd9-a7e6-1269e10d8790 None None] REQUEST_METHOD: `POST` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:36:18.138778 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ceab103-953e-4fd9-a7e6-1269e10d8790 None None] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:36:18.139057 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ceab103-953e-4fd9-a7e6-1269e10d8790 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:36:18.158070 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.auth.core [None req-6ceab103-953e-4fd9-a7e6-1269e10d8790 None None] MFA Rules not processed for user `1007f8e379b74e169b98fb7c3c0ba5b4`. Rule list: `[]` (Enabled: `True`). {{(pid=87057) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:36:18.159296 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.fernet_utils [None req-6ceab103-953e-4fd9-a7e6-1269e10d8790 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87057) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:36:18.232160 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 157/320] 10.222.0.20 () {58 vars in 1023 bytes} [Mon Aug 1 16:36:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 01 16:36:18.254216 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b35c16c-890e-48e2-8e17-f04f6e0bd7db None None] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:36:18.300455 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0268af29-a5c8-404b-92d4-85e47194d885 None None] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:36:18.300982 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0268af29-a5c8-404b-92d4-85e47194d885 None None] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:36:18.301213 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0268af29-a5c8-404b-92d4-85e47194d885 None None] PATH_INFO: `/` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:36:18.302908 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 158/321] 10.222.0.20 () {54 vars in 926 bytes} [Mon Aug 1 16:36:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 16:36:18.308169 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5f37a7-6df8-4d99-9fad-4f2e238c0be1 None None] REQUEST_METHOD: `POST` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:36:18.308448 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5f37a7-6df8-4d99-9fad-4f2e238c0be1 None None] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:36:18.308714 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5f37a7-6df8-4d99-9fad-4f2e238c0be1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:36:18.327431 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.auth.core [None req-8d5f37a7-6df8-4d99-9fad-4f2e238c0be1 None None] MFA Rules not processed for user `1007f8e379b74e169b98fb7c3c0ba5b4`. Rule list: `[]` (Enabled: `True`). {{(pid=87057) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:36:18.328603 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.fernet_utils [None req-8d5f37a7-6df8-4d99-9fad-4f2e238c0be1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87057) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:36:18.338205 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG dbcounter [-] [87057] Writing DB stats keystone:SELECT=100 {{(pid=87057) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:36:18.360796 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b35c16c-890e-48e2-8e17-f04f6e0bd7db None None] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:36:18.396947 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 159/322] 10.222.0.20 () {58 vars in 1023 bytes} [Mon Aug 1 16:36:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 01 16:36:18.398900 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a77dd65-e5b6-4ff3-a251-7543504fe335 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:36:18.399621 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a77dd65-e5b6-4ff3-a251-7543504fe335 None admin] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:36:18.399970 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a77dd65-e5b6-4ff3-a251-7543504fe335 None admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:36:18.400305 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a77dd65-e5b6-4ff3-a251-7543504fe335 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:36:18.410810 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a77dd65-e5b6-4ff3-a251-7543504fe335 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 16:36:18.416915 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application [None req-2a77dd65-e5b6-4ff3-a251-7543504fe335 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 01 16:36:18.416915 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 16:36:18.416915 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 16:36:18.416915 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 16:36:18.416915 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 16:36:18.416915 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 16:36:18.416915 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 16:36:18.416915 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 16:36:18.416915 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 16:36:18.416915 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 16:36:18.416915 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 16:36:18.416915 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 16:36:18.416915 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 16:36:18.416915 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 16:36:18.416915 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 16:36:18.416915 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 16:36:18.417807 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 16:36:18.417807 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 16:36:18.417807 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 16:36:18.417807 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 16:36:18.417807 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 16:36:18.417807 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 16:36:18.417807 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 16:36:18.417807 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 16:36:18.417807 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 16:36:18.417807 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 16:36:18.417807 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 16:36:18.417807 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application with Lock( Aug 01 16:36:18.417807 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 16:36:18.417807 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self._enter() Aug 01 16:36:18.417807 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 16:36:18.417807 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 16:36:18.418680 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 16:36:18.418680 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self.creator() Aug 01 16:36:18.418680 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 16:36:18.418680 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application created_value = creator( Aug 01 16:36:18.418680 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 16:36:18.418680 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 16:36:18.418680 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 16:36:18.418680 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 16:36:18.418680 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 16:36:18.418680 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 16:36:18.418680 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 01 16:36:18.418680 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application Aug 01 16:36:18.419289 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d5f37a7-6df8-4d99-9fad-4f2e238c0be1 None None] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:36:18.420015 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 164/323] 10.222.0.20 () {56 vars in 1099 bytes} [Mon Aug 1 16:36:18 2022] GET /identity/v3/roles/reader => generated 84 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 16:36:18.425529 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a77dd65-e5b6-4ff3-a251-7543504fe335 None admin] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:36:18.463869 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG dbcounter [-] [87058] Writing DB stats keystone:SELECT=100 {{(pid=87058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:36:18.526272 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d5f37a7-6df8-4d99-9fad-4f2e238c0be1 None None] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:36:18.532929 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a77dd65-e5b6-4ff3-a251-7543504fe335 None admin] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:36:18.561511 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c8bdd0f-ed2f-4fd9-b32e-77e460ae7c78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:36:18.562191 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c8bdd0f-ed2f-4fd9-b32e-77e460ae7c78 None admin] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:36:18.562469 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c8bdd0f-ed2f-4fd9-b32e-77e460ae7c78 None admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:36:18.562738 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c8bdd0f-ed2f-4fd9-b32e-77e460ae7c78 None admin] PATH_INFO: `/v3/domains/default` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:36:18.563876 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c8bdd0f-ed2f-4fd9-b32e-77e460ae7c78 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:36:18.565314 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c8bdd0f-ed2f-4fd9-b32e-77e460ae7c78 None admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:36:18.571831 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 160/324] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Aug 1 16:36:18 2022] GET /identity/v3/domains/default => generated 199 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 16:36:18.572800 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c8bdd0f-ed2f-4fd9-b32e-77e460ae7c78 None admin] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:36:18.577959 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e93845ee-30fa-493a-a845-d38d522fdaf0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:36:18.578674 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93845ee-30fa-493a-a845-d38d522fdaf0 None admin] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:36:18.579017 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93845ee-30fa-493a-a845-d38d522fdaf0 None admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:36:18.579353 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93845ee-30fa-493a-a845-d38d522fdaf0 None admin] PATH_INFO: `/v3/roles` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:36:18.580144 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e93845ee-30fa-493a-a845-d38d522fdaf0 None admin] RBAC: Adding query filter params (name=reader) {{(pid=87058) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:36:18.580514 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e93845ee-30fa-493a-a845-d38d522fdaf0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:36:18.582402 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e93845ee-30fa-493a-a845-d38d522fdaf0 None admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:36:18.587806 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG dbcounter [-] [87057] Writing DB stats keystone:SELECT=100 {{(pid=87057) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:36:18.592030 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 165/325] 10.222.0.20 () {56 vars in 1101 bytes} [Mon Aug 1 16:36:18 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 16:36:18.597424 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e93845ee-30fa-493a-a845-d38d522fdaf0 None admin] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:36:18.687518 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c8bdd0f-ed2f-4fd9-b32e-77e460ae7c78 None admin] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:36:18.700594 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG dbcounter [-] [87058] Writing DB stats keystone:SELECT=100 {{(pid=87058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:36:18.704085 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e93845ee-30fa-493a-a845-d38d522fdaf0 None admin] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:36:18.724653 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50cb841b-45bc-45e0-ac8e-e0ca87a1b847 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:36:18.725325 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50cb841b-45bc-45e0-ac8e-e0ca87a1b847 None admin] REQUEST_METHOD: `POST` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:36:18.725603 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50cb841b-45bc-45e0-ac8e-e0ca87a1b847 None admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:36:18.726327 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50cb841b-45bc-45e0-ac8e-e0ca87a1b847 None admin] PATH_INFO: `/v3/groups` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:36:18.726996 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50cb841b-45bc-45e0-ac8e-e0ca87a1b847 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:36:18.728867 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50cb841b-45bc-45e0-ac8e-e0ca87a1b847 None admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:36:18.741465 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd038da3-e928-4cd0-969f-b01d4d13669c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:36:18.742701 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd038da3-e928-4cd0-969f-b01d4d13669c None admin] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:36:18.742975 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd038da3-e928-4cd0-969f-b01d4d13669c None admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:36:18.743243 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd038da3-e928-4cd0-969f-b01d4d13669c None admin] PATH_INFO: `/v3/users/c173554995b744a6ba3a400282d04418` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:36:18.744403 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd038da3-e928-4cd0-969f-b01d4d13669c None admin] RBAC: Authorizing `identity:get_user(user_id=c173554995b744a6ba3a400282d04418)` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:36:18.745815 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd038da3-e928-4cd0-969f-b01d4d13669c None admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:36:18.747485 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 166/326] 10.222.0.20 () {56 vars in 1177 bytes} [Mon Aug 1 16:36:18 2022] GET /identity/v3/users/c173554995b744a6ba3a400282d04418 => generated 293 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 16:36:18.752807 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd038da3-e928-4cd0-969f-b01d4d13669c None admin] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:36:18.815411 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 161/327] 10.222.0.20 () {60 vars in 1134 bytes} [Mon Aug 1 16:36:18 2022] POST /identity/v3/groups => generated 226 bytes in 243 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 16:36:18.863284 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd038da3-e928-4cd0-969f-b01d4d13669c None admin] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:36:18.911241 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-352bdc21-1641-4a1e-9943-ffc2d97e4599 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:36:18.911990 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352bdc21-1641-4a1e-9943-ffc2d97e4599 None admin] REQUEST_METHOD: `PUT` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:36:18.912197 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352bdc21-1641-4a1e-9943-ffc2d97e4599 None admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:36:18.912468 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352bdc21-1641-4a1e-9943-ffc2d97e4599 None admin] PATH_INFO: `/v3/system/users/c173554995b744a6ba3a400282d04418/roles/02c40c90308547dd8ac5aaad0da8d68c` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:36:18.915375 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-352bdc21-1641-4a1e-9943-ffc2d97e4599 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=c173554995b744a6ba3a400282d04418, role_id=02c40c90308547dd8ac5aaad0da8d68c)` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:36:18.916787 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-352bdc21-1641-4a1e-9943-ffc2d97e4599 None admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:36:18.981900 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 167/328] 10.222.0.20 () {58 vars in 1334 bytes} [Mon Aug 1 16:36:18 2022] PUT /identity/v3/system/users/c173554995b744a6ba3a400282d04418/roles/02c40c90308547dd8ac5aaad0da8d68c => generated 0 bytes in 229 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 16:36:19.108626 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8214648c-f452-47fe-9dee-4cc4098d2f4e None None] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:36:19.108970 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8214648c-f452-47fe-9dee-4cc4098d2f4e None None] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:36:19.109176 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8214648c-f452-47fe-9dee-4cc4098d2f4e None None] PATH_INFO: `/` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:36:19.110733 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 162/329] 10.222.0.20 () {54 vars in 926 bytes} [Mon Aug 1 16:36:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 16:36:19.118016 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d40eb21-a7c8-4a3f-9d18-0eda4971ac2f None None] REQUEST_METHOD: `POST` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:36:19.118354 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d40eb21-a7c8-4a3f-9d18-0eda4971ac2f None None] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:36:19.118584 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d40eb21-a7c8-4a3f-9d18-0eda4971ac2f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:36:19.139279 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.auth.core [None req-9d40eb21-a7c8-4a3f-9d18-0eda4971ac2f None None] MFA Rules not processed for user `1007f8e379b74e169b98fb7c3c0ba5b4`. Rule list: `[]` (Enabled: `True`). {{(pid=87058) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:36:19.140493 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.fernet_utils [None req-9d40eb21-a7c8-4a3f-9d18-0eda4971ac2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:36:19.167673 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG dbcounter [-] [87058] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:36:19.217507 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 168/330] 10.222.0.20 () {58 vars in 1023 bytes} [Mon Aug 1 16:36:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 16:36:19.243632 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8214648c-f452-47fe-9dee-4cc4098d2f4e None None] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:36:19.328569 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG dbcounter [-] [87057] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87057) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:36:19.362192 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8214648c-f452-47fe-9dee-4cc4098d2f4e None None] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:36:19.400002 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77b465ba-7f2d-4261-ad6c-465020282bf9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:36:19.401030 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b465ba-7f2d-4261-ad6c-465020282bf9 None admin] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:36:19.401906 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b465ba-7f2d-4261-ad6c-465020282bf9 None admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:36:19.402214 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b465ba-7f2d-4261-ad6c-465020282bf9 None admin] PATH_INFO: `/v3/roles/member` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:36:19.413141 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77b465ba-7f2d-4261-ad6c-465020282bf9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 16:36:19.419002 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application [None req-77b465ba-7f2d-4261-ad6c-465020282bf9 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 01 16:36:19.419002 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 16:36:19.419002 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 16:36:19.419002 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 16:36:19.419002 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 16:36:19.419002 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 16:36:19.419002 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 16:36:19.419002 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 16:36:19.419002 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 16:36:19.419002 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 16:36:19.419002 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 16:36:19.419002 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 16:36:19.419002 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 16:36:19.419002 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 16:36:19.419002 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 16:36:19.419002 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 16:36:19.419880 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 16:36:19.419880 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 16:36:19.419880 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 16:36:19.419880 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 16:36:19.419880 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 16:36:19.419880 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 16:36:19.419880 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 16:36:19.419880 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 16:36:19.419880 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 16:36:19.419880 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 16:36:19.419880 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 16:36:19.419880 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application with Lock( Aug 01 16:36:19.419880 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 16:36:19.419880 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self._enter() Aug 01 16:36:19.419880 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 16:36:19.419880 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 16:36:19.420676 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 16:36:19.420676 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self.creator() Aug 01 16:36:19.420676 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 16:36:19.420676 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application created_value = creator( Aug 01 16:36:19.420676 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 16:36:19.420676 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 16:36:19.420676 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 16:36:19.420676 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 16:36:19.420676 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 16:36:19.420676 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 16:36:19.420676 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 01 16:36:19.420676 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application Aug 01 16:36:19.422870 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 163/331] 10.222.0.20 () {56 vars in 1099 bytes} [Mon Aug 1 16:36:19 2022] GET /identity/v3/roles/member => generated 84 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 16:36:19.426469 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d40eb21-a7c8-4a3f-9d18-0eda4971ac2f None None] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:36:19.549179 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d40eb21-a7c8-4a3f-9d18-0eda4971ac2f None None] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:36:19.594557 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-405bc2c1-f7d2-49c5-a11a-a5349a6724d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:36:19.595188 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405bc2c1-f7d2-49c5-a11a-a5349a6724d4 None admin] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:36:19.595469 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405bc2c1-f7d2-49c5-a11a-a5349a6724d4 None admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:36:19.595747 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405bc2c1-f7d2-49c5-a11a-a5349a6724d4 None admin] PATH_INFO: `/v3/roles` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:36:19.596479 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-405bc2c1-f7d2-49c5-a11a-a5349a6724d4 None admin] RBAC: Adding query filter params (name=member) {{(pid=87058) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:36:19.596787 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-405bc2c1-f7d2-49c5-a11a-a5349a6724d4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:36:19.598196 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-405bc2c1-f7d2-49c5-a11a-a5349a6724d4 None admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:36:19.609627 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 169/332] 10.222.0.20 () {56 vars in 1101 bytes} [Mon Aug 1 16:36:19 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:36:19.615581 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77b465ba-7f2d-4261-ad6c-465020282bf9 None admin] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:36:19.726483 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77b465ba-7f2d-4261-ad6c-465020282bf9 None admin] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:36:19.759331 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG dbcounter [-] [87057] Writing DB stats keystone:SELECT=100 {{(pid=87057) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:36:19.764520 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff70f97-fd70-4e54-bbc9-c78eafabc582 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:36:19.765177 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff70f97-fd70-4e54-bbc9-c78eafabc582 None admin] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:36:19.765456 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff70f97-fd70-4e54-bbc9-c78eafabc582 None admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:36:19.765735 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff70f97-fd70-4e54-bbc9-c78eafabc582 None admin] PATH_INFO: `/v3/users/950e03b49834417cac6a2df952ce522e` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:36:19.766953 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ff70f97-fd70-4e54-bbc9-c78eafabc582 None admin] RBAC: Authorizing `identity:get_user(user_id=950e03b49834417cac6a2df952ce522e)` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:36:19.768796 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ff70f97-fd70-4e54-bbc9-c78eafabc582 None admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:36:19.770692 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 164/333] 10.222.0.20 () {56 vars in 1177 bytes} [Mon Aug 1 16:36:19 2022] GET /identity/v3/users/950e03b49834417cac6a2df952ce522e => generated 293 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:36:19.777020 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-405bc2c1-f7d2-49c5-a11a-a5349a6724d4 None admin] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:36:19.804937 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG dbcounter [-] [87058] Writing DB stats keystone:SELECT=100 {{(pid=87058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:36:19.910055 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-405bc2c1-f7d2-49c5-a11a-a5349a6724d4 None admin] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:36:19.957528 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee10b5f6-1930-41f6-8725-e19ea028b229 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:36:19.958179 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee10b5f6-1930-41f6-8725-e19ea028b229 None admin] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:36:19.958459 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee10b5f6-1930-41f6-8725-e19ea028b229 None admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:36:19.958727 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee10b5f6-1930-41f6-8725-e19ea028b229 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:36:19.959524 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee10b5f6-1930-41f6-8725-e19ea028b229 None admin] RBAC: Adding query filter params (role.id=0999d3da63a545f18ff3252afdc90518, user.id=950e03b49834417cac6a2df952ce522e, scope.system=all) {{(pid=87058) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:36:19.959832 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee10b5f6-1930-41f6-8725-e19ea028b229 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:36:19.961461 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee10b5f6-1930-41f6-8725-e19ea028b229 None admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:36:19.978562 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 170/334] 10.222.0.20 () {56 vars in 1308 bytes} [Mon Aug 1 16:36:19 2022] GET /identity/v3/role_assignments?user.id=950e03b49834417cac6a2df952ce522e&scope.system=all&role.id=0999d3da63a545f18ff3252afdc90518 => generated 510 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:36:21.278622 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8345f5b7-5fcb-45c0-b9d3-67ad25cfb48a None None] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:36:21.279141 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8345f5b7-5fcb-45c0-b9d3-67ad25cfb48a None None] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:36:21.279141 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8345f5b7-5fcb-45c0-b9d3-67ad25cfb48a None None] PATH_INFO: `/` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:36:21.280757 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 165/335] 10.222.0.20 () {54 vars in 926 bytes} [Mon Aug 1 16:36:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 16:36:21.288752 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b721f571-398d-46f3-a51d-37561fec3c53 None None] REQUEST_METHOD: `POST` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:36:21.289375 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b721f571-398d-46f3-a51d-37561fec3c53 None None] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:36:21.289375 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b721f571-398d-46f3-a51d-37561fec3c53 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:36:21.308835 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.auth.core [None req-b721f571-398d-46f3-a51d-37561fec3c53 None None] MFA Rules not processed for user `1007f8e379b74e169b98fb7c3c0ba5b4`. Rule list: `[]` (Enabled: `True`). {{(pid=87058) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:36:21.310062 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.fernet_utils [None req-b721f571-398d-46f3-a51d-37561fec3c53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:36:21.381875 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a54ceaa-31c3-4200-b184-2d2ae4baaf1e None None] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:36:21.383146 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a54ceaa-31c3-4200-b184-2d2ae4baaf1e None None] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:36:21.383619 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a54ceaa-31c3-4200-b184-2d2ae4baaf1e None None] PATH_INFO: `/` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:36:21.385458 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 166/336] 10.222.0.20 () {54 vars in 926 bytes} [Mon Aug 1 16:36:21 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 16:36:21.391472 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1709eba2-3445-4ac2-9d2f-441439eb5025 None None] REQUEST_METHOD: `POST` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:36:21.392003 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1709eba2-3445-4ac2-9d2f-441439eb5025 None None] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:36:21.400298 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1709eba2-3445-4ac2-9d2f-441439eb5025 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:36:21.400409 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 171/337] 10.222.0.20 () {58 vars in 1023 bytes} [Mon Aug 1 16:36:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 01 16:36:21.429995 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b721f571-398d-46f3-a51d-37561fec3c53 None None] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:36:21.430209 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.auth.core [None req-1709eba2-3445-4ac2-9d2f-441439eb5025 None None] MFA Rules not processed for user `1007f8e379b74e169b98fb7c3c0ba5b4`. Rule list: `[]` (Enabled: `True`). {{(pid=87057) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:36:21.431666 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.fernet_utils [None req-1709eba2-3445-4ac2-9d2f-441439eb5025 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87057) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:36:21.442791 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG dbcounter [-] [87058] Writing DB stats keystone:SELECT=100 {{(pid=87058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:36:21.558632 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b721f571-398d-46f3-a51d-37561fec3c53 None None] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:36:21.563375 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 167/338] 10.222.0.20 () {58 vars in 1023 bytes} [Mon Aug 1 16:36:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 16:36:21.603034 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1709eba2-3445-4ac2-9d2f-441439eb5025 None None] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:36:21.614050 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06715d56-acaa-4945-89cc-c95cdd22fb09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:36:21.614686 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06715d56-acaa-4945-89cc-c95cdd22fb09 None admin] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:36:21.614963 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06715d56-acaa-4945-89cc-c95cdd22fb09 None admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:36:21.615234 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06715d56-acaa-4945-89cc-c95cdd22fb09 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:36:21.631296 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06715d56-acaa-4945-89cc-c95cdd22fb09 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 16:36:21.637022 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application [None req-06715d56-acaa-4945-89cc-c95cdd22fb09 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 01 16:36:21.637022 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 16:36:21.637022 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 16:36:21.637022 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 16:36:21.637022 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 16:36:21.637022 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 16:36:21.637022 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 16:36:21.637022 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 16:36:21.637022 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 16:36:21.637022 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 16:36:21.637022 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 16:36:21.637022 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 16:36:21.637022 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 16:36:21.637022 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 16:36:21.637022 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 16:36:21.637022 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 16:36:21.637953 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 16:36:21.637953 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 16:36:21.637953 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 16:36:21.637953 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 16:36:21.637953 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 16:36:21.637953 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 16:36:21.637953 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 16:36:21.637953 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 16:36:21.637953 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 16:36:21.637953 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 16:36:21.637953 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 16:36:21.637953 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application with Lock( Aug 01 16:36:21.637953 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 16:36:21.637953 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self._enter() Aug 01 16:36:21.637953 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 16:36:21.637953 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 16:36:21.638777 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 16:36:21.638777 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self.creator() Aug 01 16:36:21.638777 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 16:36:21.638777 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application created_value = creator( Aug 01 16:36:21.638777 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 16:36:21.638777 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 16:36:21.638777 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 16:36:21.638777 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 16:36:21.638777 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 16:36:21.638777 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 16:36:21.638777 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 01 16:36:21.638777 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application Aug 01 16:36:21.642852 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 172/339] 10.222.0.20 () {56 vars in 1114 bytes} [Mon Aug 1 16:36:21 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 01 16:36:21.648714 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06715d56-acaa-4945-89cc-c95cdd22fb09 None admin] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:36:21.745658 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG dbcounter [-] [87058] Writing DB stats keystone:SELECT=100 {{(pid=87058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:36:21.761197 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1709eba2-3445-4ac2-9d2f-441439eb5025 None None] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:36:21.773770 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06715d56-acaa-4945-89cc-c95cdd22fb09 None admin] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:36:21.805162 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG dbcounter [-] [87057] Writing DB stats keystone:SELECT=100 {{(pid=87057) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:36:21.811202 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde7537e-aa95-43bf-b1d8-c54f336d135a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:36:21.811802 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde7537e-aa95-43bf-b1d8-c54f336d135a None admin] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:36:21.812085 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde7537e-aa95-43bf-b1d8-c54f336d135a None admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:36:21.812363 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde7537e-aa95-43bf-b1d8-c54f336d135a None admin] PATH_INFO: `/v3/roles/reader` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:36:21.827273 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bde7537e-aa95-43bf-b1d8-c54f336d135a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 16:36:21.840511 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application [None req-bde7537e-aa95-43bf-b1d8-c54f336d135a None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 01 16:36:21.840511 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 16:36:21.840511 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 16:36:21.840511 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 16:36:21.840511 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 16:36:21.840511 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 16:36:21.840511 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 16:36:21.840511 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 16:36:21.840511 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 16:36:21.840511 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 16:36:21.840511 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 16:36:21.840511 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 16:36:21.840511 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 16:36:21.840511 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 16:36:21.840511 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 16:36:21.840511 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 16:36:21.841449 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 16:36:21.841449 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 16:36:21.841449 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 16:36:21.841449 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 16:36:21.841449 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 16:36:21.841449 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 16:36:21.841449 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 16:36:21.841449 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 16:36:21.841449 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 16:36:21.841449 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 16:36:21.841449 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 16:36:21.841449 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application with Lock( Aug 01 16:36:21.841449 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 16:36:21.841449 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self._enter() Aug 01 16:36:21.841449 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 16:36:21.841449 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 16:36:21.842733 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 16:36:21.842733 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self.creator() Aug 01 16:36:21.842733 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 16:36:21.842733 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application created_value = creator( Aug 01 16:36:21.842733 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 16:36:21.842733 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 16:36:21.842733 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 16:36:21.842733 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 16:36:21.842733 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 16:36:21.842733 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 16:36:21.842733 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 01 16:36:21.842733 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application Aug 01 16:36:21.842733 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 168/340] 10.222.0.20 () {56 vars in 1099 bytes} [Mon Aug 1 16:36:21 2022] GET /identity/v3/roles/reader => generated 84 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 16:36:21.853125 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde7537e-aa95-43bf-b1d8-c54f336d135a None admin] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:36:21.858745 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-372e2423-d956-42ae-8ede-7e202bbfd679 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:36:21.859377 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372e2423-d956-42ae-8ede-7e202bbfd679 None admin] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:36:21.859650 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372e2423-d956-42ae-8ede-7e202bbfd679 None admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:36:21.859918 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372e2423-d956-42ae-8ede-7e202bbfd679 None admin] PATH_INFO: `/v3/roles` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:36:21.860647 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-372e2423-d956-42ae-8ede-7e202bbfd679 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=87058) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:36:21.860952 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-372e2423-d956-42ae-8ede-7e202bbfd679 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:36:21.862340 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-372e2423-d956-42ae-8ede-7e202bbfd679 None admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:36:21.873426 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 173/341] 10.222.0.20 () {56 vars in 1111 bytes} [Mon Aug 1 16:36:21 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:36:21.878877 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-372e2423-d956-42ae-8ede-7e202bbfd679 None admin] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:36:21.984879 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde7537e-aa95-43bf-b1d8-c54f336d135a None admin] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:36:22.022966 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe4d06b8-c562-45e1-8d91-3930812ed71a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:36:22.023733 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4d06b8-c562-45e1-8d91-3930812ed71a None admin] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:36:22.024186 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4d06b8-c562-45e1-8d91-3930812ed71a None admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:36:22.024463 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4d06b8-c562-45e1-8d91-3930812ed71a None admin] PATH_INFO: `/v3/roles` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:36:22.025199 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe4d06b8-c562-45e1-8d91-3930812ed71a None admin] RBAC: Adding query filter params (name=reader) {{(pid=87057) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:36:22.025506 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe4d06b8-c562-45e1-8d91-3930812ed71a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:36:22.026921 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe4d06b8-c562-45e1-8d91-3930812ed71a None admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:36:22.036086 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 169/342] 10.222.0.20 () {56 vars in 1101 bytes} [Mon Aug 1 16:36:21 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:36:22.047146 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe4d06b8-c562-45e1-8d91-3930812ed71a None admin] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:36:22.120393 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG dbcounter [-] [87057] Writing DB stats keystone:SELECT=100 {{(pid=87057) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:36:22.134282 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-372e2423-d956-42ae-8ede-7e202bbfd679 None admin] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:36:22.171971 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe4d06b8-c562-45e1-8d91-3930812ed71a None admin] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:36:22.182273 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG dbcounter [-] [87058] Writing DB stats keystone:SELECT=100 {{(pid=87058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:36:22.199936 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f502d1b6-c7ae-474d-ac25-43ea0eb617ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:36:22.202170 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f502d1b6-c7ae-474d-ac25-43ea0eb617ef None admin] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:36:22.202658 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f502d1b6-c7ae-474d-ac25-43ea0eb617ef None admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:36:22.204027 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f502d1b6-c7ae-474d-ac25-43ea0eb617ef None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:36:22.220143 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-021c8131-01ff-43b1-9f4f-b530a2aad38d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:36:22.221042 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021c8131-01ff-43b1-9f4f-b530a2aad38d None admin] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:36:22.221471 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021c8131-01ff-43b1-9f4f-b530a2aad38d None admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:36:22.221916 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021c8131-01ff-43b1-9f4f-b530a2aad38d None admin] PATH_INFO: `/v3/users/c173554995b744a6ba3a400282d04418` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:36:22.225128 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-021c8131-01ff-43b1-9f4f-b530a2aad38d None admin] RBAC: Authorizing `identity:get_user(user_id=c173554995b744a6ba3a400282d04418)` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:36:22.228567 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-021c8131-01ff-43b1-9f4f-b530a2aad38d None admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:36:22.230407 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 170/343] 10.222.0.20 () {56 vars in 1177 bytes} [Mon Aug 1 16:36:22 2022] GET /identity/v3/users/c173554995b744a6ba3a400282d04418 => generated 293 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 16:36:22.232425 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf0be95-332c-4eb5-9088-fc29b98ae3cf None None] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:36:22.233044 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf0be95-332c-4eb5-9088-fc29b98ae3cf None None] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:36:22.234907 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf0be95-332c-4eb5-9088-fc29b98ae3cf None None] PATH_INFO: `/` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:36:22.237116 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 171/344] 10.222.0.20 () {54 vars in 926 bytes} [Mon Aug 1 16:36:22 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 16:36:22.238772 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaf0be95-332c-4eb5-9088-fc29b98ae3cf None None] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:36:22.346190 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f502d1b6-c7ae-474d-ac25-43ea0eb617ef None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:36:22.348311 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f502d1b6-c7ae-474d-ac25-43ea0eb617ef None admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:36:22.435446 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaf0be95-332c-4eb5-9088-fc29b98ae3cf None None] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:36:22.467844 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG dbcounter [-] [87057] Writing DB stats keystone:SELECT=100 {{(pid=87057) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:36:22.489946 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88413df2-fe76-4c03-a866-de98d2b2c64c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:36:22.497272 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88413df2-fe76-4c03-a866-de98d2b2c64c None admin] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:36:22.497272 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88413df2-fe76-4c03-a866-de98d2b2c64c None admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:36:22.497272 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88413df2-fe76-4c03-a866-de98d2b2c64c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:36:22.497272 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88413df2-fe76-4c03-a866-de98d2b2c64c None admin] RBAC: Adding query filter params (role.id=02c40c90308547dd8ac5aaad0da8d68c, user.id=c173554995b744a6ba3a400282d04418, scope.system=all) {{(pid=87057) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:36:22.497272 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88413df2-fe76-4c03-a866-de98d2b2c64c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:36:22.497272 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88413df2-fe76-4c03-a866-de98d2b2c64c None admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:36:22.503794 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 174/345] 10.222.0.20 () {58 vars in 1282 bytes} [Mon Aug 1 16:36:21 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 625 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 01 16:36:22.505324 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20abbbec-c6af-44fa-9cf5-f0882c523f5d None None] REQUEST_METHOD: `POST` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:36:22.505762 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20abbbec-c6af-44fa-9cf5-f0882c523f5d None None] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:36:22.506210 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20abbbec-c6af-44fa-9cf5-f0882c523f5d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:36:22.513229 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 172/346] 10.222.0.20 () {56 vars in 1308 bytes} [Mon Aug 1 16:36:22 2022] GET /identity/v3/role_assignments?user.id=c173554995b744a6ba3a400282d04418&scope.system=all&role.id=02c40c90308547dd8ac5aaad0da8d68c => generated 510 bytes in 274 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:36:22.518263 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a555988-418f-487c-bcb1-d2348671cff4 None None] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:36:22.519863 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a555988-418f-487c-bcb1-d2348671cff4 None None] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:36:22.520319 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a555988-418f-487c-bcb1-d2348671cff4 None None] PATH_INFO: `/` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:36:22.521972 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 173/347] 10.222.0.20 () {54 vars in 926 bytes} [Mon Aug 1 16:36:22 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 16:36:22.527813 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a555988-418f-487c-bcb1-d2348671cff4 None None] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:36:22.546264 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.auth.core [None req-20abbbec-c6af-44fa-9cf5-f0882c523f5d None None] MFA Rules not processed for user `1007f8e379b74e169b98fb7c3c0ba5b4`. Rule list: `[]` (Enabled: `True`). {{(pid=87058) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:36:22.548137 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.fernet_utils [None req-20abbbec-c6af-44fa-9cf5-f0882c523f5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:36:22.602881 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG dbcounter [-] [87058] Writing DB stats keystone:SELECT=100 {{(pid=87058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:36:22.657916 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a555988-418f-487c-bcb1-d2348671cff4 None None] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:36:22.682748 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 175/348] 10.222.0.20 () {58 vars in 1023 bytes} [Mon Aug 1 16:36:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 16:36:22.683911 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0150e7d9-fda8-4472-8e2f-5671bcade425 None None] REQUEST_METHOD: `POST` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:36:22.684193 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0150e7d9-fda8-4472-8e2f-5671bcade425 None None] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:36:22.684465 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0150e7d9-fda8-4472-8e2f-5671bcade425 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:36:22.698278 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f12170fa-e77f-4394-b79b-d44dc4b69170 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:36:22.699153 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12170fa-e77f-4394-b79b-d44dc4b69170 None admin] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:36:22.699594 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12170fa-e77f-4394-b79b-d44dc4b69170 None admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:36:22.700027 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12170fa-e77f-4394-b79b-d44dc4b69170 None admin] PATH_INFO: `/v3/projects/64c2a2f6f91d4b9c8d5993d28fd19631` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:36:22.701352 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f12170fa-e77f-4394-b79b-d44dc4b69170 None admin] RBAC: Authorizing `identity:get_project(project_id=64c2a2f6f91d4b9c8d5993d28fd19631)` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:36:22.703957 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f12170fa-e77f-4394-b79b-d44dc4b69170 None admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:36:22.706136 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 174/349] 10.222.0.20 () {56 vars in 1186 bytes} [Mon Aug 1 16:36:22 2022] GET /identity/v3/projects/64c2a2f6f91d4b9c8d5993d28fd19631 => generated 298 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:36:22.716036 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f12170fa-e77f-4394-b79b-d44dc4b69170 None admin] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:36:22.723042 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.auth.core [None req-0150e7d9-fda8-4472-8e2f-5671bcade425 None None] MFA Rules not processed for user `1007f8e379b74e169b98fb7c3c0ba5b4`. Rule list: `[]` (Enabled: `True`). {{(pid=87058) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:36:22.724249 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.fernet_utils [None req-0150e7d9-fda8-4472-8e2f-5671bcade425 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:36:22.772703 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG dbcounter [-] [87057] Writing DB stats keystone:SELECT=100 {{(pid=87057) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:36:22.826125 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 176/350] 10.222.0.20 () {58 vars in 1023 bytes} [Mon Aug 1 16:36:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 16:36:22.827851 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0150e7d9-fda8-4472-8e2f-5671bcade425 None None] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:36:22.847860 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f12170fa-e77f-4394-b79b-d44dc4b69170 None admin] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:36:22.895754 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d263f13-2c60-4e3c-bed6-4037b4ebbc60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:36:22.896786 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d263f13-2c60-4e3c-bed6-4037b4ebbc60 None admin] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:36:22.897491 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d263f13-2c60-4e3c-bed6-4037b4ebbc60 None admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:36:22.897772 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d263f13-2c60-4e3c-bed6-4037b4ebbc60 None admin] PATH_INFO: `/v3/groups/64b4bd12c620422ca16e3edfbdab7fe7` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:36:22.905497 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d263f13-2c60-4e3c-bed6-4037b4ebbc60 None admin] RBAC: Authorizing `identity:get_group(group_id=64b4bd12c620422ca16e3edfbdab7fe7)` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:36:22.906989 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d263f13-2c60-4e3c-bed6-4037b4ebbc60 None admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:36:22.914446 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 175/351] 10.222.0.20 () {56 vars in 1180 bytes} [Mon Aug 1 16:36:22 2022] GET /identity/v3/groups/64b4bd12c620422ca16e3edfbdab7fe7 => generated 226 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 16:36:22.915324 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d263f13-2c60-4e3c-bed6-4037b4ebbc60 None admin] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:36:22.964901 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG dbcounter [-] [87058] Writing DB stats keystone:SELECT=100 {{(pid=87058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:36:22.977241 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0150e7d9-fda8-4472-8e2f-5671bcade425 None None] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:36:23.029841 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c840a68-904a-4c47-8331-95b774fd0ca7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:36:23.031595 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c840a68-904a-4c47-8331-95b774fd0ca7 None admin] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:36:23.031882 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c840a68-904a-4c47-8331-95b774fd0ca7 None admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:36:23.032603 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c840a68-904a-4c47-8331-95b774fd0ca7 None admin] PATH_INFO: `/v3/roles/member` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:36:23.046312 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d263f13-2c60-4e3c-bed6-4037b4ebbc60 None admin] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:36:23.046597 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c840a68-904a-4c47-8331-95b774fd0ca7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 16:36:23.052593 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application [None req-4c840a68-904a-4c47-8331-95b774fd0ca7 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 01 16:36:23.052593 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 16:36:23.052593 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 16:36:23.052593 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 16:36:23.052593 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 16:36:23.052593 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 16:36:23.052593 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 16:36:23.052593 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 16:36:23.052593 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 16:36:23.052593 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 16:36:23.052593 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 16:36:23.052593 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 16:36:23.052593 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 16:36:23.052593 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 16:36:23.052593 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 16:36:23.052593 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 16:36:23.053488 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 16:36:23.053488 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 16:36:23.053488 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 16:36:23.053488 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 16:36:23.053488 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 16:36:23.053488 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 16:36:23.053488 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 16:36:23.053488 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 16:36:23.053488 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 16:36:23.053488 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 16:36:23.053488 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 16:36:23.053488 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application with Lock( Aug 01 16:36:23.053488 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 16:36:23.053488 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self._enter() Aug 01 16:36:23.053488 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 16:36:23.053488 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 16:36:23.054968 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 16:36:23.054968 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self.creator() Aug 01 16:36:23.054968 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 16:36:23.054968 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application created_value = creator( Aug 01 16:36:23.054968 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 16:36:23.054968 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 16:36:23.054968 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 16:36:23.054968 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 16:36:23.054968 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 16:36:23.054968 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 16:36:23.054968 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 01 16:36:23.054968 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application Aug 01 16:36:23.054968 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 177/352] 10.222.0.20 () {56 vars in 1099 bytes} [Mon Aug 1 16:36:22 2022] GET /identity/v3/roles/member => generated 84 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 16:36:23.055808 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c840a68-904a-4c47-8331-95b774fd0ca7 None admin] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:36:23.063724 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG dbcounter [-] [87057] Writing DB stats keystone:SELECT=100 {{(pid=87057) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:36:23.097330 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b54e2f9-53e1-4193-add9-664fcef94732 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:36:23.098289 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b54e2f9-53e1-4193-add9-664fcef94732 None admin] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:36:23.098569 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b54e2f9-53e1-4193-add9-664fcef94732 None admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:36:23.098844 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b54e2f9-53e1-4193-add9-664fcef94732 None admin] PATH_INFO: `/v3/roles/member` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:36:23.119553 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b54e2f9-53e1-4193-add9-664fcef94732 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 16:36:23.126814 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application [None req-9b54e2f9-53e1-4193-add9-664fcef94732 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 01 16:36:23.126814 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 16:36:23.126814 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 16:36:23.126814 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 16:36:23.126814 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 16:36:23.126814 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 16:36:23.126814 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 16:36:23.126814 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 16:36:23.126814 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 16:36:23.126814 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 16:36:23.126814 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 16:36:23.126814 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 16:36:23.126814 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 16:36:23.126814 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 16:36:23.126814 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 16:36:23.126814 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 16:36:23.128912 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 16:36:23.128912 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 16:36:23.128912 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 16:36:23.128912 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 16:36:23.128912 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 16:36:23.128912 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 16:36:23.128912 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 16:36:23.128912 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 16:36:23.128912 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 16:36:23.128912 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 16:36:23.128912 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 16:36:23.128912 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application with Lock( Aug 01 16:36:23.128912 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 16:36:23.128912 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self._enter() Aug 01 16:36:23.128912 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 16:36:23.128912 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 16:36:23.130298 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 16:36:23.130298 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self.creator() Aug 01 16:36:23.130298 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 16:36:23.130298 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application created_value = creator( Aug 01 16:36:23.130298 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 16:36:23.130298 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 16:36:23.130298 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 16:36:23.130298 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 16:36:23.130298 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 16:36:23.130298 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 16:36:23.130298 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 01 16:36:23.130298 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application Aug 01 16:36:23.130298 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 176/353] 10.222.0.20 () {56 vars in 1099 bytes} [Mon Aug 1 16:36:22 2022] GET /identity/v3/roles/member => generated 84 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 16:36:23.131845 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b54e2f9-53e1-4193-add9-664fcef94732 None admin] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:36:23.183633 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c840a68-904a-4c47-8331-95b774fd0ca7 None admin] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:36:23.222330 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d07af3-6800-4262-ae0a-3818ee56ea55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:36:23.222878 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d07af3-6800-4262-ae0a-3818ee56ea55 None admin] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:36:23.223158 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d07af3-6800-4262-ae0a-3818ee56ea55 None admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:36:23.223432 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d07af3-6800-4262-ae0a-3818ee56ea55 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:36:23.224234 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7d07af3-6800-4262-ae0a-3818ee56ea55 None admin] RBAC: Adding query filter params (group.id=64b4bd12c620422ca16e3edfbdab7fe7, role.id=3fac71a39f3f4ed198a3579064c30268, scope.project.id=64c2a2f6f91d4b9c8d5993d28fd19631) {{(pid=87058) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:36:23.224544 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7d07af3-6800-4262-ae0a-3818ee56ea55 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:36:23.226167 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7d07af3-6800-4262-ae0a-3818ee56ea55 None admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:36:23.232801 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 178/354] 10.222.0.20 () {56 vars in 1376 bytes} [Mon Aug 1 16:36:23 2022] GET /identity/v3/role_assignments?group.id=64b4bd12c620422ca16e3edfbdab7fe7&scope.project.id=64c2a2f6f91d4b9c8d5993d28fd19631&role.id=3fac71a39f3f4ed198a3579064c30268 => generated 260 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:36:23.233469 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d07af3-6800-4262-ae0a-3818ee56ea55 None admin] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:36:23.254627 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG dbcounter [-] [87058] Writing DB stats keystone:SELECT=100 {{(pid=87058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:36:23.264748 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b54e2f9-53e1-4193-add9-664fcef94732 None admin] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:36:23.305827 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01b79f1e-c6ee-47eb-846a-ac522f2c2a0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:36:23.306506 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b79f1e-c6ee-47eb-846a-ac522f2c2a0d None admin] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:36:23.306841 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b79f1e-c6ee-47eb-846a-ac522f2c2a0d None admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:36:23.307114 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b79f1e-c6ee-47eb-846a-ac522f2c2a0d None admin] PATH_INFO: `/v3/roles` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:36:23.307900 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01b79f1e-c6ee-47eb-846a-ac522f2c2a0d None admin] RBAC: Adding query filter params (name=member) {{(pid=87057) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:36:23.308204 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01b79f1e-c6ee-47eb-846a-ac522f2c2a0d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:36:23.309662 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01b79f1e-c6ee-47eb-846a-ac522f2c2a0d None admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:36:23.320539 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 177/355] 10.222.0.20 () {56 vars in 1101 bytes} [Mon Aug 1 16:36:23 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:36:23.326635 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01b79f1e-c6ee-47eb-846a-ac522f2c2a0d None admin] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:36:23.363836 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG dbcounter [-] [87057] Writing DB stats keystone:SELECT=100 {{(pid=87057) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:36:23.442917 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01b79f1e-c6ee-47eb-846a-ac522f2c2a0d None admin] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:36:23.443568 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d07af3-6800-4262-ae0a-3818ee56ea55 None admin] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:36:23.493638 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-584e0800-b80d-4edb-9c79-857114d2465b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:36:23.494273 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584e0800-b80d-4edb-9c79-857114d2465b None admin] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:36:23.494552 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584e0800-b80d-4edb-9c79-857114d2465b None admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:36:23.494871 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584e0800-b80d-4edb-9c79-857114d2465b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:36:23.495383 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2705a25-8773-4634-ba98-ffc2da8afefe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:36:23.496228 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2705a25-8773-4634-ba98-ffc2da8afefe None admin] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:36:23.496695 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2705a25-8773-4634-ba98-ffc2da8afefe None admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:36:23.497165 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2705a25-8773-4634-ba98-ffc2da8afefe None admin] PATH_INFO: `/v3/roles` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:36:23.498242 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2705a25-8773-4634-ba98-ffc2da8afefe None admin] RBAC: Adding query filter params (name=member) {{(pid=87058) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:36:23.498803 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2705a25-8773-4634-ba98-ffc2da8afefe None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:36:23.500379 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2705a25-8773-4634-ba98-ffc2da8afefe None admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:36:23.510117 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 179/356] 10.222.0.20 () {56 vars in 1101 bytes} [Mon Aug 1 16:36:23 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 277 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:36:23.511641 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b48c0dd-398c-46b1-a036-fe899bb6370f None None] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:36:23.512121 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b48c0dd-398c-46b1-a036-fe899bb6370f None None] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:36:23.512629 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b48c0dd-398c-46b1-a036-fe899bb6370f None None] PATH_INFO: `/` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:36:23.514374 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 180/357] 10.222.0.20 () {54 vars in 926 bytes} [Mon Aug 1 16:36:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 16:36:23.519607 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d21da6-7115-4c73-8dc4-63a5335a1286 None None] REQUEST_METHOD: `POST` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:36:23.520960 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d21da6-7115-4c73-8dc4-63a5335a1286 None None] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:36:23.521589 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d21da6-7115-4c73-8dc4-63a5335a1286 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:36:23.545210 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.auth.core [None req-24d21da6-7115-4c73-8dc4-63a5335a1286 None None] MFA Rules not processed for user `1007f8e379b74e169b98fb7c3c0ba5b4`. Rule list: `[]` (Enabled: `True`). {{(pid=87058) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:36:23.546779 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.fernet_utils [None req-24d21da6-7115-4c73-8dc4-63a5335a1286 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:36:23.582377 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-584e0800-b80d-4edb-9c79-857114d2465b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:36:23.583860 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-584e0800-b80d-4edb-9c79-857114d2465b None admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:36:23.618744 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG dbcounter [-] [87057] Writing DB stats keystone:SELECT=100 {{(pid=87057) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:36:23.625000 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 181/358] 10.222.0.20 () {58 vars in 1023 bytes} [Mon Aug 1 16:36:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 01 16:36:23.626239 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d21da6-7115-4c73-8dc4-63a5335a1286 None None] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:36:23.638298 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG dbcounter [-] [87058] Writing DB stats keystone:SELECT=100 {{(pid=87058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:36:23.697839 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 178/359] 10.222.0.20 () {58 vars in 1282 bytes} [Mon Aug 1 16:36:23 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 372 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 01 16:36:23.699069 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-584e0800-b80d-4edb-9c79-857114d2465b None admin] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:36:23.741892 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d21da6-7115-4c73-8dc4-63a5335a1286 None None] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:36:23.780564 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef9e0dbe-4bdb-4d70-b789-a83bc1e0f7ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:36:23.781286 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9e0dbe-4bdb-4d70-b789-a83bc1e0f7ca None admin] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:36:23.781627 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9e0dbe-4bdb-4d70-b789-a83bc1e0f7ca None admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:36:23.781993 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9e0dbe-4bdb-4d70-b789-a83bc1e0f7ca None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:36:23.813230 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-584e0800-b80d-4edb-9c79-857114d2465b None admin] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:36:23.855874 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG dbcounter [-] [87057] Writing DB stats keystone:SELECT=100 {{(pid=87057) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:36:23.862284 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db14a8fb-eaf6-4140-b669-543db6021a60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:36:23.863391 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db14a8fb-eaf6-4140-b669-543db6021a60 None admin] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:36:23.863866 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db14a8fb-eaf6-4140-b669-543db6021a60 None admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:36:23.864305 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db14a8fb-eaf6-4140-b669-543db6021a60 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:36:23.869429 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef9e0dbe-4bdb-4d70-b789-a83bc1e0f7ca None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:36:23.869429 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef9e0dbe-4bdb-4d70-b789-a83bc1e0f7ca None admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:36:23.880375 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG dbcounter [-] [87058] Writing DB stats keystone:SELECT=100 {{(pid=87058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:36:23.880550 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db14a8fb-eaf6-4140-b669-543db6021a60 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 16:36:23.887631 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application [None req-db14a8fb-eaf6-4140-b669-543db6021a60 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 01 16:36:23.887631 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 16:36:23.887631 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 16:36:23.887631 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 16:36:23.887631 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 16:36:23.887631 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 16:36:23.887631 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 16:36:23.887631 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 16:36:23.887631 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 16:36:23.887631 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 16:36:23.887631 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 16:36:23.887631 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 16:36:23.887631 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 16:36:23.887631 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 16:36:23.887631 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 16:36:23.887631 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 16:36:23.888554 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 16:36:23.888554 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 16:36:23.888554 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 16:36:23.888554 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 16:36:23.888554 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 16:36:23.888554 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 16:36:23.888554 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 16:36:23.888554 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 16:36:23.888554 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 16:36:23.888554 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 16:36:23.888554 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 16:36:23.888554 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application with Lock( Aug 01 16:36:23.888554 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 16:36:23.888554 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self._enter() Aug 01 16:36:23.888554 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 16:36:23.888554 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 16:36:23.889364 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 16:36:23.889364 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self.creator() Aug 01 16:36:23.889364 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 16:36:23.889364 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application created_value = creator( Aug 01 16:36:23.889364 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 16:36:23.889364 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 16:36:23.889364 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 16:36:23.889364 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 16:36:23.889364 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 16:36:23.889364 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 16:36:23.889364 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 01 16:36:23.889364 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application Aug 01 16:36:23.889364 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 179/360] 10.222.0.20 () {56 vars in 1114 bytes} [Mon Aug 1 16:36:23 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 16:36:23.890974 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db14a8fb-eaf6-4140-b669-543db6021a60 None admin] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:36:24.012463 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 182/361] 10.222.0.20 () {58 vars in 1282 bytes} [Mon Aug 1 16:36:23 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 387 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 01 16:36:24.013737 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4ca976-79fd-42df-afbc-4c6b1e32c170 None None] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:36:24.014020 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4ca976-79fd-42df-afbc-4c6b1e32c170 None None] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:36:24.014319 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4ca976-79fd-42df-afbc-4c6b1e32c170 None None] PATH_INFO: `/` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:36:24.016297 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 183/362] 10.222.0.20 () {54 vars in 926 bytes} [Mon Aug 1 16:36:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 16:36:24.016297 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff4ca976-79fd-42df-afbc-4c6b1e32c170 None None] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:36:24.032653 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db14a8fb-eaf6-4140-b669-543db6021a60 None admin] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:36:24.083317 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ca031a9-6aad-4d3c-8073-5a53ea0a1948 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:36:24.084408 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca031a9-6aad-4d3c-8073-5a53ea0a1948 None admin] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:36:24.084686 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca031a9-6aad-4d3c-8073-5a53ea0a1948 None admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:36:24.084957 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca031a9-6aad-4d3c-8073-5a53ea0a1948 None admin] PATH_INFO: `/v3/projects/64c2a2f6f91d4b9c8d5993d28fd19631` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:36:24.086712 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ca031a9-6aad-4d3c-8073-5a53ea0a1948 None admin] RBAC: Authorizing `identity:get_project(project_id=64c2a2f6f91d4b9c8d5993d28fd19631)` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:36:24.088166 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ca031a9-6aad-4d3c-8073-5a53ea0a1948 None admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:36:24.089881 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 180/363] 10.222.0.20 () {56 vars in 1186 bytes} [Mon Aug 1 16:36:23 2022] GET /identity/v3/projects/64c2a2f6f91d4b9c8d5993d28fd19631 => generated 298 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:36:24.090665 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ca031a9-6aad-4d3c-8073-5a53ea0a1948 None admin] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:36:24.159827 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff4ca976-79fd-42df-afbc-4c6b1e32c170 None None] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:36:24.165336 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG dbcounter [-] [87057] Writing DB stats keystone:SELECT=100 {{(pid=87057) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:36:24.172263 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG dbcounter [-] [87058] Writing DB stats keystone:SELECT=100 {{(pid=87058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:36:24.215639 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe3ad1fe-068d-45f7-ba79-e20f98efb102 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:36:24.216512 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3ad1fe-068d-45f7-ba79-e20f98efb102 None admin] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:36:24.216789 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3ad1fe-068d-45f7-ba79-e20f98efb102 None admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:36:24.217062 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3ad1fe-068d-45f7-ba79-e20f98efb102 None admin] PATH_INFO: `/v3/roles` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:36:24.217779 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe3ad1fe-068d-45f7-ba79-e20f98efb102 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=87058) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:36:24.217874 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ca031a9-6aad-4d3c-8073-5a53ea0a1948 None admin] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:36:24.218120 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe3ad1fe-068d-45f7-ba79-e20f98efb102 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:36:24.219491 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe3ad1fe-068d-45f7-ba79-e20f98efb102 None admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:36:24.229057 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 184/364] 10.222.0.20 () {56 vars in 1111 bytes} [Mon Aug 1 16:36:24 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:36:24.233584 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e73bc508-9000-4f08-bc1b-f399ef1227a7 None None] REQUEST_METHOD: `POST` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:36:24.233584 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e73bc508-9000-4f08-bc1b-f399ef1227a7 None None] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:36:24.233584 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e73bc508-9000-4f08-bc1b-f399ef1227a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:36:24.252488 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.auth.core [None req-e73bc508-9000-4f08-bc1b-f399ef1227a7 None None] MFA Rules not processed for user `1007f8e379b74e169b98fb7c3c0ba5b4`. Rule list: `[]` (Enabled: `True`). {{(pid=87058) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:36:24.253669 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.fernet_utils [None req-e73bc508-9000-4f08-bc1b-f399ef1227a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:36:24.264381 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a51e5645-229e-4a31-8b4b-225557e11207 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:36:24.265049 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51e5645-229e-4a31-8b4b-225557e11207 None admin] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:36:24.265328 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51e5645-229e-4a31-8b4b-225557e11207 None admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:36:24.265601 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51e5645-229e-4a31-8b4b-225557e11207 None admin] PATH_INFO: `/v3/projects/d9a43129dd714d05bd409ffb0875f7e6` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:36:24.266949 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a51e5645-229e-4a31-8b4b-225557e11207 None admin] RBAC: Authorizing `identity:get_project(project_id=d9a43129dd714d05bd409ffb0875f7e6)` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:36:24.268775 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a51e5645-229e-4a31-8b4b-225557e11207 None admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:36:24.270966 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 181/365] 10.222.0.20 () {56 vars in 1186 bytes} [Mon Aug 1 16:36:24 2022] GET /identity/v3/projects/d9a43129dd714d05bd409ffb0875f7e6 => generated 302 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:36:24.271814 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a51e5645-229e-4a31-8b4b-225557e11207 None admin] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:36:24.348977 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 185/366] 10.222.0.20 () {58 vars in 1023 bytes} [Mon Aug 1 16:36:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 16:36:24.350167 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e73bc508-9000-4f08-bc1b-f399ef1227a7 None None] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:36:24.424412 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a51e5645-229e-4a31-8b4b-225557e11207 None admin] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:36:24.469446 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG dbcounter [-] [87057] Writing DB stats keystone:SELECT=100 {{(pid=87057) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:36:24.476645 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e892dfe-7ea3-4d4a-a5cd-c3cfe38a60ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:36:24.477871 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e892dfe-7ea3-4d4a-a5cd-c3cfe38a60ff None admin] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:36:24.478668 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e892dfe-7ea3-4d4a-a5cd-c3cfe38a60ff None admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:36:24.478943 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e892dfe-7ea3-4d4a-a5cd-c3cfe38a60ff None admin] PATH_INFO: `/v3/groups/64b4bd12c620422ca16e3edfbdab7fe7` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:36:24.479653 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e73bc508-9000-4f08-bc1b-f399ef1227a7 None None] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:36:24.480141 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e892dfe-7ea3-4d4a-a5cd-c3cfe38a60ff None admin] RBAC: Authorizing `identity:get_group(group_id=64b4bd12c620422ca16e3edfbdab7fe7)` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:36:24.481241 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG dbcounter [-] [87058] Writing DB stats keystone:SELECT=100 {{(pid=87058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:36:24.481869 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e892dfe-7ea3-4d4a-a5cd-c3cfe38a60ff None admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:36:24.484610 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 182/367] 10.222.0.20 () {56 vars in 1180 bytes} [Mon Aug 1 16:36:24 2022] GET /identity/v3/groups/64b4bd12c620422ca16e3edfbdab7fe7 => generated 226 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:36:24.485346 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e892dfe-7ea3-4d4a-a5cd-c3cfe38a60ff None admin] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:36:24.525673 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d90b6e9a-7f10-45be-a08e-db297dcc6e28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:36:24.526388 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90b6e9a-7f10-45be-a08e-db297dcc6e28 None admin] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:36:24.526661 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90b6e9a-7f10-45be-a08e-db297dcc6e28 None admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:36:24.526977 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90b6e9a-7f10-45be-a08e-db297dcc6e28 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:36:24.611554 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d90b6e9a-7f10-45be-a08e-db297dcc6e28 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:36:24.613060 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d90b6e9a-7f10-45be-a08e-db297dcc6e28 None admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:36:24.619916 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e892dfe-7ea3-4d4a-a5cd-c3cfe38a60ff None admin] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:36:24.670143 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c23077-2daa-4e51-84d4-bef73651c479 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:36:24.670769 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c23077-2daa-4e51-84d4-bef73651c479 None admin] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:36:24.671047 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c23077-2daa-4e51-84d4-bef73651c479 None admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:36:24.671319 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c23077-2daa-4e51-84d4-bef73651c479 None admin] PATH_INFO: `/v3/groups/64b4bd12c620422ca16e3edfbdab7fe7` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:36:24.672465 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27c23077-2daa-4e51-84d4-bef73651c479 None admin] RBAC: Authorizing `identity:get_group(group_id=64b4bd12c620422ca16e3edfbdab7fe7)` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:36:24.673901 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27c23077-2daa-4e51-84d4-bef73651c479 None admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:36:24.676354 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 183/368] 10.222.0.20 () {56 vars in 1180 bytes} [Mon Aug 1 16:36:24 2022] GET /identity/v3/groups/64b4bd12c620422ca16e3edfbdab7fe7 => generated 226 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:36:24.676354 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c23077-2daa-4e51-84d4-bef73651c479 None admin] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:36:24.750743 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG dbcounter [-] [87058] Writing DB stats keystone:SELECT=100 {{(pid=87058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:36:24.752171 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG dbcounter [-] [87057] Writing DB stats keystone:SELECT=100 {{(pid=87057) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:36:24.755388 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 186/369] 10.222.0.20 () {58 vars in 1282 bytes} [Mon Aug 1 16:36:24 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 405 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 01 16:36:24.756316 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d90b6e9a-7f10-45be-a08e-db297dcc6e28 None admin] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:36:24.800067 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c23077-2daa-4e51-84d4-bef73651c479 None admin] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:36:24.852322 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9d46942-b915-4d03-aeb3-a1f3a11880b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:36:24.852752 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d46942-b915-4d03-aeb3-a1f3a11880b8 None admin] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:36:24.853028 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d46942-b915-4d03-aeb3-a1f3a11880b8 None admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:36:24.853295 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d46942-b915-4d03-aeb3-a1f3a11880b8 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:36:24.866983 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9d46942-b915-4d03-aeb3-a1f3a11880b8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 16:36:24.873290 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application [None req-e9d46942-b915-4d03-aeb3-a1f3a11880b8 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 16:36:24.873290 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 16:36:24.873290 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 16:36:24.873290 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 16:36:24.873290 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 16:36:24.873290 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 16:36:24.873290 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 16:36:24.873290 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 16:36:24.873290 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 16:36:24.873290 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 16:36:24.873290 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 16:36:24.873290 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 16:36:24.873290 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 16:36:24.873290 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 16:36:24.873290 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 16:36:24.873290 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 16:36:24.874238 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 16:36:24.874238 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 16:36:24.874238 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 16:36:24.874238 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 16:36:24.874238 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 16:36:24.874238 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 16:36:24.874238 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 16:36:24.874238 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 16:36:24.874238 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 16:36:24.874238 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 16:36:24.874238 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 16:36:24.874238 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application with Lock( Aug 01 16:36:24.874238 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 16:36:24.874238 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self._enter() Aug 01 16:36:24.874238 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 16:36:24.874238 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 16:36:24.875075 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 16:36:24.875075 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self.creator() Aug 01 16:36:24.875075 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 16:36:24.875075 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application created_value = creator( Aug 01 16:36:24.875075 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 16:36:24.875075 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 16:36:24.875075 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 16:36:24.875075 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 16:36:24.875075 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 16:36:24.875075 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 16:36:24.875075 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 16:36:24.875075 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application Aug 01 16:36:24.875075 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 184/370] 10.222.0.20 () {56 vars in 1096 bytes} [Mon Aug 1 16:36:24 2022] GET /identity/v3/roles/admin => generated 83 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 16:36:24.875732 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9d46942-b915-4d03-aeb3-a1f3a11880b8 None admin] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:36:24.902492 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d90b6e9a-7f10-45be-a08e-db297dcc6e28 None admin] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:36:24.954449 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73be1bad-65b5-4977-8c04-9dd63844d111 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:36:24.955089 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73be1bad-65b5-4977-8c04-9dd63844d111 None admin] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:36:24.955768 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73be1bad-65b5-4977-8c04-9dd63844d111 None admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:36:24.956050 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73be1bad-65b5-4977-8c04-9dd63844d111 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:36:24.957145 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73be1bad-65b5-4977-8c04-9dd63844d111 None admin] RBAC: Adding query filter params (group.id=64b4bd12c620422ca16e3edfbdab7fe7, role.id=0999d3da63a545f18ff3252afdc90518, scope.project.id=64c2a2f6f91d4b9c8d5993d28fd19631) {{(pid=87058) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:36:24.957461 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73be1bad-65b5-4977-8c04-9dd63844d111 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:36:24.960091 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73be1bad-65b5-4977-8c04-9dd63844d111 None admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:36:24.968563 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 187/371] 10.222.0.20 () {56 vars in 1376 bytes} [Mon Aug 1 16:36:24 2022] GET /identity/v3/role_assignments?group.id=64b4bd12c620422ca16e3edfbdab7fe7&scope.project.id=64c2a2f6f91d4b9c8d5993d28fd19631&role.id=0999d3da63a545f18ff3252afdc90518 => generated 260 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:36:24.969793 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73be1bad-65b5-4977-8c04-9dd63844d111 None admin] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:36:25.010913 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9d46942-b915-4d03-aeb3-a1f3a11880b8 None admin] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:36:25.042413 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG dbcounter [-] [87057] Writing DB stats keystone:SELECT=100 {{(pid=87057) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:36:25.057145 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-489ae3d3-119b-4ecc-8bc0-e348f5248ff4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:36:25.058309 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489ae3d3-119b-4ecc-8bc0-e348f5248ff4 None admin] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:36:25.058741 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489ae3d3-119b-4ecc-8bc0-e348f5248ff4 None admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:36:25.059292 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG dbcounter [-] [87058] Writing DB stats keystone:SELECT=100 {{(pid=87058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:36:25.060192 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489ae3d3-119b-4ecc-8bc0-e348f5248ff4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:36:25.061162 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-489ae3d3-119b-4ecc-8bc0-e348f5248ff4 None admin] RBAC: Adding query filter params (group.id=64b4bd12c620422ca16e3edfbdab7fe7, role.id=0999d3da63a545f18ff3252afdc90518, scope.project.id=d9a43129dd714d05bd409ffb0875f7e6) {{(pid=87057) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:36:25.061625 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-489ae3d3-119b-4ecc-8bc0-e348f5248ff4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:36:25.063296 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-489ae3d3-119b-4ecc-8bc0-e348f5248ff4 None admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:36:25.071705 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 185/372] 10.222.0.20 () {56 vars in 1376 bytes} [Mon Aug 1 16:36:24 2022] GET /identity/v3/role_assignments?group.id=64b4bd12c620422ca16e3edfbdab7fe7&scope.project.id=d9a43129dd714d05bd409ffb0875f7e6&role.id=0999d3da63a545f18ff3252afdc90518 => generated 260 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:36:25.072865 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-489ae3d3-119b-4ecc-8bc0-e348f5248ff4 None admin] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:36:25.125872 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73be1bad-65b5-4977-8c04-9dd63844d111 None admin] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:36:25.187307 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-410a5c1a-bc8e-4299-91f4-e47d346486a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:36:25.188135 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410a5c1a-bc8e-4299-91f4-e47d346486a6 None admin] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:36:25.188512 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410a5c1a-bc8e-4299-91f4-e47d346486a6 None admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:36:25.188916 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410a5c1a-bc8e-4299-91f4-e47d346486a6 None admin] PATH_INFO: `/v3/projects/d9a43129dd714d05bd409ffb0875f7e6` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:36:25.190285 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-410a5c1a-bc8e-4299-91f4-e47d346486a6 None admin] RBAC: Authorizing `identity:get_project(project_id=d9a43129dd714d05bd409ffb0875f7e6)` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:36:25.192024 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-410a5c1a-bc8e-4299-91f4-e47d346486a6 None admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:36:25.193823 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 188/373] 10.222.0.20 () {56 vars in 1186 bytes} [Mon Aug 1 16:36:24 2022] GET /identity/v3/projects/d9a43129dd714d05bd409ffb0875f7e6 => generated 302 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:36:25.199652 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-410a5c1a-bc8e-4299-91f4-e47d346486a6 None admin] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:36:25.202780 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-489ae3d3-119b-4ecc-8bc0-e348f5248ff4 None admin] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:36:25.250077 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9604c413-fc06-4a52-a253-eeb9d3bfe5dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:36:25.251279 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9604c413-fc06-4a52-a253-eeb9d3bfe5dd None admin] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:36:25.252418 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9604c413-fc06-4a52-a253-eeb9d3bfe5dd None admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:36:25.253537 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9604c413-fc06-4a52-a253-eeb9d3bfe5dd None admin] PATH_INFO: `/v3/roles` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:36:25.254520 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9604c413-fc06-4a52-a253-eeb9d3bfe5dd None admin] RBAC: Adding query filter params (name=admin) {{(pid=87057) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:36:25.255156 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9604c413-fc06-4a52-a253-eeb9d3bfe5dd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:36:25.259773 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9604c413-fc06-4a52-a253-eeb9d3bfe5dd None admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:36:25.276959 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 186/374] 10.222.0.20 () {56 vars in 1099 bytes} [Mon Aug 1 16:36:25 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:36:25.282423 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9604c413-fc06-4a52-a253-eeb9d3bfe5dd None admin] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:36:25.350249 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG dbcounter [-] [87057] Writing DB stats keystone:SELECT=100 {{(pid=87057) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:36:25.367241 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-410a5c1a-bc8e-4299-91f4-e47d346486a6 None admin] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:36:25.406746 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG dbcounter [-] [87058] Writing DB stats keystone:SELECT=100 {{(pid=87058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:36:25.411521 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9604c413-fc06-4a52-a253-eeb9d3bfe5dd None admin] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:36:25.412726 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a9d320-3878-4729-be52-db2712b353a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:36:25.422177 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a9d320-3878-4729-be52-db2712b353a3 None admin] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:36:25.422177 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a9d320-3878-4729-be52-db2712b353a3 None admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:36:25.422177 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a9d320-3878-4729-be52-db2712b353a3 None admin] PATH_INFO: `/v3/groups/64b4bd12c620422ca16e3edfbdab7fe7` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:36:25.422177 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5a9d320-3878-4729-be52-db2712b353a3 None admin] RBAC: Authorizing `identity:get_group(group_id=64b4bd12c620422ca16e3edfbdab7fe7)` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:36:25.422177 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5a9d320-3878-4729-be52-db2712b353a3 None admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:36:25.422177 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 189/375] 10.222.0.20 () {56 vars in 1180 bytes} [Mon Aug 1 16:36:25 2022] GET /identity/v3/groups/64b4bd12c620422ca16e3edfbdab7fe7 => generated 226 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 16:36:25.426746 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a9d320-3878-4729-be52-db2712b353a3 None admin] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:36:25.455766 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14aec97c-1070-4974-889b-b8baae7f37fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:36:25.456536 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14aec97c-1070-4974-889b-b8baae7f37fb None admin] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:36:25.456866 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14aec97c-1070-4974-889b-b8baae7f37fb None admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:36:25.457141 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14aec97c-1070-4974-889b-b8baae7f37fb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:36:25.539821 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a9d320-3878-4729-be52-db2712b353a3 None admin] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:36:25.550201 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14aec97c-1070-4974-889b-b8baae7f37fb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:36:25.551776 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14aec97c-1070-4974-889b-b8baae7f37fb None admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:36:25.575934 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a6eeb7-7a92-4970-9033-79468535ba89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:36:25.576553 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a6eeb7-7a92-4970-9033-79468535ba89 None admin] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:36:25.576827 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a6eeb7-7a92-4970-9033-79468535ba89 None admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:36:25.577092 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a6eeb7-7a92-4970-9033-79468535ba89 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:36:25.577887 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25a6eeb7-7a92-4970-9033-79468535ba89 None admin] RBAC: Adding query filter params (group.id=64b4bd12c620422ca16e3edfbdab7fe7, role.id=3fac71a39f3f4ed198a3579064c30268, scope.project.id=d9a43129dd714d05bd409ffb0875f7e6) {{(pid=87058) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:36:25.578222 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25a6eeb7-7a92-4970-9033-79468535ba89 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:36:25.579649 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25a6eeb7-7a92-4970-9033-79468535ba89 None admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:36:25.585982 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 190/376] 10.222.0.20 () {56 vars in 1376 bytes} [Mon Aug 1 16:36:25 2022] GET /identity/v3/role_assignments?group.id=64b4bd12c620422ca16e3edfbdab7fe7&scope.project.id=d9a43129dd714d05bd409ffb0875f7e6&role.id=3fac71a39f3f4ed198a3579064c30268 => generated 260 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 16:36:25.587228 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e0356f-59d7-4d4f-aa1b-29897c207e6c None None] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:36:25.587508 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e0356f-59d7-4d4f-aa1b-29897c207e6c None None] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:36:25.594104 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e0356f-59d7-4d4f-aa1b-29897c207e6c None None] PATH_INFO: `/` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:36:25.595952 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 191/377] 10.222.0.20 () {54 vars in 926 bytes} [Mon Aug 1 16:36:25 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 16:36:25.601957 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b598e9b8-2993-4403-8a7e-fd3bd0e6dfd4 None None] REQUEST_METHOD: `POST` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:36:25.602267 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b598e9b8-2993-4403-8a7e-fd3bd0e6dfd4 None None] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:36:25.602517 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b598e9b8-2993-4403-8a7e-fd3bd0e6dfd4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:36:25.610257 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG dbcounter [-] [87057] Writing DB stats keystone:SELECT=100 {{(pid=87057) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:36:25.635207 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.auth.core [None req-b598e9b8-2993-4403-8a7e-fd3bd0e6dfd4 None None] MFA Rules not processed for user `1007f8e379b74e169b98fb7c3c0ba5b4`. Rule list: `[]` (Enabled: `True`). {{(pid=87058) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:36:25.636438 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.fernet_utils [None req-b598e9b8-2993-4403-8a7e-fd3bd0e6dfd4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:36:25.668475 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 187/378] 10.222.0.20 () {58 vars in 1282 bytes} [Mon Aug 1 16:36:25 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 387 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 01 16:36:25.674018 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14aec97c-1070-4974-889b-b8baae7f37fb None admin] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:36:25.711394 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG dbcounter [-] [87058] Writing DB stats keystone:SELECT=100 {{(pid=87058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:36:25.720253 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 192/379] 10.222.0.20 () {58 vars in 1023 bytes} [Mon Aug 1 16:36:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 01 16:36:25.749677 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b598e9b8-2993-4403-8a7e-fd3bd0e6dfd4 None None] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:36:25.867382 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b598e9b8-2993-4403-8a7e-fd3bd0e6dfd4 None None] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:36:25.904121 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f86dfe-7c69-4585-af75-c844609d8121 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:36:25.904635 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f86dfe-7c69-4585-af75-c844609d8121 None admin] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:36:25.904970 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f86dfe-7c69-4585-af75-c844609d8121 None admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:36:25.905243 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f86dfe-7c69-4585-af75-c844609d8121 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:36:25.911218 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14aec97c-1070-4974-889b-b8baae7f37fb None admin] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:36:25.917011 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21f86dfe-7c69-4585-af75-c844609d8121 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 16:36:25.923420 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application [None req-21f86dfe-7c69-4585-af75-c844609d8121 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 01 16:36:25.923420 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 16:36:25.923420 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 16:36:25.923420 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 16:36:25.923420 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 16:36:25.923420 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 16:36:25.923420 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 16:36:25.923420 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 16:36:25.923420 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 16:36:25.923420 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 16:36:25.923420 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 16:36:25.923420 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 16:36:25.923420 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 16:36:25.923420 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 16:36:25.923420 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 16:36:25.923420 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 16:36:25.924315 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 16:36:25.924315 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 16:36:25.924315 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 16:36:25.924315 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 16:36:25.924315 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 16:36:25.924315 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 16:36:25.924315 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 16:36:25.924315 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 16:36:25.924315 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 16:36:25.924315 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 16:36:25.924315 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 16:36:25.924315 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application with Lock( Aug 01 16:36:25.924315 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 16:36:25.924315 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self._enter() Aug 01 16:36:25.924315 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 16:36:25.924315 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 16:36:25.925133 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 16:36:25.925133 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self.creator() Aug 01 16:36:25.925133 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 16:36:25.925133 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application created_value = creator( Aug 01 16:36:25.925133 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 16:36:25.925133 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 16:36:25.925133 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 16:36:25.925133 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 16:36:25.925133 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 16:36:25.925133 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 16:36:25.925133 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 01 16:36:25.925133 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application Aug 01 16:36:25.925133 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 193/380] 10.222.0.20 () {56 vars in 1114 bytes} [Mon Aug 1 16:36:25 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 16:36:25.934770 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f86dfe-7c69-4585-af75-c844609d8121 None admin] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:36:25.961330 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-341f679b-cd61-40f0-ba3d-a164a8c0d18f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:36:25.962445 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341f679b-cd61-40f0-ba3d-a164a8c0d18f None admin] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:36:25.962978 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341f679b-cd61-40f0-ba3d-a164a8c0d18f None admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:36:25.963450 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341f679b-cd61-40f0-ba3d-a164a8c0d18f None admin] PATH_INFO: `/v3/projects/3e43f22f36744d079b60704aff4969d4` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:36:25.964848 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-341f679b-cd61-40f0-ba3d-a164a8c0d18f None admin] RBAC: Authorizing `identity:get_project(project_id=3e43f22f36744d079b60704aff4969d4)` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:36:25.974936 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-341f679b-cd61-40f0-ba3d-a164a8c0d18f None admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:36:25.978058 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 188/381] 10.222.0.20 () {56 vars in 1186 bytes} [Mon Aug 1 16:36:25 2022] GET /identity/v3/projects/3e43f22f36744d079b60704aff4969d4 => generated 344 bytes in 304 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 16:36:25.983937 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-341f679b-cd61-40f0-ba3d-a164a8c0d18f None admin] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:36:26.012146 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG dbcounter [-] [87058] Writing DB stats keystone:SELECT=100 {{(pid=87058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:36:26.027711 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG dbcounter [-] [87057] Writing DB stats keystone:SELECT=100 {{(pid=87057) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:36:26.066804 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f86dfe-7c69-4585-af75-c844609d8121 None admin] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:36:26.108820 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c1b223b-fd52-454d-a38e-c7ee6d01f083 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:36:26.109843 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1b223b-fd52-454d-a38e-c7ee6d01f083 None admin] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:36:26.110307 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1b223b-fd52-454d-a38e-c7ee6d01f083 None admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:36:26.110781 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1b223b-fd52-454d-a38e-c7ee6d01f083 None admin] PATH_INFO: `/v3/roles` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:36:26.111818 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c1b223b-fd52-454d-a38e-c7ee6d01f083 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=87058) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:36:26.113289 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c1b223b-fd52-454d-a38e-c7ee6d01f083 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:36:26.115331 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c1b223b-fd52-454d-a38e-c7ee6d01f083 None admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:36:26.125506 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 194/382] 10.222.0.20 () {56 vars in 1111 bytes} [Mon Aug 1 16:36:25 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:36:26.131216 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c1b223b-fd52-454d-a38e-c7ee6d01f083 None admin] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:36:26.194016 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-341f679b-cd61-40f0-ba3d-a164a8c0d18f None admin] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:36:26.238658 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-215c28b7-9e20-43a8-9c9d-e9c9dfc75b99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:36:26.239539 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215c28b7-9e20-43a8-9c9d-e9c9dfc75b99 None admin] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:36:26.239920 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215c28b7-9e20-43a8-9c9d-e9c9dfc75b99 None admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:36:26.240471 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215c28b7-9e20-43a8-9c9d-e9c9dfc75b99 None admin] PATH_INFO: `/v3/groups/fb6971a50bbf408a8817d213299e6a64` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:36:26.254668 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-215c28b7-9e20-43a8-9c9d-e9c9dfc75b99 None admin] RBAC: Authorizing `identity:get_group(group_id=fb6971a50bbf408a8817d213299e6a64)` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:36:26.255040 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c1b223b-fd52-454d-a38e-c7ee6d01f083 None admin] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:36:26.256516 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-215c28b7-9e20-43a8-9c9d-e9c9dfc75b99 None admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:36:26.259087 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 189/383] 10.222.0.20 () {56 vars in 1180 bytes} [Mon Aug 1 16:36:25 2022] GET /identity/v3/groups/fb6971a50bbf408a8817d213299e6a64 => generated 229 bytes in 275 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:36:26.265988 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-215c28b7-9e20-43a8-9c9d-e9c9dfc75b99 None admin] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:36:26.274849 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG dbcounter [-] [87058] Writing DB stats keystone:SELECT=100 {{(pid=87058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:36:26.294720 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da1854be-1f5a-4c2b-b7e6-8049d9c1ac94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:36:26.295593 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1854be-1f5a-4c2b-b7e6-8049d9c1ac94 None admin] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:36:26.296437 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1854be-1f5a-4c2b-b7e6-8049d9c1ac94 None admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:36:26.296899 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1854be-1f5a-4c2b-b7e6-8049d9c1ac94 None admin] PATH_INFO: `/v3/groups/64b4bd12c620422ca16e3edfbdab7fe7` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:36:26.298515 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da1854be-1f5a-4c2b-b7e6-8049d9c1ac94 None admin] RBAC: Authorizing `identity:get_group(group_id=64b4bd12c620422ca16e3edfbdab7fe7)` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:36:26.300256 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da1854be-1f5a-4c2b-b7e6-8049d9c1ac94 None admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:36:26.302652 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 195/384] 10.222.0.20 () {56 vars in 1180 bytes} [Mon Aug 1 16:36:26 2022] GET /identity/v3/groups/64b4bd12c620422ca16e3edfbdab7fe7 => generated 226 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 16:36:26.309421 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da1854be-1f5a-4c2b-b7e6-8049d9c1ac94 None admin] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:36:26.428306 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da1854be-1f5a-4c2b-b7e6-8049d9c1ac94 None admin] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:36:26.470093 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab955bcb-3a9d-4f1a-9a26-2e5c8903de48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:36:26.470394 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG dbcounter [-] [87057] Writing DB stats keystone:SELECT=100 {{(pid=87057) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:36:26.471239 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab955bcb-3a9d-4f1a-9a26-2e5c8903de48 None admin] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:36:26.471738 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab955bcb-3a9d-4f1a-9a26-2e5c8903de48 None admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:36:26.472210 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab955bcb-3a9d-4f1a-9a26-2e5c8903de48 None admin] PATH_INFO: `/v3/projects/64c2a2f6f91d4b9c8d5993d28fd19631` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:36:26.473651 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab955bcb-3a9d-4f1a-9a26-2e5c8903de48 None admin] RBAC: Authorizing `identity:get_project(project_id=64c2a2f6f91d4b9c8d5993d28fd19631)` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:36:26.475636 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab955bcb-3a9d-4f1a-9a26-2e5c8903de48 None admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:36:26.478562 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 196/385] 10.222.0.20 () {56 vars in 1186 bytes} [Mon Aug 1 16:36:26 2022] GET /identity/v3/projects/64c2a2f6f91d4b9c8d5993d28fd19631 => generated 298 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 16:36:26.484532 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab955bcb-3a9d-4f1a-9a26-2e5c8903de48 None admin] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:36:26.516945 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-215c28b7-9e20-43a8-9c9d-e9c9dfc75b99 None admin] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:36:26.554851 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG dbcounter [-] [87058] Writing DB stats keystone:SELECT=100 {{(pid=87058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:36:26.606883 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed2d1859-fac9-4f56-a869-06ca96954102 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:36:26.607550 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2d1859-fac9-4f56-a869-06ca96954102 None admin] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:36:26.607755 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2d1859-fac9-4f56-a869-06ca96954102 None admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:36:26.607993 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2d1859-fac9-4f56-a869-06ca96954102 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:36:26.609049 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed2d1859-fac9-4f56-a869-06ca96954102 None admin] RBAC: Adding query filter params (group.id=fb6971a50bbf408a8817d213299e6a64, role.id=440bb943dd5b46fea038b903b6eaaf81, scope.project.id=3e43f22f36744d079b60704aff4969d4) {{(pid=87057) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:36:26.609157 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed2d1859-fac9-4f56-a869-06ca96954102 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:36:26.612208 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab955bcb-3a9d-4f1a-9a26-2e5c8903de48 None admin] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:36:26.618451 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed2d1859-fac9-4f56-a869-06ca96954102 None admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:36:26.640112 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 190/386] 10.222.0.20 () {56 vars in 1376 bytes} [Mon Aug 1 16:36:26 2022] GET /identity/v3/role_assignments?group.id=fb6971a50bbf408a8817d213299e6a64&scope.project.id=3e43f22f36744d079b60704aff4969d4&role.id=440bb943dd5b46fea038b903b6eaaf81 => generated 260 bytes in 376 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 16:36:26.652954 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b1e9cbd-157c-4fdc-8cf4-9a56b7aa7352 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:36:26.654119 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1e9cbd-157c-4fdc-8cf4-9a56b7aa7352 None admin] REQUEST_METHOD: `PUT` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:36:26.654602 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1e9cbd-157c-4fdc-8cf4-9a56b7aa7352 None admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:36:26.655097 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1e9cbd-157c-4fdc-8cf4-9a56b7aa7352 None admin] PATH_INFO: `/v3/projects/64c2a2f6f91d4b9c8d5993d28fd19631/groups/64b4bd12c620422ca16e3edfbdab7fe7/roles/3fac71a39f3f4ed198a3579064c30268` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:36:26.657600 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b1e9cbd-157c-4fdc-8cf4-9a56b7aa7352 None admin] RBAC: Authorizing `identity:create_grant(project_id=64c2a2f6f91d4b9c8d5993d28fd19631, group_id=64b4bd12c620422ca16e3edfbdab7fe7, role_id=3fac71a39f3f4ed198a3579064c30268)` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:36:26.659348 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b1e9cbd-157c-4fdc-8cf4-9a56b7aa7352 None admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:36:26.733030 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 197/387] 10.222.0.20 () {58 vars in 1442 bytes} [Mon Aug 1 16:36:26 2022] PUT /identity/v3/projects/64c2a2f6f91d4b9c8d5993d28fd19631/groups/64b4bd12c620422ca16e3edfbdab7fe7/roles/3fac71a39f3f4ed198a3579064c30268 => generated 0 bytes in 249 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 16:36:27.513504 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1112b3bd-3a5a-4fb0-b9a2-ccf8574ee8e1 None None] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:36:27.513886 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1112b3bd-3a5a-4fb0-b9a2-ccf8574ee8e1 None None] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:36:27.514087 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1112b3bd-3a5a-4fb0-b9a2-ccf8574ee8e1 None None] PATH_INFO: `/` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:36:27.515790 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 191/388] 10.222.0.20 () {54 vars in 926 bytes} [Mon Aug 1 16:36:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 16:36:27.521307 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da298c8-427c-457a-a432-3a4985cfa922 None None] REQUEST_METHOD: `POST` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:36:27.521600 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da298c8-427c-457a-a432-3a4985cfa922 None None] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:36:27.521876 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da298c8-427c-457a-a432-3a4985cfa922 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:36:27.547243 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.auth.core [None req-8da298c8-427c-457a-a432-3a4985cfa922 None None] MFA Rules not processed for user `1007f8e379b74e169b98fb7c3c0ba5b4`. Rule list: `[]` (Enabled: `True`). {{(pid=87058) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:36:27.548628 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.fernet_utils [None req-8da298c8-427c-457a-a432-3a4985cfa922 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:36:27.650557 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 198/389] 10.222.0.20 () {58 vars in 1023 bytes} [Mon Aug 1 16:36:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 16:36:27.693529 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1112b3bd-3a5a-4fb0-b9a2-ccf8574ee8e1 None None] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:36:27.799771 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd73d71f-06b5-43db-8a6c-8e23a495f156 None None] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:36:27.799951 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd73d71f-06b5-43db-8a6c-8e23a495f156 None None] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:36:27.800235 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd73d71f-06b5-43db-8a6c-8e23a495f156 None None] PATH_INFO: `/` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:36:27.801729 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 199/390] 10.222.0.20 () {54 vars in 926 bytes} [Mon Aug 1 16:36:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 16:36:27.815401 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f690859-da99-4660-a988-5091fdacb806 None None] REQUEST_METHOD: `POST` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:36:27.815690 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f690859-da99-4660-a988-5091fdacb806 None None] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:36:27.815948 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f690859-da99-4660-a988-5091fdacb806 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:36:27.823578 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1112b3bd-3a5a-4fb0-b9a2-ccf8574ee8e1 None None] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:36:27.835888 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.auth.core [None req-8f690859-da99-4660-a988-5091fdacb806 None None] MFA Rules not processed for user `1007f8e379b74e169b98fb7c3c0ba5b4`. Rule list: `[]` (Enabled: `True`). {{(pid=87058) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:36:27.837950 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.fernet_utils [None req-8f690859-da99-4660-a988-5091fdacb806 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:36:27.870078 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1866936d-fe77-41d7-86ea-fa9986d0c8e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:36:27.871084 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1866936d-fe77-41d7-86ea-fa9986d0c8e7 None admin] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:36:27.871399 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1866936d-fe77-41d7-86ea-fa9986d0c8e7 None admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:36:27.873054 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1866936d-fe77-41d7-86ea-fa9986d0c8e7 None admin] PATH_INFO: `/v3/roles/member` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:36:27.889020 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG dbcounter [-] [87058] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:36:27.904186 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1866936d-fe77-41d7-86ea-fa9986d0c8e7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 16:36:27.926635 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application [None req-1866936d-fe77-41d7-86ea-fa9986d0c8e7 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 01 16:36:27.926635 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 16:36:27.926635 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 16:36:27.926635 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 16:36:27.926635 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 16:36:27.926635 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 16:36:27.926635 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 16:36:27.926635 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 16:36:27.926635 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 16:36:27.926635 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 16:36:27.926635 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 16:36:27.926635 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 16:36:27.926635 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 16:36:27.926635 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 16:36:27.926635 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 16:36:27.926635 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 16:36:27.927568 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 16:36:27.927568 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 16:36:27.927568 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 16:36:27.927568 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 16:36:27.927568 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 16:36:27.927568 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 16:36:27.927568 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 16:36:27.927568 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 16:36:27.927568 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 16:36:27.927568 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 16:36:27.927568 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 16:36:27.927568 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application with Lock( Aug 01 16:36:27.927568 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 16:36:27.927568 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self._enter() Aug 01 16:36:27.927568 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 16:36:27.927568 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 16:36:27.928410 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 16:36:27.928410 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self.creator() Aug 01 16:36:27.928410 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 16:36:27.928410 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application created_value = creator( Aug 01 16:36:27.928410 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 16:36:27.928410 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 16:36:27.928410 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 16:36:27.928410 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 16:36:27.928410 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 16:36:27.928410 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 16:36:27.928410 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 01 16:36:27.928410 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application Aug 01 16:36:27.928410 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 192/391] 10.222.0.20 () {56 vars in 1099 bytes} [Mon Aug 1 16:36:27 2022] GET /identity/v3/roles/member => generated 84 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 16:36:27.928410 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1866936d-fe77-41d7-86ea-fa9986d0c8e7 None admin] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:36:27.931910 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG dbcounter [-] [87057] Writing DB stats keystone:SELECT=100 {{(pid=87057) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:36:27.971063 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 200/392] 10.222.0.20 () {58 vars in 1023 bytes} [Mon Aug 1 16:36:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 16:36:27.998298 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f690859-da99-4660-a988-5091fdacb806 None None] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:36:28.097524 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1866936d-fe77-41d7-86ea-fa9986d0c8e7 None admin] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:36:28.168579 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b2c57f1-b883-4e9d-ab6b-0852ca33f888 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:36:28.169226 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2c57f1-b883-4e9d-ab6b-0852ca33f888 None admin] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:36:28.169506 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2c57f1-b883-4e9d-ab6b-0852ca33f888 None admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:36:28.169781 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2c57f1-b883-4e9d-ab6b-0852ca33f888 None admin] PATH_INFO: `/v3/roles` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:36:28.170603 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b2c57f1-b883-4e9d-ab6b-0852ca33f888 None admin] RBAC: Adding query filter params (name=member) {{(pid=87057) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:36:28.170912 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b2c57f1-b883-4e9d-ab6b-0852ca33f888 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:36:28.172500 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b2c57f1-b883-4e9d-ab6b-0852ca33f888 None admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:36:28.187812 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 193/393] 10.222.0.20 () {56 vars in 1101 bytes} [Mon Aug 1 16:36:27 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:36:28.189349 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b2c57f1-b883-4e9d-ab6b-0852ca33f888 None admin] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:36:28.209529 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f690859-da99-4660-a988-5091fdacb806 None None] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:36:28.246571 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96de0b3d-d65c-4352-933a-404ee785a496 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:36:28.247227 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96de0b3d-d65c-4352-933a-404ee785a496 None admin] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:36:28.247671 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96de0b3d-d65c-4352-933a-404ee785a496 None admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:36:28.248153 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96de0b3d-d65c-4352-933a-404ee785a496 None admin] PATH_INFO: `/v3/roles/member` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:36:28.259544 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96de0b3d-d65c-4352-933a-404ee785a496 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 16:36:28.264474 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application [None req-96de0b3d-d65c-4352-933a-404ee785a496 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 01 16:36:28.264474 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 16:36:28.264474 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 16:36:28.264474 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 16:36:28.264474 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 16:36:28.264474 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 16:36:28.264474 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 16:36:28.264474 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 16:36:28.264474 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 16:36:28.264474 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 16:36:28.264474 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 16:36:28.264474 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 16:36:28.264474 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 16:36:28.264474 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 16:36:28.264474 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 16:36:28.264474 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 16:36:28.265392 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 16:36:28.265392 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 16:36:28.265392 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 16:36:28.265392 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 16:36:28.265392 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 16:36:28.265392 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 16:36:28.265392 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 16:36:28.265392 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 16:36:28.265392 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 16:36:28.265392 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 16:36:28.265392 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 16:36:28.265392 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application with Lock( Aug 01 16:36:28.265392 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 16:36:28.265392 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self._enter() Aug 01 16:36:28.265392 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 16:36:28.265392 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 16:36:28.266623 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 16:36:28.266623 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self.creator() Aug 01 16:36:28.266623 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 16:36:28.266623 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application created_value = creator( Aug 01 16:36:28.266623 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 16:36:28.266623 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 16:36:28.266623 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 16:36:28.266623 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 16:36:28.266623 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 16:36:28.266623 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 16:36:28.266623 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 01 16:36:28.266623 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application Aug 01 16:36:28.266623 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 201/394] 10.222.0.20 () {56 vars in 1099 bytes} [Mon Aug 1 16:36:27 2022] GET /identity/v3/roles/member => generated 84 bytes in 268 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 16:36:28.267658 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30be3a7-be61-4386-bba8-38deefa40602 None None] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:36:28.267935 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30be3a7-be61-4386-bba8-38deefa40602 None None] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:36:28.268125 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30be3a7-be61-4386-bba8-38deefa40602 None None] PATH_INFO: `/` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:36:28.268412 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG dbcounter [-] [87057] Writing DB stats keystone:SELECT=100 {{(pid=87057) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:36:28.269676 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 202/395] 10.222.0.20 () {54 vars in 926 bytes} [Mon Aug 1 16:36:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 16:36:28.274921 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a30be3a7-be61-4386-bba8-38deefa40602 None None] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:36:28.307673 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b2c57f1-b883-4e9d-ab6b-0852ca33f888 None admin] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:36:28.313037 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG dbcounter [-] [87058] Writing DB stats keystone:SELECT=100 {{(pid=87058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:36:28.360242 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ac96ed-6a54-4f02-93b7-edc289bea6f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:36:28.361232 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ac96ed-6a54-4f02-93b7-edc289bea6f6 None admin] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:36:28.361737 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ac96ed-6a54-4f02-93b7-edc289bea6f6 None admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:36:28.362243 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ac96ed-6a54-4f02-93b7-edc289bea6f6 None admin] PATH_INFO: `/v3/groups/64b4bd12c620422ca16e3edfbdab7fe7` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:36:28.364049 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4ac96ed-6a54-4f02-93b7-edc289bea6f6 None admin] RBAC: Authorizing `identity:get_group(group_id=64b4bd12c620422ca16e3edfbdab7fe7)` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:36:28.366108 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4ac96ed-6a54-4f02-93b7-edc289bea6f6 None admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:36:28.374138 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 194/396] 10.222.0.20 () {56 vars in 1180 bytes} [Mon Aug 1 16:36:28 2022] GET /identity/v3/groups/64b4bd12c620422ca16e3edfbdab7fe7 => generated 226 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:36:28.376184 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39713ce7-c965-42a5-ad74-cd0b39ca3349 None None] REQUEST_METHOD: `POST` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:36:28.376738 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39713ce7-c965-42a5-ad74-cd0b39ca3349 None None] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:36:28.377190 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39713ce7-c965-42a5-ad74-cd0b39ca3349 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:36:28.396102 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a30be3a7-be61-4386-bba8-38deefa40602 None None] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:36:28.400465 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.auth.core [None req-39713ce7-c965-42a5-ad74-cd0b39ca3349 None None] MFA Rules not processed for user `1007f8e379b74e169b98fb7c3c0ba5b4`. Rule list: `[]` (Enabled: `True`). {{(pid=87057) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:36:28.401841 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.fernet_utils [None req-39713ce7-c965-42a5-ad74-cd0b39ca3349 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87057) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:36:28.433621 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b510aff-2bd2-44ec-a538-8cbb12e15feb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:36:28.434452 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b510aff-2bd2-44ec-a538-8cbb12e15feb None admin] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:36:28.434808 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b510aff-2bd2-44ec-a538-8cbb12e15feb None admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:36:28.435178 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b510aff-2bd2-44ec-a538-8cbb12e15feb None admin] PATH_INFO: `/v3/roles` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:36:28.436025 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b510aff-2bd2-44ec-a538-8cbb12e15feb None admin] RBAC: Adding query filter params (name=member) {{(pid=87058) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:36:28.436396 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b510aff-2bd2-44ec-a538-8cbb12e15feb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:36:28.438183 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b510aff-2bd2-44ec-a538-8cbb12e15feb None admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:36:28.452735 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 203/397] 10.222.0.20 () {56 vars in 1101 bytes} [Mon Aug 1 16:36:28 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:36:28.453934 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b510aff-2bd2-44ec-a538-8cbb12e15feb None admin] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:36:28.498152 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 195/398] 10.222.0.20 () {58 vars in 1023 bytes} [Mon Aug 1 16:36:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 16:36:28.498152 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39713ce7-c965-42a5-ad74-cd0b39ca3349 None None] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:36:28.575490 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG dbcounter [-] [87058] Writing DB stats keystone:SELECT=100 {{(pid=87058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:36:28.583332 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b510aff-2bd2-44ec-a538-8cbb12e15feb None admin] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:36:28.617592 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG dbcounter [-] [87057] Writing DB stats keystone:SELECT=100 {{(pid=87057) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:36:28.628052 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c4687ff-bc6e-451c-8f22-9cffadc90802 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:36:28.640618 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4687ff-bc6e-451c-8f22-9cffadc90802 None admin] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:36:28.640618 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4687ff-bc6e-451c-8f22-9cffadc90802 None admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:36:28.640618 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4687ff-bc6e-451c-8f22-9cffadc90802 None admin] PATH_INFO: `/v3/projects/d9a43129dd714d05bd409ffb0875f7e6` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:36:28.640618 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c4687ff-bc6e-451c-8f22-9cffadc90802 None admin] RBAC: Authorizing `identity:get_project(project_id=d9a43129dd714d05bd409ffb0875f7e6)` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:36:28.640618 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c4687ff-bc6e-451c-8f22-9cffadc90802 None admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:36:28.641027 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39713ce7-c965-42a5-ad74-cd0b39ca3349 None None] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:36:28.641118 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 204/399] 10.222.0.20 () {56 vars in 1186 bytes} [Mon Aug 1 16:36:28 2022] GET /identity/v3/projects/d9a43129dd714d05bd409ffb0875f7e6 => generated 302 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:36:28.641118 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c4687ff-bc6e-451c-8f22-9cffadc90802 None admin] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:36:28.679832 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6975082c-b2c7-4be1-acca-053222155195 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:36:28.680858 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6975082c-b2c7-4be1-acca-053222155195 None admin] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:36:28.681338 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6975082c-b2c7-4be1-acca-053222155195 None admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:36:28.681950 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6975082c-b2c7-4be1-acca-053222155195 None admin] PATH_INFO: `/v3/groups/64b4bd12c620422ca16e3edfbdab7fe7` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:36:28.683805 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6975082c-b2c7-4be1-acca-053222155195 None admin] RBAC: Authorizing `identity:get_group(group_id=64b4bd12c620422ca16e3edfbdab7fe7)` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:36:28.685451 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6975082c-b2c7-4be1-acca-053222155195 None admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:36:28.687455 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 196/400] 10.222.0.20 () {56 vars in 1180 bytes} [Mon Aug 1 16:36:28 2022] GET /identity/v3/groups/64b4bd12c620422ca16e3edfbdab7fe7 => generated 226 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:36:28.693025 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6975082c-b2c7-4be1-acca-053222155195 None admin] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:36:28.774998 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c4687ff-bc6e-451c-8f22-9cffadc90802 None admin] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:36:28.808363 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6975082c-b2c7-4be1-acca-053222155195 None admin] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:36:28.821399 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e833744b-9117-4c6d-a0c8-754ad003a3a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:36:28.822245 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e833744b-9117-4c6d-a0c8-754ad003a3a3 None admin] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:36:28.822629 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e833744b-9117-4c6d-a0c8-754ad003a3a3 None admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:36:28.822911 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e833744b-9117-4c6d-a0c8-754ad003a3a3 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:36:28.837630 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e833744b-9117-4c6d-a0c8-754ad003a3a3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 16:36:28.845393 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application [None req-e833744b-9117-4c6d-a0c8-754ad003a3a3 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 01 16:36:28.845393 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 16:36:28.845393 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 16:36:28.845393 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 16:36:28.845393 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 16:36:28.845393 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 16:36:28.845393 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 16:36:28.845393 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 16:36:28.845393 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 16:36:28.845393 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 16:36:28.845393 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 16:36:28.845393 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 16:36:28.845393 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 16:36:28.845393 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 16:36:28.845393 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 16:36:28.845393 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 16:36:28.846335 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 16:36:28.846335 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 16:36:28.846335 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 16:36:28.846335 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 16:36:28.846335 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 16:36:28.846335 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 16:36:28.846335 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 16:36:28.846335 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 16:36:28.846335 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 16:36:28.846335 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 16:36:28.846335 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 16:36:28.846335 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application with Lock( Aug 01 16:36:28.846335 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 16:36:28.846335 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self._enter() Aug 01 16:36:28.846335 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 16:36:28.846335 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 16:36:28.847148 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 16:36:28.847148 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self.creator() Aug 01 16:36:28.847148 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 16:36:28.847148 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application created_value = creator( Aug 01 16:36:28.847148 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 16:36:28.847148 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 16:36:28.847148 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 16:36:28.847148 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 16:36:28.847148 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 16:36:28.847148 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 16:36:28.847148 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 01 16:36:28.847148 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application Aug 01 16:36:28.850110 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 205/401] 10.222.0.20 () {56 vars in 1114 bytes} [Mon Aug 1 16:36:28 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 16:36:28.851704 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e833744b-9117-4c6d-a0c8-754ad003a3a3 None admin] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:36:28.854798 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cabfe5e2-6a8c-4251-b0f7-bd9b3e5dda24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:36:28.855736 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cabfe5e2-6a8c-4251-b0f7-bd9b3e5dda24 None admin] REQUEST_METHOD: `PUT` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:36:28.856213 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cabfe5e2-6a8c-4251-b0f7-bd9b3e5dda24 None admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:36:28.856735 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cabfe5e2-6a8c-4251-b0f7-bd9b3e5dda24 None admin] PATH_INFO: `/v3/projects/d9a43129dd714d05bd409ffb0875f7e6/groups/64b4bd12c620422ca16e3edfbdab7fe7/roles/0999d3da63a545f18ff3252afdc90518` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:36:28.858971 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cabfe5e2-6a8c-4251-b0f7-bd9b3e5dda24 None admin] RBAC: Authorizing `identity:create_grant(project_id=d9a43129dd714d05bd409ffb0875f7e6, group_id=64b4bd12c620422ca16e3edfbdab7fe7, role_id=0999d3da63a545f18ff3252afdc90518)` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:36:28.860720 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cabfe5e2-6a8c-4251-b0f7-bd9b3e5dda24 None admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:36:28.863610 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG dbcounter [-] [87057] Writing DB stats keystone:SELECT=100 {{(pid=87057) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:36:28.883175 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG dbcounter [-] [87058] Writing DB stats keystone:SELECT=100 {{(pid=87058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:36:28.966339 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e833744b-9117-4c6d-a0c8-754ad003a3a3 None admin] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:36:28.983230 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 197/402] 10.222.0.20 () {58 vars in 1442 bytes} [Mon Aug 1 16:36:28 2022] PUT /identity/v3/projects/d9a43129dd714d05bd409ffb0875f7e6/groups/64b4bd12c620422ca16e3edfbdab7fe7/roles/0999d3da63a545f18ff3252afdc90518 => generated 0 bytes in 295 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 16:36:28.987657 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cabfe5e2-6a8c-4251-b0f7-bd9b3e5dda24 None admin] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:36:29.032421 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04ad6aa3-8984-4d18-a997-4b201adaef09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:36:29.033675 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ad6aa3-8984-4d18-a997-4b201adaef09 None admin] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:36:29.035474 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ad6aa3-8984-4d18-a997-4b201adaef09 None admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:36:29.036540 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ad6aa3-8984-4d18-a997-4b201adaef09 None admin] PATH_INFO: `/v3/projects/64c2a2f6f91d4b9c8d5993d28fd19631` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:36:29.041015 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04ad6aa3-8984-4d18-a997-4b201adaef09 None admin] RBAC: Authorizing `identity:get_project(project_id=64c2a2f6f91d4b9c8d5993d28fd19631)` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:36:29.043934 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04ad6aa3-8984-4d18-a997-4b201adaef09 None admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:36:29.050625 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 206/403] 10.222.0.20 () {56 vars in 1186 bytes} [Mon Aug 1 16:36:28 2022] GET /identity/v3/projects/64c2a2f6f91d4b9c8d5993d28fd19631 => generated 298 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:36:29.056059 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04ad6aa3-8984-4d18-a997-4b201adaef09 None admin] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:36:29.161196 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cabfe5e2-6a8c-4251-b0f7-bd9b3e5dda24 None admin] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:36:29.184136 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG dbcounter [-] [87058] Writing DB stats keystone:SELECT=100 {{(pid=87058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:36:29.195751 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04ad6aa3-8984-4d18-a997-4b201adaef09 None admin] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:36:29.198956 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6591d6f-aa80-4a7d-a901-bc0d93ee7b17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:36:29.199574 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6591d6f-aa80-4a7d-a901-bc0d93ee7b17 None admin] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:36:29.199857 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6591d6f-aa80-4a7d-a901-bc0d93ee7b17 None admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:36:29.200120 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6591d6f-aa80-4a7d-a901-bc0d93ee7b17 None admin] PATH_INFO: `/v3/roles` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:36:29.200833 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6591d6f-aa80-4a7d-a901-bc0d93ee7b17 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=87057) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:36:29.201135 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6591d6f-aa80-4a7d-a901-bc0d93ee7b17 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:36:29.202546 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6591d6f-aa80-4a7d-a901-bc0d93ee7b17 None admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:36:29.212574 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 198/404] 10.222.0.20 () {56 vars in 1111 bytes} [Mon Aug 1 16:36:28 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:36:29.213868 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37dc119-8065-4a5f-a1da-ef0bec3d203a None None] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:36:29.214207 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37dc119-8065-4a5f-a1da-ef0bec3d203a None None] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:36:29.214493 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37dc119-8065-4a5f-a1da-ef0bec3d203a None None] PATH_INFO: `/` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:36:29.215723 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 199/405] 10.222.0.20 () {54 vars in 926 bytes} [Mon Aug 1 16:36:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 16:36:29.219629 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a37dc119-8065-4a5f-a1da-ef0bec3d203a None None] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:36:29.252106 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74354de2-8545-436a-8f2c-9ef4b485d20b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:36:29.253202 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74354de2-8545-436a-8f2c-9ef4b485d20b None admin] REQUEST_METHOD: `PUT` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:36:29.253625 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74354de2-8545-436a-8f2c-9ef4b485d20b None admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:36:29.254398 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74354de2-8545-436a-8f2c-9ef4b485d20b None admin] PATH_INFO: `/v3/projects/64c2a2f6f91d4b9c8d5993d28fd19631/groups/64b4bd12c620422ca16e3edfbdab7fe7/roles/0999d3da63a545f18ff3252afdc90518` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:36:29.256962 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74354de2-8545-436a-8f2c-9ef4b485d20b None admin] RBAC: Authorizing `identity:create_grant(project_id=64c2a2f6f91d4b9c8d5993d28fd19631, group_id=64b4bd12c620422ca16e3edfbdab7fe7, role_id=0999d3da63a545f18ff3252afdc90518)` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:36:29.260820 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74354de2-8545-436a-8f2c-9ef4b485d20b None admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:36:29.282929 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG dbcounter [-] [87057] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=87057) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:36:29.329804 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a37dc119-8065-4a5f-a1da-ef0bec3d203a None None] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:36:29.351972 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 207/406] 10.222.0.20 () {58 vars in 1442 bytes} [Mon Aug 1 16:36:29 2022] PUT /identity/v3/projects/64c2a2f6f91d4b9c8d5993d28fd19631/groups/64b4bd12c620422ca16e3edfbdab7fe7/roles/0999d3da63a545f18ff3252afdc90518 => generated 0 bytes in 297 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 16:36:29.353952 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc5a877-2e8f-4439-be6d-ca3be96de9b1 None None] REQUEST_METHOD: `POST` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:36:29.354386 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc5a877-2e8f-4439-be6d-ca3be96de9b1 None None] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:36:29.354908 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc5a877-2e8f-4439-be6d-ca3be96de9b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:36:29.399531 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db56e47e-502d-415a-8d85-509375c23ae9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:36:29.403611 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db56e47e-502d-415a-8d85-509375c23ae9 None admin] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:36:29.404124 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db56e47e-502d-415a-8d85-509375c23ae9 None admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:36:29.404545 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db56e47e-502d-415a-8d85-509375c23ae9 None admin] PATH_INFO: `/v3/groups/64b4bd12c620422ca16e3edfbdab7fe7` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:36:29.406484 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.auth.core [None req-9bc5a877-2e8f-4439-be6d-ca3be96de9b1 None None] MFA Rules not processed for user `1007f8e379b74e169b98fb7c3c0ba5b4`. Rule list: `[]` (Enabled: `True`). {{(pid=87058) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:36:29.408384 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.fernet_utils [None req-9bc5a877-2e8f-4439-be6d-ca3be96de9b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:36:29.408935 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db56e47e-502d-415a-8d85-509375c23ae9 None admin] RBAC: Authorizing `identity:get_group(group_id=64b4bd12c620422ca16e3edfbdab7fe7)` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:36:29.410834 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db56e47e-502d-415a-8d85-509375c23ae9 None admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:36:29.422258 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 200/407] 10.222.0.20 () {56 vars in 1180 bytes} [Mon Aug 1 16:36:29 2022] GET /identity/v3/groups/64b4bd12c620422ca16e3edfbdab7fe7 => generated 226 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:36:29.426784 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db56e47e-502d-415a-8d85-509375c23ae9 None admin] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:36:29.548016 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 208/408] 10.222.0.20 () {58 vars in 1023 bytes} [Mon Aug 1 16:36:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 16:36:29.550134 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a3c0f8-5b6d-4eff-9b63-70329ec795ef None None] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:36:29.550957 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a3c0f8-5b6d-4eff-9b63-70329ec795ef None None] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:36:29.551388 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a3c0f8-5b6d-4eff-9b63-70329ec795ef None None] PATH_INFO: `/` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:36:29.553692 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 209/409] 10.222.0.20 () {54 vars in 926 bytes} [Mon Aug 1 16:36:29 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 16:36:29.561315 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b93c43-9023-447b-99ef-bec5f977e1c7 None None] REQUEST_METHOD: `POST` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:36:29.561795 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b93c43-9023-447b-99ef-bec5f977e1c7 None None] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:36:29.562541 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b93c43-9023-447b-99ef-bec5f977e1c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:36:29.578803 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db56e47e-502d-415a-8d85-509375c23ae9 None admin] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:36:29.585073 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.auth.core [None req-f9b93c43-9023-447b-99ef-bec5f977e1c7 None None] MFA Rules not processed for user `1007f8e379b74e169b98fb7c3c0ba5b4`. Rule list: `[]` (Enabled: `True`). {{(pid=87058) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:36:29.586498 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.fernet_utils [None req-f9b93c43-9023-447b-99ef-bec5f977e1c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:36:29.607170 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG dbcounter [-] [87057] Writing DB stats keystone:SELECT=100 {{(pid=87057) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:36:29.618562 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2fa239c-21d1-4bc7-9885-516cffcbf856 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:36:29.624039 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2fa239c-21d1-4bc7-9885-516cffcbf856 None admin] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:36:29.624039 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2fa239c-21d1-4bc7-9885-516cffcbf856 None admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:36:29.624039 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2fa239c-21d1-4bc7-9885-516cffcbf856 None admin] PATH_INFO: `/v3/projects/d9a43129dd714d05bd409ffb0875f7e6` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:36:29.624039 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2fa239c-21d1-4bc7-9885-516cffcbf856 None admin] RBAC: Authorizing `identity:get_project(project_id=d9a43129dd714d05bd409ffb0875f7e6)` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:36:29.624039 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2fa239c-21d1-4bc7-9885-516cffcbf856 None admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:36:29.624823 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 201/410] 10.222.0.20 () {56 vars in 1186 bytes} [Mon Aug 1 16:36:29 2022] GET /identity/v3/projects/d9a43129dd714d05bd409ffb0875f7e6 => generated 302 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 16:36:29.630382 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2fa239c-21d1-4bc7-9885-516cffcbf856 None admin] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:36:29.682136 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 210/411] 10.222.0.20 () {58 vars in 1023 bytes} [Mon Aug 1 16:36:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 01 16:36:29.682136 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9b93c43-9023-447b-99ef-bec5f977e1c7 None None] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:36:29.689422 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG dbcounter [-] [87058] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:36:29.770549 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2fa239c-21d1-4bc7-9885-516cffcbf856 None admin] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:36:29.806892 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9b93c43-9023-447b-99ef-bec5f977e1c7 None None] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:36:29.817575 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f96b0dc-6692-4d2e-8fa9-93ff6bc8a0e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:36:29.818823 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f96b0dc-6692-4d2e-8fa9-93ff6bc8a0e5 None admin] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:36:29.819315 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f96b0dc-6692-4d2e-8fa9-93ff6bc8a0e5 None admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:36:29.821410 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f96b0dc-6692-4d2e-8fa9-93ff6bc8a0e5 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:36:29.832869 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f96b0dc-6692-4d2e-8fa9-93ff6bc8a0e5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 16:36:29.840056 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application [None req-3f96b0dc-6692-4d2e-8fa9-93ff6bc8a0e5 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 16:36:29.840056 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 16:36:29.840056 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 16:36:29.840056 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 16:36:29.840056 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 16:36:29.840056 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 16:36:29.840056 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 16:36:29.840056 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 16:36:29.840056 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 16:36:29.840056 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 16:36:29.840056 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 16:36:29.840056 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 16:36:29.840056 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 16:36:29.840056 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 16:36:29.840056 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 16:36:29.840056 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 16:36:29.841028 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 16:36:29.841028 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 16:36:29.841028 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 16:36:29.841028 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 16:36:29.841028 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 16:36:29.841028 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 16:36:29.841028 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 16:36:29.841028 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 16:36:29.841028 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 16:36:29.841028 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 16:36:29.841028 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 16:36:29.841028 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application with Lock( Aug 01 16:36:29.841028 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 16:36:29.841028 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self._enter() Aug 01 16:36:29.841028 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 16:36:29.841028 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 16:36:29.841812 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 16:36:29.841812 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self.creator() Aug 01 16:36:29.841812 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 16:36:29.841812 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application created_value = creator( Aug 01 16:36:29.841812 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 16:36:29.841812 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 16:36:29.841812 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 16:36:29.841812 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 16:36:29.841812 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 16:36:29.841812 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 16:36:29.841812 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 16:36:29.841812 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application Aug 01 16:36:29.848098 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 202/412] 10.222.0.20 () {56 vars in 1096 bytes} [Mon Aug 1 16:36:29 2022] GET /identity/v3/roles/admin => generated 83 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 16:36:29.848098 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f96b0dc-6692-4d2e-8fa9-93ff6bc8a0e5 None admin] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:36:29.848305 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a9a6123-547b-492c-b861-d7fd6baf4d5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:36:29.848305 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9a6123-547b-492c-b861-d7fd6baf4d5d None admin] REQUEST_METHOD: `PUT` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:36:29.848305 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9a6123-547b-492c-b861-d7fd6baf4d5d None admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:36:29.848305 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9a6123-547b-492c-b861-d7fd6baf4d5d None admin] PATH_INFO: `/v3/projects/d9a43129dd714d05bd409ffb0875f7e6/groups/64b4bd12c620422ca16e3edfbdab7fe7/roles/3fac71a39f3f4ed198a3579064c30268` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:36:29.853441 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a9a6123-547b-492c-b861-d7fd6baf4d5d None admin] RBAC: Authorizing `identity:create_grant(project_id=d9a43129dd714d05bd409ffb0875f7e6, group_id=64b4bd12c620422ca16e3edfbdab7fe7, role_id=3fac71a39f3f4ed198a3579064c30268)` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:36:29.855446 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a9a6123-547b-492c-b861-d7fd6baf4d5d None admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:36:29.908675 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG dbcounter [-] [87057] Writing DB stats keystone:SELECT=100 {{(pid=87057) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:36:29.922396 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 211/413] 10.222.0.20 () {58 vars in 1442 bytes} [Mon Aug 1 16:36:29 2022] PUT /identity/v3/projects/d9a43129dd714d05bd409ffb0875f7e6/groups/64b4bd12c620422ca16e3edfbdab7fe7/roles/3fac71a39f3f4ed198a3579064c30268 => generated 0 bytes in 246 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 16:36:29.924496 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a9a6123-547b-492c-b861-d7fd6baf4d5d None admin] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:36:29.962451 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f96b0dc-6692-4d2e-8fa9-93ff6bc8a0e5 None admin] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:36:29.999996 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd3bf39-3fb9-4ccd-ae7e-ac9d55eaf269 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:36:30.000647 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd3bf39-3fb9-4ccd-ae7e-ac9d55eaf269 None admin] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:36:30.000924 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd3bf39-3fb9-4ccd-ae7e-ac9d55eaf269 None admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:36:30.001200 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd3bf39-3fb9-4ccd-ae7e-ac9d55eaf269 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:36:30.013604 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dd3bf39-3fb9-4ccd-ae7e-ac9d55eaf269 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 16:36:30.019353 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application [None req-5dd3bf39-3fb9-4ccd-ae7e-ac9d55eaf269 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 01 16:36:30.019353 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 16:36:30.019353 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 16:36:30.019353 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 16:36:30.019353 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 16:36:30.019353 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 16:36:30.019353 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 16:36:30.019353 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 16:36:30.019353 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 16:36:30.019353 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 16:36:30.019353 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 16:36:30.019353 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 16:36:30.019353 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 16:36:30.019353 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 16:36:30.019353 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 16:36:30.019353 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 16:36:30.020385 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 16:36:30.020385 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 16:36:30.020385 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 16:36:30.020385 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 16:36:30.020385 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 16:36:30.020385 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 16:36:30.020385 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 16:36:30.020385 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 16:36:30.020385 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 16:36:30.020385 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 16:36:30.020385 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 16:36:30.020385 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application with Lock( Aug 01 16:36:30.020385 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 16:36:30.020385 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self._enter() Aug 01 16:36:30.020385 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 16:36:30.020385 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 16:36:30.021634 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 16:36:30.021634 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self.creator() Aug 01 16:36:30.021634 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 16:36:30.021634 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application created_value = creator( Aug 01 16:36:30.021634 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 16:36:30.021634 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 16:36:30.021634 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 16:36:30.021634 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 16:36:30.021634 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 16:36:30.021634 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 16:36:30.021634 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 01 16:36:30.021634 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application Aug 01 16:36:30.021634 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 203/414] 10.222.0.20 () {56 vars in 1114 bytes} [Mon Aug 1 16:36:29 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 16:36:30.026857 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd3bf39-3fb9-4ccd-ae7e-ac9d55eaf269 None admin] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:36:30.038770 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG dbcounter [-] [87058] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:36:30.086533 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a9a6123-547b-492c-b861-d7fd6baf4d5d None admin] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:36:30.130633 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf732c5c-5b9b-4daf-91f9-628b5921ce18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:36:30.131942 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf732c5c-5b9b-4daf-91f9-628b5921ce18 None admin] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:36:30.133363 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf732c5c-5b9b-4daf-91f9-628b5921ce18 None admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:36:30.133902 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf732c5c-5b9b-4daf-91f9-628b5921ce18 None admin] PATH_INFO: `/v3/roles` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:36:30.134926 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf732c5c-5b9b-4daf-91f9-628b5921ce18 None admin] RBAC: Adding query filter params (name=admin) {{(pid=87058) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:36:30.135447 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf732c5c-5b9b-4daf-91f9-628b5921ce18 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:36:30.137089 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf732c5c-5b9b-4daf-91f9-628b5921ce18 None admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:36:30.150523 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 212/415] 10.222.0.20 () {56 vars in 1099 bytes} [Mon Aug 1 16:36:29 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:36:30.156666 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf732c5c-5b9b-4daf-91f9-628b5921ce18 None admin] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:36:30.217311 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd3bf39-3fb9-4ccd-ae7e-ac9d55eaf269 None admin] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:36:30.237212 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG dbcounter [-] [87057] Writing DB stats keystone:SELECT=100 {{(pid=87057) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:36:30.290198 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd5a47d9-b058-472b-8cd7-a58d1812cdaf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:36:30.290827 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5a47d9-b058-472b-8cd7-a58d1812cdaf None admin] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:36:30.291101 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5a47d9-b058-472b-8cd7-a58d1812cdaf None admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:36:30.291369 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5a47d9-b058-472b-8cd7-a58d1812cdaf None admin] PATH_INFO: `/v3/roles` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:36:30.292099 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd5a47d9-b058-472b-8cd7-a58d1812cdaf None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=87057) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:36:30.292398 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd5a47d9-b058-472b-8cd7-a58d1812cdaf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:36:30.293773 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd5a47d9-b058-472b-8cd7-a58d1812cdaf None admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:36:30.324185 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 204/416] 10.222.0.20 () {56 vars in 1111 bytes} [Mon Aug 1 16:36:30 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 298 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 16:36:30.331277 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd5a47d9-b058-472b-8cd7-a58d1812cdaf None admin] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:36:30.333370 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf732c5c-5b9b-4daf-91f9-628b5921ce18 None admin] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:36:30.370403 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG dbcounter [-] [87058] Writing DB stats keystone:SELECT=100 {{(pid=87058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:36:30.376427 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a04b0a75-41c3-437e-a297-592939ab6e38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:36:30.377193 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04b0a75-41c3-437e-a297-592939ab6e38 None admin] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:36:30.378317 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04b0a75-41c3-437e-a297-592939ab6e38 None admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:36:30.383116 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04b0a75-41c3-437e-a297-592939ab6e38 None admin] PATH_INFO: `/v3/groups/fb6971a50bbf408a8817d213299e6a64` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:36:30.383116 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a04b0a75-41c3-437e-a297-592939ab6e38 None admin] RBAC: Authorizing `identity:get_group(group_id=fb6971a50bbf408a8817d213299e6a64)` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:36:30.384548 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a04b0a75-41c3-437e-a297-592939ab6e38 None admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:36:30.390062 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 213/417] 10.222.0.20 () {56 vars in 1180 bytes} [Mon Aug 1 16:36:30 2022] GET /identity/v3/groups/fb6971a50bbf408a8817d213299e6a64 => generated 229 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:36:30.394149 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a04b0a75-41c3-437e-a297-592939ab6e38 None admin] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:36:30.541547 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a04b0a75-41c3-437e-a297-592939ab6e38 None admin] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:36:30.547989 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd5a47d9-b058-472b-8cd7-a58d1812cdaf None admin] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:36:30.598108 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c57cb130-9c32-4357-80df-d8f663cba6dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:36:30.598750 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-425259c0-9d2b-406b-a7c5-5d69f7e7b4d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:36:30.599365 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57cb130-9c32-4357-80df-d8f663cba6dd None admin] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:36:30.599471 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425259c0-9d2b-406b-a7c5-5d69f7e7b4d7 None admin] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:36:30.599685 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425259c0-9d2b-406b-a7c5-5d69f7e7b4d7 None admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:36:30.599963 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425259c0-9d2b-406b-a7c5-5d69f7e7b4d7 None admin] PATH_INFO: `/v3/projects/3e43f22f36744d079b60704aff4969d4` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:36:30.600340 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57cb130-9c32-4357-80df-d8f663cba6dd None admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:36:30.601345 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57cb130-9c32-4357-80df-d8f663cba6dd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:36:30.601888 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-425259c0-9d2b-406b-a7c5-5d69f7e7b4d7 None admin] RBAC: Authorizing `identity:get_project(project_id=3e43f22f36744d079b60704aff4969d4)` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:36:30.603431 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-425259c0-9d2b-406b-a7c5-5d69f7e7b4d7 None admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:36:30.609596 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 214/418] 10.222.0.20 () {56 vars in 1186 bytes} [Mon Aug 1 16:36:30 2022] GET /identity/v3/projects/3e43f22f36744d079b60704aff4969d4 => generated 344 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:36:30.610692 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-425259c0-9d2b-406b-a7c5-5d69f7e7b4d7 None admin] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:36:30.652063 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG dbcounter [-] [87057] Writing DB stats keystone:SELECT=100 {{(pid=87057) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:36:30.680645 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c57cb130-9c32-4357-80df-d8f663cba6dd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:36:30.682591 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c57cb130-9c32-4357-80df-d8f663cba6dd None admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:36:30.720405 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG dbcounter [-] [87058] Writing DB stats keystone:SELECT=100 {{(pid=87058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:36:30.780929 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-425259c0-9d2b-406b-a7c5-5d69f7e7b4d7 None admin] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:36:30.793924 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 205/419] 10.222.0.20 () {58 vars in 1282 bytes} [Mon Aug 1 16:36:30 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 463 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 01 16:36:30.799172 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c57cb130-9c32-4357-80df-d8f663cba6dd None admin] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:36:30.878162 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42bb28fe-0777-4204-a5eb-479a1338aa75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:36:30.878868 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42bb28fe-0777-4204-a5eb-479a1338aa75 None admin] REQUEST_METHOD: `PUT` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:36:30.879117 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42bb28fe-0777-4204-a5eb-479a1338aa75 None admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:36:30.879389 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42bb28fe-0777-4204-a5eb-479a1338aa75 None admin] PATH_INFO: `/v3/projects/3e43f22f36744d079b60704aff4969d4/groups/fb6971a50bbf408a8817d213299e6a64/roles/440bb943dd5b46fea038b903b6eaaf81` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:36:30.881103 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG dbcounter [-] [87057] Writing DB stats keystone:SELECT=100 {{(pid=87057) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:36:30.881491 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42bb28fe-0777-4204-a5eb-479a1338aa75 None admin] RBAC: Authorizing `identity:create_grant(project_id=3e43f22f36744d079b60704aff4969d4, group_id=fb6971a50bbf408a8817d213299e6a64, role_id=440bb943dd5b46fea038b903b6eaaf81)` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:36:30.891067 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42bb28fe-0777-4204-a5eb-479a1338aa75 None admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:36:30.910211 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c57cb130-9c32-4357-80df-d8f663cba6dd None admin] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:36:30.947842 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-057b8722-d9ac-4fb4-a494-91760ec5ee27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:36:30.948841 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057b8722-d9ac-4fb4-a494-91760ec5ee27 None admin] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:36:30.949533 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057b8722-d9ac-4fb4-a494-91760ec5ee27 None admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:36:30.950004 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057b8722-d9ac-4fb4-a494-91760ec5ee27 None admin] PATH_INFO: `/v3/projects/64c2a2f6f91d4b9c8d5993d28fd19631` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:36:30.951400 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-057b8722-d9ac-4fb4-a494-91760ec5ee27 None admin] RBAC: Authorizing `identity:get_project(project_id=64c2a2f6f91d4b9c8d5993d28fd19631)` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:36:30.953022 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-057b8722-d9ac-4fb4-a494-91760ec5ee27 None admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:36:30.955331 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 206/420] 10.222.0.20 () {56 vars in 1186 bytes} [Mon Aug 1 16:36:30 2022] GET /identity/v3/projects/64c2a2f6f91d4b9c8d5993d28fd19631 => generated 298 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:36:30.961069 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-057b8722-d9ac-4fb4-a494-91760ec5ee27 None admin] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:36:30.995060 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 215/421] 10.222.0.20 () {58 vars in 1442 bytes} [Mon Aug 1 16:36:30 2022] PUT /identity/v3/projects/3e43f22f36744d079b60704aff4969d4/groups/fb6971a50bbf408a8817d213299e6a64/roles/440bb943dd5b46fea038b903b6eaaf81 => generated 0 bytes in 385 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 16:36:31.075021 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-057b8722-d9ac-4fb4-a494-91760ec5ee27 None admin] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:36:31.114076 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22df570d-9812-450c-9602-2bfd3e3dbfbb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:36:31.115048 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22df570d-9812-450c-9602-2bfd3e3dbfbb None admin] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:36:31.115507 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22df570d-9812-450c-9602-2bfd3e3dbfbb None admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:36:31.115956 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22df570d-9812-450c-9602-2bfd3e3dbfbb None admin] PATH_INFO: `/v3/groups/64b4bd12c620422ca16e3edfbdab7fe7` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:36:31.118067 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22df570d-9812-450c-9602-2bfd3e3dbfbb None admin] RBAC: Authorizing `identity:get_group(group_id=64b4bd12c620422ca16e3edfbdab7fe7)` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:36:31.119783 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22df570d-9812-450c-9602-2bfd3e3dbfbb None admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:36:31.122003 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 207/422] 10.222.0.20 () {56 vars in 1180 bytes} [Mon Aug 1 16:36:30 2022] GET /identity/v3/groups/64b4bd12c620422ca16e3edfbdab7fe7 => generated 226 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:36:31.129794 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42bb28fe-0777-4204-a5eb-479a1338aa75 None admin] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:36:31.263693 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42bb28fe-0777-4204-a5eb-479a1338aa75 None admin] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:36:31.293843 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG dbcounter [-] [87058] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:36:31.301909 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-206cedc4-33cd-4164-afe2-930508644f49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:36:31.302573 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206cedc4-33cd-4164-afe2-930508644f49 None admin] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:36:31.302854 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206cedc4-33cd-4164-afe2-930508644f49 None admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:36:31.303129 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206cedc4-33cd-4164-afe2-930508644f49 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:36:31.303733 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22a5c8a-1b72-4cfd-b348-9326677a23df None None] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:36:31.304590 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-206cedc4-33cd-4164-afe2-930508644f49 None admin] RBAC: Adding query filter params (group.id=64b4bd12c620422ca16e3edfbdab7fe7, role.id=3fac71a39f3f4ed198a3579064c30268, scope.project.id=64c2a2f6f91d4b9c8d5993d28fd19631) {{(pid=87058) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:36:31.304914 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-206cedc4-33cd-4164-afe2-930508644f49 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:36:31.305301 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22a5c8a-1b72-4cfd-b348-9326677a23df None None] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:36:31.305802 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22a5c8a-1b72-4cfd-b348-9326677a23df None None] PATH_INFO: `/` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:36:31.306498 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-206cedc4-33cd-4164-afe2-930508644f49 None admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:36:31.307694 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 208/423] 10.222.0.20 () {54 vars in 926 bytes} [Mon Aug 1 16:36:31 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 01 16:36:31.313090 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d0978b-de8b-4940-ae46-fda3bc569c58 None None] REQUEST_METHOD: `POST` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:36:31.313360 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d0978b-de8b-4940-ae46-fda3bc569c58 None None] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:36:31.313474 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 216/424] 10.222.0.20 () {56 vars in 1376 bytes} [Mon Aug 1 16:36:31 2022] GET /identity/v3/role_assignments?group.id=64b4bd12c620422ca16e3edfbdab7fe7&scope.project.id=64c2a2f6f91d4b9c8d5993d28fd19631&role.id=3fac71a39f3f4ed198a3579064c30268 => generated 611 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:36:31.313645 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d0978b-de8b-4940-ae46-fda3bc569c58 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:36:31.337069 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.auth.core [None req-b1d0978b-de8b-4940-ae46-fda3bc569c58 None None] MFA Rules not processed for user `1007f8e379b74e169b98fb7c3c0ba5b4`. Rule list: `[]` (Enabled: `True`). {{(pid=87057) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:36:31.338244 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.fernet_utils [None req-b1d0978b-de8b-4940-ae46-fda3bc569c58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87057) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:36:31.342405 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG dbcounter [-] [87057] Writing DB stats keystone:SELECT=100 {{(pid=87057) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:36:31.421384 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 209/425] 10.222.0.20 () {58 vars in 1023 bytes} [Mon Aug 1 16:36:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 01 16:36:31.457047 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-206cedc4-33cd-4164-afe2-930508644f49 None admin] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:36:31.593686 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-206cedc4-33cd-4164-afe2-930508644f49 None admin] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:36:31.628031 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db7ab166-4721-4910-b835-36f99adf2f11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:36:31.628669 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7ab166-4721-4910-b835-36f99adf2f11 None admin] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:36:31.628947 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7ab166-4721-4910-b835-36f99adf2f11 None admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:36:31.629218 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7ab166-4721-4910-b835-36f99adf2f11 None admin] PATH_INFO: `/v3/roles/member` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:36:31.639584 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db7ab166-4721-4910-b835-36f99adf2f11 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 16:36:31.645268 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application [None req-db7ab166-4721-4910-b835-36f99adf2f11 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 01 16:36:31.645268 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 16:36:31.645268 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 16:36:31.645268 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 16:36:31.645268 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 16:36:31.645268 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 16:36:31.645268 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 16:36:31.645268 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 16:36:31.645268 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 16:36:31.645268 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 16:36:31.645268 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 16:36:31.645268 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 16:36:31.645268 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 16:36:31.645268 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 16:36:31.645268 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 16:36:31.645268 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 16:36:31.646241 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 16:36:31.646241 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 16:36:31.646241 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 16:36:31.646241 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 16:36:31.646241 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 16:36:31.646241 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 16:36:31.646241 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 16:36:31.646241 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 16:36:31.646241 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 16:36:31.646241 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 16:36:31.646241 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 16:36:31.646241 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application with Lock( Aug 01 16:36:31.646241 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 16:36:31.646241 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self._enter() Aug 01 16:36:31.646241 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 16:36:31.646241 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 16:36:31.647041 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 16:36:31.647041 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self.creator() Aug 01 16:36:31.647041 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 16:36:31.647041 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application created_value = creator( Aug 01 16:36:31.647041 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 16:36:31.647041 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 16:36:31.647041 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 16:36:31.647041 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 16:36:31.647041 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 16:36:31.647041 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 16:36:31.647041 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 01 16:36:31.647041 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application Aug 01 16:36:31.647041 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 217/426] 10.222.0.20 () {56 vars in 1099 bytes} [Mon Aug 1 16:36:31 2022] GET /identity/v3/roles/member => generated 84 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 16:36:31.652969 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d0978b-de8b-4940-ae46-fda3bc569c58 None None] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:36:31.739480 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94496ea-837f-4505-aa62-de53a096b986 None None] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:36:31.739766 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94496ea-837f-4505-aa62-de53a096b986 None None] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:36:31.740046 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94496ea-837f-4505-aa62-de53a096b986 None None] PATH_INFO: `/` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:36:31.744397 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 218/427] 10.222.0.20 () {54 vars in 926 bytes} [Mon Aug 1 16:36:31 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 01 16:36:31.747647 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c774985-003f-4369-9492-8883893924d6 None None] REQUEST_METHOD: `POST` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:36:31.747647 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c774985-003f-4369-9492-8883893924d6 None None] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:36:31.748001 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c774985-003f-4369-9492-8883893924d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:36:31.767928 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d0978b-de8b-4940-ae46-fda3bc569c58 None None] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:36:31.768736 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.auth.core [None req-9c774985-003f-4369-9492-8883893924d6 None None] MFA Rules not processed for user `1007f8e379b74e169b98fb7c3c0ba5b4`. Rule list: `[]` (Enabled: `True`). {{(pid=87058) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:36:31.770598 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.fernet_utils [None req-9c774985-003f-4369-9492-8883893924d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:36:31.812771 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feed58d4-7773-4dda-a879-5938d35b153a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:36:31.813397 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feed58d4-7773-4dda-a879-5938d35b153a None admin] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:36:31.813678 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feed58d4-7773-4dda-a879-5938d35b153a None admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:36:31.813953 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feed58d4-7773-4dda-a879-5938d35b153a None admin] PATH_INFO: `/v3/roles` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:36:31.814709 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feed58d4-7773-4dda-a879-5938d35b153a None admin] RBAC: Adding query filter params (name=member) {{(pid=87057) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:36:31.815013 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feed58d4-7773-4dda-a879-5938d35b153a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:36:31.816397 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feed58d4-7773-4dda-a879-5938d35b153a None admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:36:31.828096 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 210/428] 10.222.0.20 () {56 vars in 1101 bytes} [Mon Aug 1 16:36:31 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 16:36:31.829674 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG dbcounter [-] [87058] Writing DB stats keystone:SELECT=100 {{(pid=87058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:36:31.833145 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feed58d4-7773-4dda-a879-5938d35b153a None admin] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:36:31.837439 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG dbcounter [-] [87057] Writing DB stats keystone:SELECT=100 {{(pid=87057) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:36:31.854494 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 219/429] 10.222.0.20 () {58 vars in 1023 bytes} [Mon Aug 1 16:36:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 01 16:36:31.883389 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c774985-003f-4369-9492-8883893924d6 None None] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:36:31.959466 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feed58d4-7773-4dda-a879-5938d35b153a None admin] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:36:32.000932 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7abb6e31-d8e8-4023-98e5-fc77a941585f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:36:32.001551 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7abb6e31-d8e8-4023-98e5-fc77a941585f None admin] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:36:32.001832 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7abb6e31-d8e8-4023-98e5-fc77a941585f None admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:36:32.002133 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7abb6e31-d8e8-4023-98e5-fc77a941585f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:36:32.014711 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c774985-003f-4369-9492-8883893924d6 None None] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:36:32.057250 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9168daa7-e7ff-443d-8414-c609c0a3a6d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:36:32.057911 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9168daa7-e7ff-443d-8414-c609c0a3a6d0 None admin] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:36:32.058230 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9168daa7-e7ff-443d-8414-c609c0a3a6d0 None admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:36:32.058514 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9168daa7-e7ff-443d-8414-c609c0a3a6d0 None admin] PATH_INFO: `/v3/roles/member` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:36:32.070020 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9168daa7-e7ff-443d-8414-c609c0a3a6d0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 16:36:32.076456 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application [None req-9168daa7-e7ff-443d-8414-c609c0a3a6d0 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 01 16:36:32.076456 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 16:36:32.076456 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 16:36:32.076456 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 16:36:32.076456 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 16:36:32.076456 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 16:36:32.076456 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 16:36:32.076456 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 16:36:32.076456 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 16:36:32.076456 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 16:36:32.076456 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 16:36:32.076456 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 16:36:32.076456 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 16:36:32.076456 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 16:36:32.076456 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 16:36:32.076456 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 16:36:32.077374 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 16:36:32.077374 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 16:36:32.077374 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 16:36:32.077374 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 16:36:32.077374 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 16:36:32.077374 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 16:36:32.077374 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 16:36:32.077374 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 16:36:32.077374 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 16:36:32.077374 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 16:36:32.077374 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 16:36:32.077374 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application with Lock( Aug 01 16:36:32.077374 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 16:36:32.077374 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self._enter() Aug 01 16:36:32.077374 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 16:36:32.077374 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 16:36:32.078234 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 16:36:32.078234 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self.creator() Aug 01 16:36:32.078234 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 16:36:32.078234 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application created_value = creator( Aug 01 16:36:32.078234 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 16:36:32.078234 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 16:36:32.078234 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 16:36:32.078234 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 16:36:32.078234 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 16:36:32.078234 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 16:36:32.078234 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 01 16:36:32.078234 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application Aug 01 16:36:32.078234 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 220/430] 10.222.0.20 () {56 vars in 1099 bytes} [Mon Aug 1 16:36:31 2022] GET /identity/v3/roles/member => generated 84 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 01 16:36:32.085785 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9168daa7-e7ff-443d-8414-c609c0a3a6d0 None admin] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:36:32.095062 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7abb6e31-d8e8-4023-98e5-fc77a941585f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:36:32.097308 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7abb6e31-d8e8-4023-98e5-fc77a941585f None admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:36:32.100232 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG dbcounter [-] [87057] Writing DB stats keystone:SELECT=100 {{(pid=87057) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:36:32.138249 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG dbcounter [-] [87058] Writing DB stats keystone:SELECT=100 {{(pid=87058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:36:32.204270 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9168daa7-e7ff-443d-8414-c609c0a3a6d0 None admin] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:36:32.233252 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 211/431] 10.222.0.20 () {58 vars in 1282 bytes} [Mon Aug 1 16:36:31 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 400 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 01 16:36:32.239992 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7abb6e31-d8e8-4023-98e5-fc77a941585f None admin] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:36:32.243792 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a3d543-fa08-4465-91bb-d02dc1683287 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:36:32.244516 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a3d543-fa08-4465-91bb-d02dc1683287 None admin] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:36:32.244867 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a3d543-fa08-4465-91bb-d02dc1683287 None admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:36:32.245213 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a3d543-fa08-4465-91bb-d02dc1683287 None admin] PATH_INFO: `/v3/roles` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:36:32.246095 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12a3d543-fa08-4465-91bb-d02dc1683287 None admin] RBAC: Adding query filter params (name=member) {{(pid=87058) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:36:32.246472 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12a3d543-fa08-4465-91bb-d02dc1683287 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:36:32.248034 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12a3d543-fa08-4465-91bb-d02dc1683287 None admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:36:32.262906 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 221/432] 10.222.0.20 () {56 vars in 1101 bytes} [Mon Aug 1 16:36:32 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:36:32.268459 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a3d543-fa08-4465-91bb-d02dc1683287 None admin] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:36:32.353830 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7abb6e31-d8e8-4023-98e5-fc77a941585f None admin] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:36:32.354466 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG dbcounter [-] [87057] Writing DB stats keystone:SELECT=100 {{(pid=87057) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:36:32.390739 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a3d543-fa08-4465-91bb-d02dc1683287 None admin] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:36:32.391034 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58a6864c-dffc-4b3f-81f0-d8c7700591cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:36:32.391680 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a6864c-dffc-4b3f-81f0-d8c7700591cf None admin] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:36:32.392251 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a6864c-dffc-4b3f-81f0-d8c7700591cf None admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:36:32.392594 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a6864c-dffc-4b3f-81f0-d8c7700591cf None admin] PATH_INFO: `/v3/projects/d9a43129dd714d05bd409ffb0875f7e6` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:36:32.393333 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG dbcounter [-] [87058] Writing DB stats keystone:SELECT=100 {{(pid=87058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:36:32.394677 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58a6864c-dffc-4b3f-81f0-d8c7700591cf None admin] RBAC: Authorizing `identity:get_project(project_id=d9a43129dd714d05bd409ffb0875f7e6)` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:36:32.396300 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58a6864c-dffc-4b3f-81f0-d8c7700591cf None admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:36:32.403692 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 212/433] 10.222.0.20 () {56 vars in 1186 bytes} [Mon Aug 1 16:36:32 2022] GET /identity/v3/projects/d9a43129dd714d05bd409ffb0875f7e6 => generated 302 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 16:36:32.403692 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c897edd7-fd33-4028-bea5-d6cd521da9d6 None None] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:36:32.403692 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c897edd7-fd33-4028-bea5-d6cd521da9d6 None None] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:36:32.403692 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c897edd7-fd33-4028-bea5-d6cd521da9d6 None None] PATH_INFO: `/` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:36:32.403692 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 213/434] 10.222.0.20 () {54 vars in 926 bytes} [Mon Aug 1 16:36:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 16:36:32.408514 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c897edd7-fd33-4028-bea5-d6cd521da9d6 None None] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:36:32.433540 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b95f2e-5ff8-4df7-836b-fc651e2daaf2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:36:32.434285 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b95f2e-5ff8-4df7-836b-fc651e2daaf2 None admin] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:36:32.434641 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b95f2e-5ff8-4df7-836b-fc651e2daaf2 None admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:36:32.434985 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b95f2e-5ff8-4df7-836b-fc651e2daaf2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:36:32.508049 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00b95f2e-5ff8-4df7-836b-fc651e2daaf2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:36:32.509737 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00b95f2e-5ff8-4df7-836b-fc651e2daaf2 None admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:36:32.516723 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c897edd7-fd33-4028-bea5-d6cd521da9d6 None None] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:36:32.553626 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c4f4b8-fdde-4e87-aeda-c7a63102329d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:36:32.554952 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c4f4b8-fdde-4e87-aeda-c7a63102329d None admin] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:36:32.555233 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c4f4b8-fdde-4e87-aeda-c7a63102329d None admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:36:32.555507 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c4f4b8-fdde-4e87-aeda-c7a63102329d None admin] PATH_INFO: `/v3/groups/64b4bd12c620422ca16e3edfbdab7fe7` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:36:32.556666 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14c4f4b8-fdde-4e87-aeda-c7a63102329d None admin] RBAC: Authorizing `identity:get_group(group_id=64b4bd12c620422ca16e3edfbdab7fe7)` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:36:32.558138 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14c4f4b8-fdde-4e87-aeda-c7a63102329d None admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:36:32.559799 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 214/435] 10.222.0.20 () {56 vars in 1180 bytes} [Mon Aug 1 16:36:32 2022] GET /identity/v3/groups/64b4bd12c620422ca16e3edfbdab7fe7 => generated 226 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:36:32.560912 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e589b0-d6a3-4d36-9317-f671001da696 None None] REQUEST_METHOD: `POST` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:36:32.561188 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e589b0-d6a3-4d36-9317-f671001da696 None None] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:36:32.561458 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e589b0-d6a3-4d36-9317-f671001da696 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:36:32.579865 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.auth.core [None req-02e589b0-d6a3-4d36-9317-f671001da696 None None] MFA Rules not processed for user `1007f8e379b74e169b98fb7c3c0ba5b4`. Rule list: `[]` (Enabled: `True`). {{(pid=87057) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:36:32.581052 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.fernet_utils [None req-02e589b0-d6a3-4d36-9317-f671001da696 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87057) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:36:32.611757 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG dbcounter [-] [87057] Writing DB stats keystone:SELECT=100 {{(pid=87057) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:36:32.622379 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 222/436] 10.222.0.20 () {58 vars in 1282 bytes} [Mon Aug 1 16:36:32 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 354 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 01 16:36:32.623919 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b95f2e-5ff8-4df7-836b-fc651e2daaf2 None admin] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:36:32.627809 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG dbcounter [-] [87058] Writing DB stats keystone:SELECT=100 {{(pid=87058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:36:32.652979 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 215/437] 10.222.0.20 () {58 vars in 1023 bytes} [Mon Aug 1 16:36:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 16:36:32.654008 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02e589b0-d6a3-4d36-9317-f671001da696 None None] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:36:32.728377 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b95f2e-5ff8-4df7-836b-fc651e2daaf2 None admin] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:36:32.762599 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d454ebb6-aaa7-4406-954a-c3c365c9f17f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:36:32.763187 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02e589b0-d6a3-4d36-9317-f671001da696 None None] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:36:32.763324 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d454ebb6-aaa7-4406-954a-c3c365c9f17f None admin] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:36:32.763486 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d454ebb6-aaa7-4406-954a-c3c365c9f17f None admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:36:32.763734 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d454ebb6-aaa7-4406-954a-c3c365c9f17f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:36:32.764546 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d454ebb6-aaa7-4406-954a-c3c365c9f17f None admin] RBAC: Adding query filter params (group.id=64b4bd12c620422ca16e3edfbdab7fe7, role.id=0999d3da63a545f18ff3252afdc90518, scope.project.id=d9a43129dd714d05bd409ffb0875f7e6) {{(pid=87058) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:36:32.764856 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d454ebb6-aaa7-4406-954a-c3c365c9f17f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:36:32.766312 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d454ebb6-aaa7-4406-954a-c3c365c9f17f None admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:36:32.772487 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 223/438] 10.222.0.20 () {56 vars in 1376 bytes} [Mon Aug 1 16:36:32 2022] GET /identity/v3/role_assignments?group.id=64b4bd12c620422ca16e3edfbdab7fe7&scope.project.id=d9a43129dd714d05bd409ffb0875f7e6&role.id=0999d3da63a545f18ff3252afdc90518 => generated 611 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:36:32.773171 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d454ebb6-aaa7-4406-954a-c3c365c9f17f None admin] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:36:32.814717 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9add1775-f5ba-4298-961b-6e417a092fa2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:36:32.815768 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9add1775-f5ba-4298-961b-6e417a092fa2 None admin] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:36:32.816321 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9add1775-f5ba-4298-961b-6e417a092fa2 None admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:36:32.816821 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9add1775-f5ba-4298-961b-6e417a092fa2 None admin] PATH_INFO: `/v3/projects/64c2a2f6f91d4b9c8d5993d28fd19631` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:36:32.818569 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9add1775-f5ba-4298-961b-6e417a092fa2 None admin] RBAC: Authorizing `identity:get_project(project_id=64c2a2f6f91d4b9c8d5993d28fd19631)` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:36:32.820467 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9add1775-f5ba-4298-961b-6e417a092fa2 None admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:36:32.823227 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 216/439] 10.222.0.20 () {56 vars in 1186 bytes} [Mon Aug 1 16:36:32 2022] GET /identity/v3/projects/64c2a2f6f91d4b9c8d5993d28fd19631 => generated 298 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:36:32.828590 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9add1775-f5ba-4298-961b-6e417a092fa2 None admin] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:36:32.860756 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG dbcounter [-] [87058] Writing DB stats keystone:SELECT=100 {{(pid=87058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:36:32.872598 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG dbcounter [-] [87057] Writing DB stats keystone:SELECT=100 {{(pid=87057) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:36:32.903253 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d454ebb6-aaa7-4406-954a-c3c365c9f17f None admin] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:36:32.946425 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e0b48b3-66fc-4d2a-8c5e-064afbeaf013 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:36:32.947459 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0b48b3-66fc-4d2a-8c5e-064afbeaf013 None admin] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:36:32.948029 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0b48b3-66fc-4d2a-8c5e-064afbeaf013 None admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:36:32.948503 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0b48b3-66fc-4d2a-8c5e-064afbeaf013 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:36:32.952207 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9add1775-f5ba-4298-961b-6e417a092fa2 None admin] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:36:32.960066 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e0b48b3-66fc-4d2a-8c5e-064afbeaf013 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 16:36:32.967152 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application [None req-9e0b48b3-66fc-4d2a-8c5e-064afbeaf013 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 01 16:36:32.967152 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 16:36:32.967152 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 16:36:32.967152 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 16:36:32.967152 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 16:36:32.967152 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 16:36:32.967152 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 16:36:32.967152 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 16:36:32.967152 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 16:36:32.967152 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 16:36:32.967152 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 16:36:32.967152 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 16:36:32.967152 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 16:36:32.967152 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 16:36:32.967152 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 16:36:32.967152 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 16:36:32.968167 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 16:36:32.968167 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 16:36:32.968167 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 16:36:32.968167 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 16:36:32.968167 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 16:36:32.968167 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 16:36:32.968167 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 16:36:32.968167 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 16:36:32.968167 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 16:36:32.968167 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 16:36:32.968167 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 16:36:32.968167 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application with Lock( Aug 01 16:36:32.968167 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 16:36:32.968167 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self._enter() Aug 01 16:36:32.968167 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 16:36:32.968167 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 16:36:32.968985 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 16:36:32.968985 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self.creator() Aug 01 16:36:32.968985 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 16:36:32.968985 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application created_value = creator( Aug 01 16:36:32.968985 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 16:36:32.968985 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 16:36:32.968985 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 16:36:32.968985 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 16:36:32.968985 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 16:36:32.968985 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 16:36:32.968985 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 01 16:36:32.968985 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application Aug 01 16:36:32.971092 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 224/440] 10.222.0.20 () {56 vars in 1114 bytes} [Mon Aug 1 16:36:32 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 16:36:32.977475 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e0b48b3-66fc-4d2a-8c5e-064afbeaf013 None admin] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:36:32.998509 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc097f6-784c-4ec0-b741-ffd20fa42801 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:36:32.999169 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc097f6-784c-4ec0-b741-ffd20fa42801 None admin] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:36:32.999454 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc097f6-784c-4ec0-b741-ffd20fa42801 None admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:36:33.000807 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc097f6-784c-4ec0-b741-ffd20fa42801 None admin] PATH_INFO: `/v3/groups/64b4bd12c620422ca16e3edfbdab7fe7` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:36:33.006589 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdc097f6-784c-4ec0-b741-ffd20fa42801 None admin] RBAC: Authorizing `identity:get_group(group_id=64b4bd12c620422ca16e3edfbdab7fe7)` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:36:33.006589 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdc097f6-784c-4ec0-b741-ffd20fa42801 None admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:36:33.011005 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 217/441] 10.222.0.20 () {56 vars in 1180 bytes} [Mon Aug 1 16:36:32 2022] GET /identity/v3/groups/64b4bd12c620422ca16e3edfbdab7fe7 => generated 226 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 16:36:33.012909 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc097f6-784c-4ec0-b741-ffd20fa42801 None admin] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:36:33.098245 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e0b48b3-66fc-4d2a-8c5e-064afbeaf013 None admin] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:36:33.124255 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc097f6-784c-4ec0-b741-ffd20fa42801 None admin] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:36:33.126632 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG dbcounter [-] [87057] Writing DB stats keystone:SELECT=100 {{(pid=87057) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:36:33.128253 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG dbcounter [-] [87058] Writing DB stats keystone:SELECT=100 {{(pid=87058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:36:33.137394 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-432f6a5a-740a-49a7-8bd8-019348db22d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:36:33.138018 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432f6a5a-740a-49a7-8bd8-019348db22d6 None admin] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:36:33.138328 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432f6a5a-740a-49a7-8bd8-019348db22d6 None admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:36:33.138617 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432f6a5a-740a-49a7-8bd8-019348db22d6 None admin] PATH_INFO: `/v3/roles` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:36:33.139622 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-432f6a5a-740a-49a7-8bd8-019348db22d6 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=87058) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:36:33.139622 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-432f6a5a-740a-49a7-8bd8-019348db22d6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:36:33.140981 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-432f6a5a-740a-49a7-8bd8-019348db22d6 None admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:36:33.149814 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 225/442] 10.222.0.20 () {56 vars in 1111 bytes} [Mon Aug 1 16:36:32 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 16:36:33.155092 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-432f6a5a-740a-49a7-8bd8-019348db22d6 None admin] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:36:33.160846 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaef7199-03f7-4a8a-80d1-c4a538ee71ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:36:33.161518 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaef7199-03f7-4a8a-80d1-c4a538ee71ff None admin] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:36:33.161797 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaef7199-03f7-4a8a-80d1-c4a538ee71ff None admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:36:33.162095 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaef7199-03f7-4a8a-80d1-c4a538ee71ff None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:36:33.162895 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaef7199-03f7-4a8a-80d1-c4a538ee71ff None admin] RBAC: Adding query filter params (group.id=64b4bd12c620422ca16e3edfbdab7fe7, role.id=0999d3da63a545f18ff3252afdc90518, scope.project.id=64c2a2f6f91d4b9c8d5993d28fd19631) {{(pid=87057) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:36:33.168229 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaef7199-03f7-4a8a-80d1-c4a538ee71ff None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:36:33.168229 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaef7199-03f7-4a8a-80d1-c4a538ee71ff None admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:36:33.171891 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 218/443] 10.222.0.20 () {56 vars in 1376 bytes} [Mon Aug 1 16:36:33 2022] GET /identity/v3/role_assignments?group.id=64b4bd12c620422ca16e3edfbdab7fe7&scope.project.id=64c2a2f6f91d4b9c8d5993d28fd19631&role.id=0999d3da63a545f18ff3252afdc90518 => generated 611 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:36:33.270005 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-432f6a5a-740a-49a7-8bd8-019348db22d6 None admin] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:36:33.307158 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c179c02-0983-4db7-8590-11a22fc4356e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:36:33.307796 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c179c02-0983-4db7-8590-11a22fc4356e None admin] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:36:33.308077 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c179c02-0983-4db7-8590-11a22fc4356e None admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:36:33.308349 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c179c02-0983-4db7-8590-11a22fc4356e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:36:33.336738 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9031954-7981-4904-80df-54a38713641c None None] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:36:33.337031 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9031954-7981-4904-80df-54a38713641c None None] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:36:33.337313 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9031954-7981-4904-80df-54a38713641c None None] PATH_INFO: `/` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:36:33.338730 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 219/444] 10.222.0.20 () {54 vars in 926 bytes} [Mon Aug 1 16:36:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 01 16:36:33.343941 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a49ce13-3550-4d3f-aba4-bf824cd35c3c None None] REQUEST_METHOD: `POST` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:36:33.344234 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a49ce13-3550-4d3f-aba4-bf824cd35c3c None None] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:36:33.344510 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a49ce13-3550-4d3f-aba4-bf824cd35c3c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:36:33.364866 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.auth.core [None req-6a49ce13-3550-4d3f-aba4-bf824cd35c3c None None] MFA Rules not processed for user `1007f8e379b74e169b98fb7c3c0ba5b4`. Rule list: `[]` (Enabled: `True`). {{(pid=87057) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:36:33.366355 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.fernet_utils [None req-6a49ce13-3550-4d3f-aba4-bf824cd35c3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87057) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:36:33.377812 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG dbcounter [-] [87058] Writing DB stats keystone:SELECT=100 {{(pid=87058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:36:33.389747 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c179c02-0983-4db7-8590-11a22fc4356e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:36:33.391259 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c179c02-0983-4db7-8590-11a22fc4356e None admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:36:33.452201 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 220/445] 10.222.0.20 () {58 vars in 1023 bytes} [Mon Aug 1 16:36:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 01 16:36:33.477044 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a49ce13-3550-4d3f-aba4-bf824cd35c3c None None] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:36:33.508260 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 226/446] 10.222.0.20 () {58 vars in 1282 bytes} [Mon Aug 1 16:36:33 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 353 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 01 16:36:33.513656 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c179c02-0983-4db7-8590-11a22fc4356e None admin] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:36:33.595169 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG dbcounter [-] [87057] Writing DB stats keystone:SELECT=100 {{(pid=87057) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:36:33.602246 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a49ce13-3550-4d3f-aba4-bf824cd35c3c None None] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:36:33.628491 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG dbcounter [-] [87058] Writing DB stats keystone:SELECT=100 {{(pid=87058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:36:33.641288 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c179c02-0983-4db7-8590-11a22fc4356e None admin] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:36:33.644583 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c3e313-8922-4c88-b6c3-acc95b0af9d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:36:33.645245 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c3e313-8922-4c88-b6c3-acc95b0af9d6 None admin] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:36:33.645526 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c3e313-8922-4c88-b6c3-acc95b0af9d6 None admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:36:33.645798 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c3e313-8922-4c88-b6c3-acc95b0af9d6 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:36:33.659293 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3c3e313-8922-4c88-b6c3-acc95b0af9d6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 16:36:33.665254 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application [None req-c3c3e313-8922-4c88-b6c3-acc95b0af9d6 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 16:36:33.665254 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 16:36:33.665254 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 16:36:33.665254 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 16:36:33.665254 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 16:36:33.665254 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 16:36:33.665254 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 16:36:33.665254 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 16:36:33.665254 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 16:36:33.665254 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 16:36:33.665254 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 16:36:33.665254 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 16:36:33.665254 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 16:36:33.665254 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 16:36:33.665254 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 16:36:33.665254 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 16:36:33.666265 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 16:36:33.666265 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 16:36:33.666265 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 16:36:33.666265 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 16:36:33.666265 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 16:36:33.666265 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 16:36:33.666265 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 16:36:33.666265 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 16:36:33.666265 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 16:36:33.666265 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 16:36:33.666265 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 16:36:33.666265 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application with Lock( Aug 01 16:36:33.666265 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 16:36:33.666265 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self._enter() Aug 01 16:36:33.666265 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 16:36:33.666265 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 16:36:33.667079 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 16:36:33.667079 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self.creator() Aug 01 16:36:33.667079 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 16:36:33.667079 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application created_value = creator( Aug 01 16:36:33.667079 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 16:36:33.667079 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 16:36:33.667079 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 16:36:33.667079 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 16:36:33.667079 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 16:36:33.667079 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 16:36:33.667079 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 16:36:33.667079 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application Aug 01 16:36:33.667697 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 221/447] 10.222.0.20 () {56 vars in 1096 bytes} [Mon Aug 1 16:36:33 2022] GET /identity/v3/roles/admin => generated 83 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 01 16:36:33.673692 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c3e313-8922-4c88-b6c3-acc95b0af9d6 None admin] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:36:33.681023 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4769a624-3217-464d-918a-769e1aee0dd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:36:33.681682 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4769a624-3217-464d-918a-769e1aee0dd0 None admin] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:36:33.681965 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4769a624-3217-464d-918a-769e1aee0dd0 None admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:36:33.682267 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4769a624-3217-464d-918a-769e1aee0dd0 None admin] PATH_INFO: `/v3/projects/d9a43129dd714d05bd409ffb0875f7e6` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:36:33.683448 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4769a624-3217-464d-918a-769e1aee0dd0 None admin] RBAC: Authorizing `identity:get_project(project_id=d9a43129dd714d05bd409ffb0875f7e6)` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:36:33.685000 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4769a624-3217-464d-918a-769e1aee0dd0 None admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:36:33.687022 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 227/448] 10.222.0.20 () {56 vars in 1186 bytes} [Mon Aug 1 16:36:33 2022] GET /identity/v3/projects/d9a43129dd714d05bd409ffb0875f7e6 => generated 302 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:36:33.692390 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4769a624-3217-464d-918a-769e1aee0dd0 None admin] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:36:33.808298 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c3e313-8922-4c88-b6c3-acc95b0af9d6 None admin] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:36:33.811236 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4769a624-3217-464d-918a-769e1aee0dd0 None admin] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:36:33.849372 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f753b32e-8cb3-4564-95cf-32818c23d3ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:36:33.849997 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f753b32e-8cb3-4564-95cf-32818c23d3ad None admin] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:36:33.850230 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd947384-06e7-4954-b634-dd4d0aac358d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:36:33.850336 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f753b32e-8cb3-4564-95cf-32818c23d3ad None admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:36:33.850704 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f753b32e-8cb3-4564-95cf-32818c23d3ad None admin] PATH_INFO: `/v3/roles` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:36:33.851491 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd947384-06e7-4954-b634-dd4d0aac358d None admin] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:36:33.851772 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd947384-06e7-4954-b634-dd4d0aac358d None admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:36:33.851948 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f753b32e-8cb3-4564-95cf-32818c23d3ad None admin] RBAC: Adding query filter params (name=admin) {{(pid=87057) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:36:33.852041 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd947384-06e7-4954-b634-dd4d0aac358d None admin] PATH_INFO: `/v3/groups/64b4bd12c620422ca16e3edfbdab7fe7` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:36:33.852254 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f753b32e-8cb3-4564-95cf-32818c23d3ad None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:36:33.856072 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f753b32e-8cb3-4564-95cf-32818c23d3ad None admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:36:33.856199 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd947384-06e7-4954-b634-dd4d0aac358d None admin] RBAC: Authorizing `identity:get_group(group_id=64b4bd12c620422ca16e3edfbdab7fe7)` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:36:33.856199 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd947384-06e7-4954-b634-dd4d0aac358d None admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:36:33.857212 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 228/449] 10.222.0.20 () {56 vars in 1180 bytes} [Mon Aug 1 16:36:33 2022] GET /identity/v3/groups/64b4bd12c620422ca16e3edfbdab7fe7 => generated 226 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 16:36:33.864025 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd947384-06e7-4954-b634-dd4d0aac358d None admin] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:36:33.864423 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 222/450] 10.222.0.20 () {56 vars in 1099 bytes} [Mon Aug 1 16:36:33 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 16:36:33.870921 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f753b32e-8cb3-4564-95cf-32818c23d3ad None admin] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:36:33.892291 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG dbcounter [-] [87058] Writing DB stats keystone:SELECT=100 {{(pid=87058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:36:33.893042 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG dbcounter [-] [87057] Writing DB stats keystone:SELECT=100 {{(pid=87057) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:36:33.967986 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd947384-06e7-4954-b634-dd4d0aac358d None admin] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:36:33.972182 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f753b32e-8cb3-4564-95cf-32818c23d3ad None admin] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:36:34.001834 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d593ea1-e363-4706-bd27-4fe2c7195cbc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:36:34.002474 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d593ea1-e363-4706-bd27-4fe2c7195cbc None admin] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:36:34.002755 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d593ea1-e363-4706-bd27-4fe2c7195cbc None admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:36:34.003030 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d593ea1-e363-4706-bd27-4fe2c7195cbc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:36:34.003815 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d593ea1-e363-4706-bd27-4fe2c7195cbc None admin] RBAC: Adding query filter params (group.id=64b4bd12c620422ca16e3edfbdab7fe7, role.id=3fac71a39f3f4ed198a3579064c30268, scope.project.id=d9a43129dd714d05bd409ffb0875f7e6) {{(pid=87058) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:36:34.004122 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d593ea1-e363-4706-bd27-4fe2c7195cbc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:36:34.005234 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be9e8d8b-cae7-4195-a76c-00703c1c7a9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:36:34.005648 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d593ea1-e363-4706-bd27-4fe2c7195cbc None admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:36:34.005847 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9e8d8b-cae7-4195-a76c-00703c1c7a9d None admin] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:36:34.006170 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9e8d8b-cae7-4195-a76c-00703c1c7a9d None admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:36:34.006448 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9e8d8b-cae7-4195-a76c-00703c1c7a9d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:36:34.012050 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 229/451] 10.222.0.20 () {56 vars in 1376 bytes} [Mon Aug 1 16:36:33 2022] GET /identity/v3/role_assignments?group.id=64b4bd12c620422ca16e3edfbdab7fe7&scope.project.id=d9a43129dd714d05bd409ffb0875f7e6&role.id=3fac71a39f3f4ed198a3579064c30268 => generated 611 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 16:36:34.078179 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be9e8d8b-cae7-4195-a76c-00703c1c7a9d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:36:34.079567 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be9e8d8b-cae7-4195-a76c-00703c1c7a9d None admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:36:34.101860 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG dbcounter [-] [87057] Writing DB stats keystone:SELECT=100 {{(pid=87057) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:36:34.186059 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 223/452] 10.222.0.20 () {58 vars in 1282 bytes} [Mon Aug 1 16:36:33 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 01 16:36:34.190100 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d593ea1-e363-4706-bd27-4fe2c7195cbc None admin] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:36:34.286485 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG dbcounter [-] [87058] Writing DB stats keystone:SELECT=100 {{(pid=87058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:36:34.296321 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d593ea1-e363-4706-bd27-4fe2c7195cbc None admin] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:36:34.331578 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75842e04-4afc-487c-9cf8-82f65168f73e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:36:34.332153 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75842e04-4afc-487c-9cf8-82f65168f73e None admin] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:36:34.332433 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75842e04-4afc-487c-9cf8-82f65168f73e None admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:36:34.332706 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75842e04-4afc-487c-9cf8-82f65168f73e None admin] PATH_INFO: `/v3/projects/3e43f22f36744d079b60704aff4969d4` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:36:34.333881 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75842e04-4afc-487c-9cf8-82f65168f73e None admin] RBAC: Authorizing `identity:get_project(project_id=3e43f22f36744d079b60704aff4969d4)` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:36:34.335535 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75842e04-4afc-487c-9cf8-82f65168f73e None admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:36:34.337391 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 230/453] 10.222.0.20 () {56 vars in 1186 bytes} [Mon Aug 1 16:36:34 2022] GET /identity/v3/projects/3e43f22f36744d079b60704aff4969d4 => generated 344 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 16:36:34.342977 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be9e8d8b-cae7-4195-a76c-00703c1c7a9d None admin] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:36:34.441229 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be9e8d8b-cae7-4195-a76c-00703c1c7a9d None admin] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:36:34.461361 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG dbcounter [-] [87057] Writing DB stats keystone:SELECT=100 {{(pid=87057) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:36:34.474530 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0848c17-39a9-44d4-875b-1a53926018a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:36:34.475164 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0848c17-39a9-44d4-875b-1a53926018a0 None admin] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:36:34.475443 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0848c17-39a9-44d4-875b-1a53926018a0 None admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:36:34.475719 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0848c17-39a9-44d4-875b-1a53926018a0 None admin] PATH_INFO: `/v3/groups/fb6971a50bbf408a8817d213299e6a64` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:36:34.477072 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0848c17-39a9-44d4-875b-1a53926018a0 None admin] RBAC: Authorizing `identity:get_group(group_id=fb6971a50bbf408a8817d213299e6a64)` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:36:34.478554 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0848c17-39a9-44d4-875b-1a53926018a0 None admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:36:34.480127 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 224/454] 10.222.0.20 () {56 vars in 1180 bytes} [Mon Aug 1 16:36:34 2022] GET /identity/v3/groups/fb6971a50bbf408a8817d213299e6a64 => generated 229 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:36:34.486192 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75842e04-4afc-487c-9cf8-82f65168f73e None admin] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:36:34.583621 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75842e04-4afc-487c-9cf8-82f65168f73e None admin] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:36:34.616041 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e6b0b7e-4f75-45a0-a435-c2dcaf015af5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:36:34.617265 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6b0b7e-4f75-45a0-a435-c2dcaf015af5 None admin] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:36:34.617545 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6b0b7e-4f75-45a0-a435-c2dcaf015af5 None admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:36:34.617818 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6b0b7e-4f75-45a0-a435-c2dcaf015af5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:36:34.618631 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e6b0b7e-4f75-45a0-a435-c2dcaf015af5 None admin] RBAC: Adding query filter params (group.id=fb6971a50bbf408a8817d213299e6a64, role.id=440bb943dd5b46fea038b903b6eaaf81, scope.project.id=3e43f22f36744d079b60704aff4969d4) {{(pid=87058) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:36:34.618940 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e6b0b7e-4f75-45a0-a435-c2dcaf015af5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:36:34.620354 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e6b0b7e-4f75-45a0-a435-c2dcaf015af5 None admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:36:34.625915 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 231/455] 10.222.0.20 () {56 vars in 1376 bytes} [Mon Aug 1 16:36:34 2022] GET /identity/v3/role_assignments?group.id=fb6971a50bbf408a8817d213299e6a64&scope.project.id=3e43f22f36744d079b60704aff4969d4&role.id=440bb943dd5b46fea038b903b6eaaf81 => generated 611 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:36:36.995469 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ae3d52-b7b0-4a75-aa2c-36fbb735c9df None None] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:36:36.996034 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ae3d52-b7b0-4a75-aa2c-36fbb735c9df None None] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:36:36.996034 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ae3d52-b7b0-4a75-aa2c-36fbb735c9df None None] PATH_INFO: `/` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:36:36.997520 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 225/456] 10.222.0.20 () {54 vars in 926 bytes} [Mon Aug 1 16:36:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 16:36:37.002913 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f441a92a-bce2-4224-97a6-cddddf88acce None None] REQUEST_METHOD: `POST` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:36:37.003299 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f441a92a-bce2-4224-97a6-cddddf88acce None None] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:36:37.003460 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f441a92a-bce2-4224-97a6-cddddf88acce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:36:37.026347 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.auth.core [None req-f441a92a-bce2-4224-97a6-cddddf88acce None None] MFA Rules not processed for user `1007f8e379b74e169b98fb7c3c0ba5b4`. Rule list: `[]` (Enabled: `True`). {{(pid=87058) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:36:37.027562 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.fernet_utils [None req-f441a92a-bce2-4224-97a6-cddddf88acce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:36:37.050950 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG dbcounter [-] [87058] Writing DB stats keystone:SELECT=100 {{(pid=87058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:36:37.117743 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 232/457] 10.222.0.20 () {58 vars in 1023 bytes} [Mon Aug 1 16:36:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 01 16:36:37.163886 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563c1047-91be-4204-ac81-348d1ed6c15d None None] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:36:37.164066 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563c1047-91be-4204-ac81-348d1ed6c15d None None] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:36:37.164346 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563c1047-91be-4204-ac81-348d1ed6c15d None None] PATH_INFO: `/` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:36:37.165836 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 226/458] 10.222.0.20 () {54 vars in 926 bytes} [Mon Aug 1 16:36:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 16:36:37.166745 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f441a92a-bce2-4224-97a6-cddddf88acce None None] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:36:37.179400 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3ad4d9-3804-40c9-950a-c5609cee647f None None] REQUEST_METHOD: `POST` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:36:37.179968 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3ad4d9-3804-40c9-950a-c5609cee647f None None] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:36:37.180458 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3ad4d9-3804-40c9-950a-c5609cee647f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:36:37.199472 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.auth.core [None req-4c3ad4d9-3804-40c9-950a-c5609cee647f None None] MFA Rules not processed for user `1007f8e379b74e169b98fb7c3c0ba5b4`. Rule list: `[]` (Enabled: `True`). {{(pid=87057) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:36:37.201235 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.fernet_utils [None req-4c3ad4d9-3804-40c9-950a-c5609cee647f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87057) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:36:37.294118 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 227/459] 10.222.0.20 () {58 vars in 1023 bytes} [Mon Aug 1 16:36:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 16:36:37.296390 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a094286f-30ee-4a3c-b397-cb24098419f8 None None] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:36:37.297064 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a094286f-30ee-4a3c-b397-cb24098419f8 None None] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:36:37.298786 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a094286f-30ee-4a3c-b397-cb24098419f8 None None] PATH_INFO: `/` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:36:37.300311 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 228/460] 10.222.0.20 () {54 vars in 926 bytes} [Mon Aug 1 16:36:37 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 16:36:37.310361 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14dbb58-47a8-4ac2-b182-fe216ee63b0b None None] REQUEST_METHOD: `POST` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:36:37.310898 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14dbb58-47a8-4ac2-b182-fe216ee63b0b None None] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:36:37.311456 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14dbb58-47a8-4ac2-b182-fe216ee63b0b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:36:37.333507 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.auth.core [None req-c14dbb58-47a8-4ac2-b182-fe216ee63b0b None None] MFA Rules not processed for user `1007f8e379b74e169b98fb7c3c0ba5b4`. Rule list: `[]` (Enabled: `True`). {{(pid=87057) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:36:37.334139 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f441a92a-bce2-4224-97a6-cddddf88acce None None] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:36:37.336013 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.fernet_utils [None req-c14dbb58-47a8-4ac2-b182-fe216ee63b0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87057) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:36:37.370248 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ace946-2a60-40a6-915a-48cabaefaa60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:36:37.370899 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ace946-2a60-40a6-915a-48cabaefaa60 None admin] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:36:37.371175 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ace946-2a60-40a6-915a-48cabaefaa60 None admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:36:37.371445 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ace946-2a60-40a6-915a-48cabaefaa60 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:36:37.382748 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56ace946-2a60-40a6-915a-48cabaefaa60 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:36:37.384238 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56ace946-2a60-40a6-915a-48cabaefaa60 None admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:36:37.400539 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application [None req-56ace946-2a60-40a6-915a-48cabaefaa60 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 16:36:37.400539 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 16:36:37.400539 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 16:36:37.400539 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 16:36:37.400539 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 16:36:37.400539 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 16:36:37.400539 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 16:36:37.400539 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 16:36:37.400539 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 16:36:37.400539 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 16:36:37.400539 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 16:36:37.400539 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 16:36:37.400539 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 16:36:37.400539 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 16:36:37.400539 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 16:36:37.401433 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 16:36:37.401433 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 16:36:37.401433 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 16:36:37.401433 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 16:36:37.401433 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 16:36:37.401433 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 16:36:37.401433 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 16:36:37.401433 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 16:36:37.401433 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application with Lock( Aug 01 16:36:37.401433 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 16:36:37.401433 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self._enter() Aug 01 16:36:37.401433 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 16:36:37.401433 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 16:36:37.401433 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 16:36:37.401433 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self.creator() Aug 01 16:36:37.401433 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 16:36:37.401433 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application created_value = creator( Aug 01 16:36:37.402304 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 16:36:37.402304 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 16:36:37.402304 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 16:36:37.402304 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application Aug 01 16:36:37.402304 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 233/461] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Aug 1 16:36:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 16:36:37.403062 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ace946-2a60-40a6-915a-48cabaefaa60 None admin] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:36:37.407132 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG dbcounter [-] [87058] Writing DB stats keystone:SELECT=100 {{(pid=87058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:36:37.419876 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 229/462] 10.222.0.20 () {58 vars in 1023 bytes} [Mon Aug 1 16:36:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 16:36:37.419876 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c14dbb58-47a8-4ac2-b182-fe216ee63b0b None None] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:36:37.468641 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG dbcounter [-] [87057] Writing DB stats keystone:SELECT=100 {{(pid=87057) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:36:37.524568 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c14dbb58-47a8-4ac2-b182-fe216ee63b0b None None] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:36:37.535578 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ace946-2a60-40a6-915a-48cabaefaa60 None admin] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:36:37.560640 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6228ef95-b30a-4270-be13-ef7ae80526cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:36:37.561272 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6228ef95-b30a-4270-be13-ef7ae80526cd None admin] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:36:37.561550 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6228ef95-b30a-4270-be13-ef7ae80526cd None admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:36:37.561820 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6228ef95-b30a-4270-be13-ef7ae80526cd None admin] PATH_INFO: `/v3/domains` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:36:37.562547 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6228ef95-b30a-4270-be13-ef7ae80526cd None admin] RBAC: Adding query filter params (name=Default) {{(pid=87057) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:36:37.562845 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6228ef95-b30a-4270-be13-ef7ae80526cd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:36:37.564213 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6228ef95-b30a-4270-be13-ef7ae80526cd None admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:36:37.572375 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c50e38e-1778-463e-8d57-9eba8e510d3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:36:37.573021 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c50e38e-1778-463e-8d57-9eba8e510d3f None admin] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:36:37.573299 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c50e38e-1778-463e-8d57-9eba8e510d3f None admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:36:37.573572 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c50e38e-1778-463e-8d57-9eba8e510d3f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:36:37.578363 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 230/463] 10.222.0.20 () {56 vars in 1109 bytes} [Mon Aug 1 16:36:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:36:37.579192 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6228ef95-b30a-4270-be13-ef7ae80526cd None admin] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:36:37.588531 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c50e38e-1778-463e-8d57-9eba8e510d3f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:36:37.590325 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c50e38e-1778-463e-8d57-9eba8e510d3f None admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:36:37.606198 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application [None req-1c50e38e-1778-463e-8d57-9eba8e510d3f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 16:36:37.606198 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 16:36:37.606198 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 16:36:37.606198 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 16:36:37.606198 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 16:36:37.606198 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 16:36:37.606198 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 16:36:37.606198 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 16:36:37.606198 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 16:36:37.606198 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 16:36:37.606198 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 16:36:37.606198 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 16:36:37.606198 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 16:36:37.606198 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 16:36:37.606198 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 16:36:37.607131 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 16:36:37.607131 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 16:36:37.607131 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 16:36:37.607131 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 16:36:37.607131 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 16:36:37.607131 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 16:36:37.607131 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 16:36:37.607131 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 16:36:37.607131 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application with Lock( Aug 01 16:36:37.607131 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 16:36:37.607131 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self._enter() Aug 01 16:36:37.607131 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 16:36:37.607131 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 16:36:37.607131 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 16:36:37.607131 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self.creator() Aug 01 16:36:37.607131 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 16:36:37.607131 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application created_value = creator( Aug 01 16:36:37.607983 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 16:36:37.607983 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 16:36:37.607983 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 16:36:37.607983 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application Aug 01 16:36:37.609964 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 234/464] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Aug 1 16:36:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 16:36:37.611027 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c50e38e-1778-463e-8d57-9eba8e510d3f None admin] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:36:37.671492 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG dbcounter [-] [87058] Writing DB stats keystone:SELECT=100 {{(pid=87058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:36:37.685566 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6228ef95-b30a-4270-be13-ef7ae80526cd None admin] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:36:37.695216 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG dbcounter [-] [87057] Writing DB stats keystone:SELECT=100 {{(pid=87057) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:36:37.720294 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c50e38e-1778-463e-8d57-9eba8e510d3f None admin] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:36:37.720920 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebdd7880-55a8-41c8-b70b-2c97ab3ca6c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:36:37.721568 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebdd7880-55a8-41c8-b70b-2c97ab3ca6c9 None admin] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:36:37.721842 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebdd7880-55a8-41c8-b70b-2c97ab3ca6c9 None admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:36:37.722137 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebdd7880-55a8-41c8-b70b-2c97ab3ca6c9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:36:37.733902 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebdd7880-55a8-41c8-b70b-2c97ab3ca6c9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:36:37.735480 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebdd7880-55a8-41c8-b70b-2c97ab3ca6c9 None admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:36:37.750816 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application [None req-ebdd7880-55a8-41c8-b70b-2c97ab3ca6c9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 16:36:37.750816 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 16:36:37.750816 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 16:36:37.750816 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 16:36:37.750816 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 16:36:37.750816 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 16:36:37.750816 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 16:36:37.750816 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 16:36:37.750816 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 16:36:37.750816 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 16:36:37.750816 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 16:36:37.750816 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 16:36:37.750816 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 16:36:37.750816 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 16:36:37.750816 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 16:36:37.752139 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 16:36:37.752139 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 16:36:37.752139 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 16:36:37.752139 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 16:36:37.752139 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 16:36:37.752139 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 16:36:37.752139 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 16:36:37.752139 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 16:36:37.752139 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application with Lock( Aug 01 16:36:37.752139 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 16:36:37.752139 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self._enter() Aug 01 16:36:37.752139 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 16:36:37.752139 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 16:36:37.752139 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 16:36:37.752139 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self.creator() Aug 01 16:36:37.752139 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 16:36:37.752139 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application created_value = creator( Aug 01 16:36:37.752974 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 16:36:37.752974 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 16:36:37.752974 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 16:36:37.752974 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application Aug 01 16:36:37.753567 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 231/465] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Aug 1 16:36:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 16:36:37.753921 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebdd7880-55a8-41c8-b70b-2c97ab3ca6c9 None admin] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:36:37.762308 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-054ba122-c3f7-489b-9ae4-f54f48f0e269 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:36:37.763222 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054ba122-c3f7-489b-9ae4-f54f48f0e269 None admin] REQUEST_METHOD: `POST` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:36:37.763725 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054ba122-c3f7-489b-9ae4-f54f48f0e269 None admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:36:37.764197 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054ba122-c3f7-489b-9ae4-f54f48f0e269 None admin] PATH_INFO: `/v3/users` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:36:37.765086 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-054ba122-c3f7-489b-9ae4-f54f48f0e269 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:36:37.767015 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-054ba122-c3f7-489b-9ae4-f54f48f0e269 None admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:36:37.863592 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 235/466] 10.222.0.20 () {60 vars in 1132 bytes} [Mon Aug 1 16:36:37 2022] POST /identity/v3/users => generated 246 bytes in 247 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 16:36:37.863592 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-054ba122-c3f7-489b-9ae4-f54f48f0e269 None admin] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:36:37.870391 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebdd7880-55a8-41c8-b70b-2c97ab3ca6c9 None admin] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:36:37.949269 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8098d470-9886-4dfe-9755-64311cbb1255 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:36:37.949961 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8098d470-9886-4dfe-9755-64311cbb1255 None admin] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:36:37.950264 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8098d470-9886-4dfe-9755-64311cbb1255 None admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:36:37.950531 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8098d470-9886-4dfe-9755-64311cbb1255 None admin] PATH_INFO: `/v3/domains` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:36:37.951239 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8098d470-9886-4dfe-9755-64311cbb1255 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87057) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:36:37.954596 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8098d470-9886-4dfe-9755-64311cbb1255 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:36:37.956020 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8098d470-9886-4dfe-9755-64311cbb1255 None admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:36:37.980329 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 232/467] 10.222.0.20 () {56 vars in 1109 bytes} [Mon Aug 1 16:36:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:36:37.985604 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8098d470-9886-4dfe-9755-64311cbb1255 None admin] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:36:38.031698 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG dbcounter [-] [87057] Writing DB stats keystone:SELECT=100 {{(pid=87057) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:36:38.081709 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-054ba122-c3f7-489b-9ae4-f54f48f0e269 None admin] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:36:38.097475 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG dbcounter [-] [87058] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:36:38.124418 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50af3f55-5353-425f-b659-13abd5ea9c8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:36:38.125046 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50af3f55-5353-425f-b659-13abd5ea9c8a None admin] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:36:38.125329 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50af3f55-5353-425f-b659-13abd5ea9c8a None admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:36:38.125606 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50af3f55-5353-425f-b659-13abd5ea9c8a None admin] PATH_INFO: `/v3/domains` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:36:38.126641 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50af3f55-5353-425f-b659-13abd5ea9c8a None admin] RBAC: Adding query filter params (name=Default) {{(pid=87058) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:36:38.126641 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50af3f55-5353-425f-b659-13abd5ea9c8a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:36:38.128295 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50af3f55-5353-425f-b659-13abd5ea9c8a None admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:36:38.140646 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8098d470-9886-4dfe-9755-64311cbb1255 None admin] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:36:38.146103 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 236/468] 10.222.0.20 () {56 vars in 1109 bytes} [Mon Aug 1 16:36:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 283 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:36:38.148232 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b170921-379b-419e-bce1-941616281321 None None] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:36:38.148368 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b170921-379b-419e-bce1-941616281321 None None] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:36:38.149112 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b170921-379b-419e-bce1-941616281321 None None] PATH_INFO: `/` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:36:38.151109 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 237/469] 10.222.0.20 () {54 vars in 926 bytes} [Mon Aug 1 16:36:38 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 16:36:38.155569 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b170921-379b-419e-bce1-941616281321 None None] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:36:38.184019 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b177f963-76de-4fd8-bac7-0143e2404f65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:36:38.184643 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b177f963-76de-4fd8-bac7-0143e2404f65 None admin] REQUEST_METHOD: `POST` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:36:38.184922 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b177f963-76de-4fd8-bac7-0143e2404f65 None admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:36:38.185195 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b177f963-76de-4fd8-bac7-0143e2404f65 None admin] PATH_INFO: `/v3/users` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:36:38.185878 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b177f963-76de-4fd8-bac7-0143e2404f65 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:36:38.187423 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b177f963-76de-4fd8-bac7-0143e2404f65 None admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:36:38.255662 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 233/470] 10.222.0.20 () {60 vars in 1132 bytes} [Mon Aug 1 16:36:37 2022] POST /identity/v3/users => generated 248 bytes in 270 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 16:36:38.266346 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0396c34-ba27-4057-b141-e5b8fa31dbdb None None] REQUEST_METHOD: `POST` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:36:38.268111 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0396c34-ba27-4057-b141-e5b8fa31dbdb None None] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:36:38.268111 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0396c34-ba27-4057-b141-e5b8fa31dbdb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:36:38.272999 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b170921-379b-419e-bce1-941616281321 None None] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:36:38.300216 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.auth.core [None req-a0396c34-ba27-4057-b141-e5b8fa31dbdb None None] MFA Rules not processed for user `1007f8e379b74e169b98fb7c3c0ba5b4`. Rule list: `[]` (Enabled: `True`). {{(pid=87057) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:36:38.305716 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.fernet_utils [None req-a0396c34-ba27-4057-b141-e5b8fa31dbdb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87057) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:36:38.323605 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2878023-2be5-41d3-9ee5-9b2666fc591e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:36:38.324952 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2878023-2be5-41d3-9ee5-9b2666fc591e None admin] REQUEST_METHOD: `POST` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:36:38.325386 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2878023-2be5-41d3-9ee5-9b2666fc591e None admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:36:38.325810 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2878023-2be5-41d3-9ee5-9b2666fc591e None admin] PATH_INFO: `/v3/users` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:36:38.326765 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2878023-2be5-41d3-9ee5-9b2666fc591e None admin] RBAC: Authorizing `identity:create_user()` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:36:38.328465 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2878023-2be5-41d3-9ee5-9b2666fc591e None admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:36:38.408815 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 234/471] 10.222.0.20 () {58 vars in 1023 bytes} [Mon Aug 1 16:36:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 16:36:38.437158 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0396c34-ba27-4057-b141-e5b8fa31dbdb None None] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:36:38.447727 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG dbcounter [-] [87057] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87057) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:36:38.520440 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 238/472] 10.222.0.20 () {60 vars in 1132 bytes} [Mon Aug 1 16:36:38 2022] POST /identity/v3/users => generated 248 bytes in 365 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 16:36:38.534643 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d94aa1d-2de7-41dc-be6b-ef57d5e09dd6 None None] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:36:38.534999 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d94aa1d-2de7-41dc-be6b-ef57d5e09dd6 None None] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:36:38.535249 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d94aa1d-2de7-41dc-be6b-ef57d5e09dd6 None None] PATH_INFO: `/` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:36:38.536794 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 239/473] 10.222.0.20 () {54 vars in 926 bytes} [Mon Aug 1 16:36:38 2022] GET /identity => generated 270 bytes in 16 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 16:36:38.545035 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c1e6c8-f16a-499d-9afd-bbaf8da897f8 None None] REQUEST_METHOD: `POST` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:36:38.545650 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c1e6c8-f16a-499d-9afd-bbaf8da897f8 None None] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:36:38.546199 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c1e6c8-f16a-499d-9afd-bbaf8da897f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:36:38.550459 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0396c34-ba27-4057-b141-e5b8fa31dbdb None None] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:36:38.570501 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.auth.core [None req-59c1e6c8-f16a-499d-9afd-bbaf8da897f8 None None] MFA Rules not processed for user `1007f8e379b74e169b98fb7c3c0ba5b4`. Rule list: `[]` (Enabled: `True`). {{(pid=87058) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:36:38.572731 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.fernet_utils [None req-59c1e6c8-f16a-499d-9afd-bbaf8da897f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:36:38.595215 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ee543f-a7e3-4ba3-9481-48888dde8c74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:36:38.595619 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ee543f-a7e3-4ba3-9481-48888dde8c74 None admin] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:36:38.595905 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ee543f-a7e3-4ba3-9481-48888dde8c74 None admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:36:38.596177 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ee543f-a7e3-4ba3-9481-48888dde8c74 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:36:38.609158 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG dbcounter [-] [87058] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:36:38.610202 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7ee543f-a7e3-4ba3-9481-48888dde8c74 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:36:38.611861 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7ee543f-a7e3-4ba3-9481-48888dde8c74 None admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:36:38.632871 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application [None req-f7ee543f-a7e3-4ba3-9481-48888dde8c74 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 16:36:38.632871 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 16:36:38.632871 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 16:36:38.632871 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 16:36:38.632871 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 16:36:38.632871 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 16:36:38.632871 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 16:36:38.632871 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 16:36:38.632871 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 16:36:38.632871 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 16:36:38.632871 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 16:36:38.632871 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 16:36:38.632871 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 16:36:38.632871 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 16:36:38.632871 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 16:36:38.633791 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 16:36:38.633791 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 16:36:38.633791 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 16:36:38.633791 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 16:36:38.633791 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 16:36:38.633791 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 16:36:38.633791 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 16:36:38.633791 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 16:36:38.633791 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application with Lock( Aug 01 16:36:38.633791 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 16:36:38.633791 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self._enter() Aug 01 16:36:38.633791 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 16:36:38.633791 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 16:36:38.633791 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 16:36:38.633791 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self.creator() Aug 01 16:36:38.633791 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 16:36:38.633791 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application created_value = creator( Aug 01 16:36:38.638822 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 16:36:38.638822 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 16:36:38.638822 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 16:36:38.638822 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application Aug 01 16:36:38.638822 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 235/474] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Aug 1 16:36:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 01 16:36:38.642699 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ee543f-a7e3-4ba3-9481-48888dde8c74 None admin] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:36:38.706057 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 240/475] 10.222.0.20 () {58 vars in 1023 bytes} [Mon Aug 1 16:36:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 16:36:38.719023 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG dbcounter [-] [87057] Writing DB stats keystone:SELECT=100 {{(pid=87057) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:36:38.735164 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59c1e6c8-f16a-499d-9afd-bbaf8da897f8 None None] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:36:38.770176 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ee543f-a7e3-4ba3-9481-48888dde8c74 None admin] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:36:38.817648 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77182371-b0d3-4297-b0ce-4b4851fd9483 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:36:38.819009 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77182371-b0d3-4297-b0ce-4b4851fd9483 None admin] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:36:38.820056 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77182371-b0d3-4297-b0ce-4b4851fd9483 None admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:36:38.820472 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77182371-b0d3-4297-b0ce-4b4851fd9483 None admin] PATH_INFO: `/v3/domains` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:36:38.821470 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77182371-b0d3-4297-b0ce-4b4851fd9483 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87057) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:36:38.822261 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77182371-b0d3-4297-b0ce-4b4851fd9483 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:36:38.824101 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77182371-b0d3-4297-b0ce-4b4851fd9483 None admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:36:38.842465 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 236/476] 10.222.0.20 () {56 vars in 1109 bytes} [Mon Aug 1 16:36:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:36:38.851571 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77182371-b0d3-4297-b0ce-4b4851fd9483 None admin] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:36:38.870653 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59c1e6c8-f16a-499d-9afd-bbaf8da897f8 None None] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:36:38.917827 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f76281dc-28fd-4240-a067-139056f9802a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:36:38.918999 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76281dc-28fd-4240-a067-139056f9802a None admin] REQUEST_METHOD: `POST` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:36:38.918999 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76281dc-28fd-4240-a067-139056f9802a None admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:36:38.918999 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76281dc-28fd-4240-a067-139056f9802a None admin] PATH_INFO: `/v3/roles` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:36:38.919728 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f76281dc-28fd-4240-a067-139056f9802a None admin] RBAC: Authorizing `identity:create_role()` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:36:38.921098 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f76281dc-28fd-4240-a067-139056f9802a None admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:36:38.976498 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.sql.core [None req-f76281dc-28fd-4240-a067-139056f9802a None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Aug 01 16:36:38.976498 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Aug 01 16:36:38.976498 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [parameters: {'id': 'a8b0350e33914c1aa5db10faf681f4a3', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Aug 01 16:36:38.976498 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=87058) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Aug 01 16:36:39.000479 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77182371-b0d3-4297-b0ce-4b4851fd9483 None admin] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:36:39.009334 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application [None req-f76281dc-28fd-4240-a067-139056f9802a None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Aug 01 16:36:39.009334 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 16:36:39.009334 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Aug 01 16:36:39.009334 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 01 16:36:39.009334 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 01 16:36:39.009334 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 01 16:36:39.009334 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 01 16:36:39.009334 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application result = self._query(query) Aug 01 16:36:39.009334 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 01 16:36:39.009334 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application conn.query(q) Aug 01 16:36:39.009334 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 01 16:36:39.009334 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 01 16:36:39.009334 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 01 16:36:39.009334 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application result.read() Aug 01 16:36:39.009334 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 01 16:36:39.010266 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 01 16:36:39.010266 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 01 16:36:39.010266 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 01 16:36:39.010266 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 01 16:36:39.010266 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 01 16:36:39.010266 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 01 16:36:39.010266 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 01 16:36:39.010266 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Aug 01 16:36:39.010266 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application Aug 01 16:36:39.010266 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Aug 01 16:36:39.010266 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application Aug 01 16:36:39.010266 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 16:36:39.010266 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Aug 01 16:36:39.010266 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return method(*args, **kwargs) Aug 01 16:36:39.010266 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role Aug 01 16:36:39.010266 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return ref.to_dict() Aug 01 16:36:39.010266 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 01 16:36:39.010266 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application next(self.gen) Aug 01 16:36:39.011225 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Aug 01 16:36:39.011225 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application yield resource Aug 01 16:36:39.011225 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 01 16:36:39.011225 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application next(self.gen) Aug 01 16:36:39.011225 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Aug 01 16:36:39.011225 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application self.session.rollback() Aug 01 16:36:39.011225 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Aug 01 16:36:39.011225 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application self.force_reraise() Aug 01 16:36:39.011225 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Aug 01 16:36:39.011225 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application raise self.value Aug 01 16:36:39.011225 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Aug 01 16:36:39.011225 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Aug 01 16:36:39.011225 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Aug 01 16:36:39.011225 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application session.commit() Aug 01 16:36:39.011225 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Aug 01 16:36:39.011225 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Aug 01 16:36:39.011225 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Aug 01 16:36:39.012113 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application self._prepare_impl() Aug 01 16:36:39.012113 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Aug 01 16:36:39.012113 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application self.session.flush() Aug 01 16:36:39.012113 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Aug 01 16:36:39.012113 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application self._flush(objects) Aug 01 16:36:39.012113 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Aug 01 16:36:39.012113 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Aug 01 16:36:39.012113 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Aug 01 16:36:39.012113 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application compat.raise_( Aug 01 16:36:39.012113 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 01 16:36:39.012113 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application raise exception Aug 01 16:36:39.012113 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Aug 01 16:36:39.012113 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application flush_context.execute() Aug 01 16:36:39.012113 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Aug 01 16:36:39.012113 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application rec.execute(self) Aug 01 16:36:39.012113 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Aug 01 16:36:39.012113 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Aug 01 16:36:39.020785 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Aug 01 16:36:39.020785 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application _emit_insert_statements( Aug 01 16:36:39.020785 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Aug 01 16:36:39.020785 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application c = connection._execute_20( Aug 01 16:36:39.020785 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Aug 01 16:36:39.020785 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Aug 01 16:36:39.020785 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Aug 01 16:36:39.020785 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Aug 01 16:36:39.020785 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Aug 01 16:36:39.020785 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application ret = self._execute_context( Aug 01 16:36:39.020785 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Aug 01 16:36:39.020785 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Aug 01 16:36:39.020785 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Aug 01 16:36:39.020785 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Aug 01 16:36:39.020785 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 01 16:36:39.020785 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application raise exception Aug 01 16:36:39.023600 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Aug 01 16:36:39.023600 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 01 16:36:39.023600 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 01 16:36:39.023600 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 01 16:36:39.023600 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 01 16:36:39.023600 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application result = self._query(query) Aug 01 16:36:39.023600 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 01 16:36:39.023600 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application conn.query(q) Aug 01 16:36:39.023600 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 01 16:36:39.023600 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 01 16:36:39.023600 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 01 16:36:39.023600 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application result.read() Aug 01 16:36:39.023600 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 01 16:36:39.023600 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 01 16:36:39.023600 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 01 16:36:39.023600 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 01 16:36:39.026570 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 01 16:36:39.026570 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 01 16:36:39.026570 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 01 16:36:39.026570 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 01 16:36:39.026570 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Aug 01 16:36:39.026570 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Aug 01 16:36:39.026570 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application [parameters: {'id': 'a8b0350e33914c1aa5db10faf681f4a3', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Aug 01 16:36:39.026570 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Aug 01 16:36:39.026570 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application Aug 01 16:36:39.026570 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 01 16:36:39.026570 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application Aug 01 16:36:39.026570 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 16:36:39.026570 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 16:36:39.026570 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 16:36:39.026570 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 16:36:39.026570 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 16:36:39.033450 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 16:36:39.033450 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 16:36:39.033450 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 16:36:39.033450 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 16:36:39.033450 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 16:36:39.033450 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 16:36:39.033450 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post Aug 01 16:36:39.033450 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( Aug 01 16:36:39.033450 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 16:36:39.033450 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 16:36:39.033450 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role Aug 01 16:36:39.033450 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) Aug 01 16:36:39.033450 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Aug 01 16:36:39.033450 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Aug 01 16:36:39.033450 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Aug 01 16:36:39.033450 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application Aug 01 16:36:39.034379 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 241/477] 10.222.0.20 () {60 vars in 1131 bytes} [Mon Aug 1 16:36:38 2022] POST /identity/v3/roles => generated 144 bytes in 276 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Aug 01 16:36:39.034379 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f76281dc-28fd-4240-a067-139056f9802a None admin] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:36:39.038947 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG dbcounter [-] [87057] Writing DB stats keystone:SELECT=100 {{(pid=87057) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:36:39.058787 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d46a9bc8-81d7-445a-a8a2-e80e32809222 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:36:39.059837 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46a9bc8-81d7-445a-a8a2-e80e32809222 None admin] REQUEST_METHOD: `POST` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:36:39.060285 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46a9bc8-81d7-445a-a8a2-e80e32809222 None admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:36:39.060726 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46a9bc8-81d7-445a-a8a2-e80e32809222 None admin] PATH_INFO: `/v3/users` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:36:39.062426 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d46a9bc8-81d7-445a-a8a2-e80e32809222 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:36:39.063632 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d46a9bc8-81d7-445a-a8a2-e80e32809222 None admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:36:39.085484 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG dbcounter [-] [87058] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:36:39.152688 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 237/478] 10.222.0.20 () {60 vars in 1132 bytes} [Mon Aug 1 16:36:38 2022] POST /identity/v3/users => generated 249 bytes in 302 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 16:36:39.186116 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f76281dc-28fd-4240-a067-139056f9802a None admin] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:36:39.235946 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb7492ac-a067-47ad-b7d1-3a03218ee2f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:36:39.237031 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7492ac-a067-47ad-b7d1-3a03218ee2f4 None admin] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:36:39.237530 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7492ac-a067-47ad-b7d1-3a03218ee2f4 None admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:36:39.238008 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7492ac-a067-47ad-b7d1-3a03218ee2f4 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:36:39.255686 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb7492ac-a067-47ad-b7d1-3a03218ee2f4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 16:36:39.262322 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application [None req-fb7492ac-a067-47ad-b7d1-3a03218ee2f4 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 01 16:36:39.262322 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 16:36:39.262322 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 16:36:39.262322 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 16:36:39.262322 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 16:36:39.262322 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 16:36:39.262322 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 16:36:39.262322 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 16:36:39.262322 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 16:36:39.262322 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 16:36:39.262322 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 16:36:39.262322 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 16:36:39.262322 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 16:36:39.262322 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 16:36:39.262322 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 16:36:39.262322 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 16:36:39.263307 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 16:36:39.263307 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 16:36:39.263307 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 16:36:39.263307 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 16:36:39.263307 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 16:36:39.263307 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 16:36:39.263307 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 16:36:39.263307 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 16:36:39.263307 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 16:36:39.263307 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 16:36:39.263307 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 16:36:39.263307 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application with Lock( Aug 01 16:36:39.263307 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 16:36:39.263307 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self._enter() Aug 01 16:36:39.263307 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 16:36:39.263307 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 16:36:39.264117 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 16:36:39.264117 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self.creator() Aug 01 16:36:39.264117 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 16:36:39.264117 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application created_value = creator( Aug 01 16:36:39.264117 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 16:36:39.264117 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 16:36:39.264117 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 16:36:39.264117 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 16:36:39.264117 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 16:36:39.264117 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 16:36:39.264117 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 01 16:36:39.264117 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application Aug 01 16:36:39.270800 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 242/479] 10.222.0.20 () {56 vars in 1114 bytes} [Mon Aug 1 16:36:39 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 254 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 01 16:36:39.276975 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d46a9bc8-81d7-445a-a8a2-e80e32809222 None admin] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:36:39.478095 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d46a9bc8-81d7-445a-a8a2-e80e32809222 None admin] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:36:39.521940 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f7086f-02a9-4267-8950-42d646f41630 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:36:39.523271 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f7086f-02a9-4267-8950-42d646f41630 None admin] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:36:39.523586 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f7086f-02a9-4267-8950-42d646f41630 None admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:36:39.523809 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f7086f-02a9-4267-8950-42d646f41630 None admin] PATH_INFO: `/v3/roles` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:36:39.524544 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3f7086f-02a9-4267-8950-42d646f41630 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=87057) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:36:39.524845 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3f7086f-02a9-4267-8950-42d646f41630 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:36:39.526229 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3f7086f-02a9-4267-8950-42d646f41630 None admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:36:39.534720 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 238/480] 10.222.0.20 () {56 vars in 1111 bytes} [Mon Aug 1 16:36:39 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:36:40.152811 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34fcfd03-6a65-4726-b6db-aeeb74b9f7d1 None None] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:36:40.153828 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34fcfd03-6a65-4726-b6db-aeeb74b9f7d1 None None] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:36:40.154134 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34fcfd03-6a65-4726-b6db-aeeb74b9f7d1 None None] PATH_INFO: `/` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:36:40.155694 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 243/481] 10.222.0.20 () {54 vars in 926 bytes} [Mon Aug 1 16:36:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 16:36:40.161250 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff03370-ae65-4bd6-9792-5e398c49556e None None] REQUEST_METHOD: `POST` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:36:40.162070 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff03370-ae65-4bd6-9792-5e398c49556e None None] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:36:40.162545 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff03370-ae65-4bd6-9792-5e398c49556e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:36:40.182263 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.auth.core [None req-9ff03370-ae65-4bd6-9792-5e398c49556e None None] MFA Rules not processed for user `1007f8e379b74e169b98fb7c3c0ba5b4`. Rule list: `[]` (Enabled: `True`). {{(pid=87057) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:36:40.183764 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.fernet_utils [None req-9ff03370-ae65-4bd6-9792-5e398c49556e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87057) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:36:40.223740 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG dbcounter [-] [87057] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87057) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:36:40.272591 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 239/482] 10.222.0.20 () {58 vars in 1023 bytes} [Mon Aug 1 16:36:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 16:36:40.296430 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34fcfd03-6a65-4726-b6db-aeeb74b9f7d1 None None] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:36:40.427835 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG dbcounter [-] [87058] Writing DB stats keystone:SELECT=100 {{(pid=87058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:36:40.443716 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34fcfd03-6a65-4726-b6db-aeeb74b9f7d1 None None] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:36:40.489552 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df5072d-c375-482a-9453-c084baafdcf0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:36:40.490225 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df5072d-c375-482a-9453-c084baafdcf0 None admin] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:36:40.490507 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df5072d-c375-482a-9453-c084baafdcf0 None admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:36:40.490783 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df5072d-c375-482a-9453-c084baafdcf0 None admin] PATH_INFO: `/v3/roles/service` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:36:40.507790 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3df5072d-c375-482a-9453-c084baafdcf0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 16:36:40.513042 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application [None req-3df5072d-c375-482a-9453-c084baafdcf0 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 01 16:36:40.513042 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 16:36:40.513042 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 16:36:40.513042 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 16:36:40.513042 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 16:36:40.513042 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 16:36:40.513042 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 16:36:40.513042 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 16:36:40.513042 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 16:36:40.513042 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 16:36:40.513042 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 16:36:40.513042 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 16:36:40.513042 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 16:36:40.513042 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 16:36:40.513042 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 16:36:40.513042 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 16:36:40.513905 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 16:36:40.513905 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 16:36:40.513905 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 16:36:40.513905 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 16:36:40.513905 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 16:36:40.513905 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 16:36:40.513905 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 16:36:40.513905 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 16:36:40.513905 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 16:36:40.513905 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 16:36:40.513905 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 16:36:40.513905 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application with Lock( Aug 01 16:36:40.513905 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 16:36:40.513905 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self._enter() Aug 01 16:36:40.513905 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 16:36:40.513905 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 16:36:40.515134 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 16:36:40.515134 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self.creator() Aug 01 16:36:40.515134 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 16:36:40.515134 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application created_value = creator( Aug 01 16:36:40.515134 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 16:36:40.515134 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 16:36:40.515134 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 16:36:40.515134 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 16:36:40.515134 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 16:36:40.515134 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 16:36:40.515134 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 01 16:36:40.515134 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application Aug 01 16:36:40.515134 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 244/483] 10.222.0.20 () {56 vars in 1102 bytes} [Mon Aug 1 16:36:40 2022] GET /identity/v3/roles/service => generated 85 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 16:36:40.523179 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ff03370-ae65-4bd6-9792-5e398c49556e None None] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:36:40.665255 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ff03370-ae65-4bd6-9792-5e398c49556e None None] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:36:40.711931 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1379e502-0c3f-43ac-a17c-58c3432f61e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:36:40.712479 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1379e502-0c3f-43ac-a17c-58c3432f61e1 None admin] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:36:40.712760 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1379e502-0c3f-43ac-a17c-58c3432f61e1 None admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:36:40.713038 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1379e502-0c3f-43ac-a17c-58c3432f61e1 None admin] PATH_INFO: `/v3/roles` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:36:40.713772 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1379e502-0c3f-43ac-a17c-58c3432f61e1 None admin] RBAC: Adding query filter params (name=service) {{(pid=87057) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:36:40.714095 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1379e502-0c3f-43ac-a17c-58c3432f61e1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:36:40.715471 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1379e502-0c3f-43ac-a17c-58c3432f61e1 None admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:36:40.729486 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 240/484] 10.222.0.20 () {56 vars in 1103 bytes} [Mon Aug 1 16:36:40 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:36:40.734997 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df5072d-c375-482a-9453-c084baafdcf0 None admin] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:36:40.830460 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2aad2c-3f82-47db-a222-cf7afa04fbc7 None None] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:36:40.830779 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2aad2c-3f82-47db-a222-cf7afa04fbc7 None None] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:36:40.831028 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2aad2c-3f82-47db-a222-cf7afa04fbc7 None None] PATH_INFO: `/` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:36:40.832630 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 241/485] 10.222.0.20 () {54 vars in 926 bytes} [Mon Aug 1 16:36:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 16:36:40.837475 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9c7220-7cd9-4ecb-82e6-9f81986390f1 None None] REQUEST_METHOD: `POST` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:36:40.837763 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9c7220-7cd9-4ecb-82e6-9f81986390f1 None None] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:36:40.838051 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9c7220-7cd9-4ecb-82e6-9f81986390f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:36:40.879541 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.auth.core [None req-5d9c7220-7cd9-4ecb-82e6-9f81986390f1 None None] MFA Rules not processed for user `1007f8e379b74e169b98fb7c3c0ba5b4`. Rule list: `[]` (Enabled: `True`). {{(pid=87057) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:36:40.881131 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.fernet_utils [None req-5d9c7220-7cd9-4ecb-82e6-9f81986390f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87057) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:36:40.882947 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df5072d-c375-482a-9453-c084baafdcf0 None admin] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:36:40.922729 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b124d21a-855b-4b20-b832-1531e8a31a7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:36:40.923378 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b124d21a-855b-4b20-b832-1531e8a31a7c None admin] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:36:40.923657 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b124d21a-855b-4b20-b832-1531e8a31a7c None admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:36:40.923925 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b124d21a-855b-4b20-b832-1531e8a31a7c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:36:40.945095 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b124d21a-855b-4b20-b832-1531e8a31a7c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:36:40.946732 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b124d21a-855b-4b20-b832-1531e8a31a7c None admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:36:40.948581 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG dbcounter [-] [87057] Writing DB stats keystone:SELECT=100 {{(pid=87057) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:36:40.951012 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG dbcounter [-] [87058] Writing DB stats keystone:SELECT=100 {{(pid=87058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:36:40.972696 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application [None req-b124d21a-855b-4b20-b832-1531e8a31a7c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 16:36:40.972696 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 16:36:40.972696 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 16:36:40.972696 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 16:36:40.972696 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 16:36:40.972696 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 16:36:40.972696 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 16:36:40.972696 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 16:36:40.972696 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 16:36:40.972696 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 16:36:40.972696 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 16:36:40.972696 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 16:36:40.972696 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 16:36:40.972696 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 16:36:40.972696 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 16:36:40.973929 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 16:36:40.973929 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 16:36:40.973929 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 16:36:40.973929 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 16:36:40.973929 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 16:36:40.973929 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 16:36:40.973929 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 16:36:40.973929 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 16:36:40.973929 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application with Lock( Aug 01 16:36:40.973929 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 16:36:40.973929 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self._enter() Aug 01 16:36:40.973929 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 16:36:40.973929 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 16:36:40.973929 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 16:36:40.973929 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self.creator() Aug 01 16:36:40.973929 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 16:36:40.973929 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application created_value = creator( Aug 01 16:36:40.974878 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 16:36:40.974878 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 16:36:40.974878 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 16:36:40.974878 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application Aug 01 16:36:40.974878 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 245/486] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Aug 1 16:36:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 16:36:40.974878 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f14053d-5bf5-427d-8566-f3265745fb08 None None] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:36:40.974878 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f14053d-5bf5-427d-8566-f3265745fb08 None None] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:36:40.974878 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f14053d-5bf5-427d-8566-f3265745fb08 None None] PATH_INFO: `/` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:36:40.974878 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 246/487] 10.222.0.20 () {54 vars in 926 bytes} [Mon Aug 1 16:36:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 16:36:40.975359 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f14053d-5bf5-427d-8566-f3265745fb08 None None] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:36:41.003794 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 242/488] 10.222.0.20 () {58 vars in 1023 bytes} [Mon Aug 1 16:36:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 16:36:41.005136 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27684381-05a2-4dad-9473-cc38e88e3f36 None None] REQUEST_METHOD: `POST` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:36:41.011319 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27684381-05a2-4dad-9473-cc38e88e3f36 None None] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:36:41.011676 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27684381-05a2-4dad-9473-cc38e88e3f36 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:36:41.048742 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.auth.core [None req-27684381-05a2-4dad-9473-cc38e88e3f36 None None] MFA Rules not processed for user `1007f8e379b74e169b98fb7c3c0ba5b4`. Rule list: `[]` (Enabled: `True`). {{(pid=87057) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:36:41.050054 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.fernet_utils [None req-27684381-05a2-4dad-9473-cc38e88e3f36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87057) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:36:41.103248 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f14053d-5bf5-427d-8566-f3265745fb08 None None] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:36:41.123293 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 243/489] 10.222.0.20 () {58 vars in 1023 bytes} [Mon Aug 1 16:36:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 16:36:41.124238 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27684381-05a2-4dad-9473-cc38e88e3f36 None None] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:36:41.157755 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aefb8b9-0a3e-4d2c-8775-9a2ce2d3a342 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:36:41.158548 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aefb8b9-0a3e-4d2c-8775-9a2ce2d3a342 None admin] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:36:41.158902 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aefb8b9-0a3e-4d2c-8775-9a2ce2d3a342 None admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:36:41.159246 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aefb8b9-0a3e-4d2c-8775-9a2ce2d3a342 None admin] PATH_INFO: `/v3/domains` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:36:41.160129 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aefb8b9-0a3e-4d2c-8775-9a2ce2d3a342 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87058) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:36:41.160512 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aefb8b9-0a3e-4d2c-8775-9a2ce2d3a342 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:36:41.162059 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aefb8b9-0a3e-4d2c-8775-9a2ce2d3a342 None admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:36:41.176528 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 247/490] 10.222.0.20 () {56 vars in 1109 bytes} [Mon Aug 1 16:36:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:36:41.177459 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aefb8b9-0a3e-4d2c-8775-9a2ce2d3a342 None admin] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:36:41.234579 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG dbcounter [-] [87057] Writing DB stats keystone:SELECT=100 {{(pid=87057) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:36:41.244370 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG dbcounter [-] [87058] Writing DB stats keystone:SELECT=100 {{(pid=87058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:36:41.248095 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27684381-05a2-4dad-9473-cc38e88e3f36 None None] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:36:41.284015 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d09297f7-63fe-4b2a-b26e-05407fd0d13c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:36:41.284706 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d09297f7-63fe-4b2a-b26e-05407fd0d13c None admin] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:36:41.285000 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d09297f7-63fe-4b2a-b26e-05407fd0d13c None admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:36:41.285282 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d09297f7-63fe-4b2a-b26e-05407fd0d13c None admin] PATH_INFO: `/v3/roles/service` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:36:41.292097 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aefb8b9-0a3e-4d2c-8775-9a2ce2d3a342 None admin] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:36:41.296373 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d09297f7-63fe-4b2a-b26e-05407fd0d13c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 16:36:41.301524 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application [None req-d09297f7-63fe-4b2a-b26e-05407fd0d13c None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 01 16:36:41.301524 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 16:36:41.301524 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 16:36:41.301524 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 16:36:41.301524 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 16:36:41.301524 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 16:36:41.301524 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 16:36:41.301524 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 16:36:41.301524 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 16:36:41.301524 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 16:36:41.301524 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 16:36:41.301524 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 16:36:41.301524 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 16:36:41.301524 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 16:36:41.301524 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 16:36:41.301524 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 16:36:41.302587 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 16:36:41.302587 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 16:36:41.302587 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 16:36:41.302587 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 16:36:41.302587 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 16:36:41.302587 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 16:36:41.302587 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 16:36:41.302587 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 16:36:41.302587 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 16:36:41.302587 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 16:36:41.302587 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 16:36:41.302587 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application with Lock( Aug 01 16:36:41.302587 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 16:36:41.302587 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self._enter() Aug 01 16:36:41.302587 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 16:36:41.302587 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 16:36:41.303386 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 16:36:41.303386 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self.creator() Aug 01 16:36:41.303386 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 16:36:41.303386 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application created_value = creator( Aug 01 16:36:41.303386 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 16:36:41.303386 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 16:36:41.303386 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 16:36:41.303386 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 16:36:41.303386 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 16:36:41.303386 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 16:36:41.303386 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 01 16:36:41.303386 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application Aug 01 16:36:41.303386 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 244/491] 10.222.0.20 () {56 vars in 1102 bytes} [Mon Aug 1 16:36:41 2022] GET /identity/v3/roles/service => generated 85 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 16:36:41.310072 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d09297f7-63fe-4b2a-b26e-05407fd0d13c None admin] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:36:41.336190 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9cca5d7-7c33-4427-bc67-e6d2c0150c21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:36:41.337080 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9cca5d7-7c33-4427-bc67-e6d2c0150c21 None admin] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:36:41.337506 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9cca5d7-7c33-4427-bc67-e6d2c0150c21 None admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:36:41.338071 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9cca5d7-7c33-4427-bc67-e6d2c0150c21 None admin] PATH_INFO: `/v3/roles/service` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:36:41.350001 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9cca5d7-7c33-4427-bc67-e6d2c0150c21 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 16:36:41.355647 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application [None req-c9cca5d7-7c33-4427-bc67-e6d2c0150c21 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 01 16:36:41.355647 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 16:36:41.355647 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 16:36:41.355647 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 16:36:41.355647 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 16:36:41.355647 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 16:36:41.355647 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 16:36:41.355647 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 16:36:41.355647 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 16:36:41.355647 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 16:36:41.355647 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 16:36:41.355647 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 16:36:41.355647 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 16:36:41.355647 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 16:36:41.355647 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 16:36:41.355647 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 16:36:41.356733 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 16:36:41.356733 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 16:36:41.356733 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 16:36:41.356733 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 16:36:41.356733 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 16:36:41.356733 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 16:36:41.356733 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 16:36:41.356733 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 16:36:41.356733 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 16:36:41.356733 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 16:36:41.356733 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 16:36:41.356733 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application with Lock( Aug 01 16:36:41.356733 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 16:36:41.356733 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self._enter() Aug 01 16:36:41.356733 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 16:36:41.356733 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 16:36:41.357691 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 16:36:41.357691 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self.creator() Aug 01 16:36:41.357691 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 16:36:41.357691 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application created_value = creator( Aug 01 16:36:41.357691 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 16:36:41.357691 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 16:36:41.357691 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 16:36:41.357691 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 16:36:41.357691 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 16:36:41.357691 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 16:36:41.357691 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 01 16:36:41.357691 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application Aug 01 16:36:41.364737 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 248/492] 10.222.0.20 () {56 vars in 1102 bytes} [Mon Aug 1 16:36:41 2022] GET /identity/v3/roles/service => generated 85 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 16:36:41.365757 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9cca5d7-7c33-4427-bc67-e6d2c0150c21 None admin] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:36:41.433877 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d09297f7-63fe-4b2a-b26e-05407fd0d13c None admin] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:36:41.483189 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f55290e-c3b3-4e1e-9f0d-b9a4a73c0a15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:36:41.483835 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f55290e-c3b3-4e1e-9f0d-b9a4a73c0a15 None admin] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:36:41.484117 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f55290e-c3b3-4e1e-9f0d-b9a4a73c0a15 None admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:36:41.484391 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f55290e-c3b3-4e1e-9f0d-b9a4a73c0a15 None admin] PATH_INFO: `/v3/users/nova` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:36:41.485549 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9cca5d7-7c33-4427-bc67-e6d2c0150c21 None admin] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:36:41.492466 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f55290e-c3b3-4e1e-9f0d-b9a4a73c0a15 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:36:41.494044 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f55290e-c3b3-4e1e-9f0d-b9a4a73c0a15 None admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:36:41.501119 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG dbcounter [-] [87058] Writing DB stats keystone:SELECT=100 {{(pid=87058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:36:41.508894 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application [None req-0f55290e-c3b3-4e1e-9f0d-b9a4a73c0a15 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 01 16:36:41.508894 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 16:36:41.508894 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 16:36:41.508894 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 16:36:41.508894 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 16:36:41.508894 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 16:36:41.508894 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 16:36:41.508894 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 16:36:41.508894 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 16:36:41.508894 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 16:36:41.508894 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 16:36:41.508894 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 16:36:41.508894 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 01 16:36:41.508894 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 01 16:36:41.508894 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 01 16:36:41.508894 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 01 16:36:41.510045 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 16:36:41.510045 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 16:36:41.510045 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 01 16:36:41.510045 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 16:36:41.510045 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 01 16:36:41.510045 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 16:36:41.510045 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 16:36:41.510045 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 16:36:41.510045 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 16:36:41.510045 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 16:36:41.510045 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 16:36:41.510045 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application with Lock( Aug 01 16:36:41.510045 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 16:36:41.510045 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self._enter() Aug 01 16:36:41.510045 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 16:36:41.510045 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 16:36:41.510045 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 16:36:41.510045 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self.creator() Aug 01 16:36:41.518069 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 16:36:41.518069 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application created_value = creator( Aug 01 16:36:41.518069 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 01 16:36:41.518069 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 01 16:36:41.518069 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 01 16:36:41.518069 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 01 16:36:41.518069 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 01 16:36:41.518069 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 01 16:36:41.518069 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 01 16:36:41.518069 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application Aug 01 16:36:41.518069 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 245/493] 10.222.0.20 () {56 vars in 1093 bytes} [Mon Aug 1 16:36:41 2022] GET /identity/v3/users/nova => generated 82 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 16:36:41.518069 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f55290e-c3b3-4e1e-9f0d-b9a4a73c0a15 None admin] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:36:41.524026 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f6787f-223f-4a11-83a4-786ed99fda66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:36:41.524657 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f6787f-223f-4a11-83a4-786ed99fda66 None admin] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:36:41.524940 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f6787f-223f-4a11-83a4-786ed99fda66 None admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:36:41.525215 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f6787f-223f-4a11-83a4-786ed99fda66 None admin] PATH_INFO: `/v3/roles` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:36:41.525950 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83f6787f-223f-4a11-83a4-786ed99fda66 None admin] RBAC: Adding query filter params (name=service) {{(pid=87058) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:36:41.526305 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83f6787f-223f-4a11-83a4-786ed99fda66 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:36:41.527758 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83f6787f-223f-4a11-83a4-786ed99fda66 None admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:36:41.528289 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG dbcounter [-] [87057] Writing DB stats keystone:SELECT=100 {{(pid=87057) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:36:41.542103 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 249/494] 10.222.0.20 () {56 vars in 1103 bytes} [Mon Aug 1 16:36:41 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:36:41.542103 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f6787f-223f-4a11-83a4-786ed99fda66 None admin] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:36:41.641270 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f55290e-c3b3-4e1e-9f0d-b9a4a73c0a15 None admin] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:36:41.654832 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f6787f-223f-4a11-83a4-786ed99fda66 None admin] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:36:41.677789 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05999b73-7f94-478a-8a32-957993c1f414 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:36:41.679458 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05999b73-7f94-478a-8a32-957993c1f414 None admin] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:36:41.679975 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05999b73-7f94-478a-8a32-957993c1f414 None admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:36:41.680448 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05999b73-7f94-478a-8a32-957993c1f414 None admin] PATH_INFO: `/v3/roles` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:36:41.681368 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05999b73-7f94-478a-8a32-957993c1f414 None admin] RBAC: Adding query filter params (name=service) {{(pid=87057) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:36:41.681870 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05999b73-7f94-478a-8a32-957993c1f414 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:36:41.683499 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05999b73-7f94-478a-8a32-957993c1f414 None admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:36:41.692138 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 246/495] 10.222.0.20 () {56 vars in 1103 bytes} [Mon Aug 1 16:36:41 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:36:41.693319 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05999b73-7f94-478a-8a32-957993c1f414 None admin] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:36:41.698962 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e60bcf5-6658-42c2-baf4-d641b782f008 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:36:41.699598 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e60bcf5-6658-42c2-baf4-d641b782f008 None admin] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:36:41.699880 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e60bcf5-6658-42c2-baf4-d641b782f008 None admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:36:41.700153 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e60bcf5-6658-42c2-baf4-d641b782f008 None admin] PATH_INFO: `/v3/users` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:36:41.701033 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e60bcf5-6658-42c2-baf4-d641b782f008 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=87058) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:36:41.701372 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e60bcf5-6658-42c2-baf4-d641b782f008 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:36:41.702862 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e60bcf5-6658-42c2-baf4-d641b782f008 None admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:36:41.744654 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 250/496] 10.222.0.20 () {56 vars in 1133 bytes} [Mon Aug 1 16:36:41 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:36:41.745415 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e60bcf5-6658-42c2-baf4-d641b782f008 None admin] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:36:41.769450 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG dbcounter [-] [87057] Writing DB stats keystone:SELECT=100 {{(pid=87057) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:36:41.783226 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG dbcounter [-] [87058] Writing DB stats keystone:SELECT=100 {{(pid=87058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:36:41.802579 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05999b73-7f94-478a-8a32-957993c1f414 None admin] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:36:41.840089 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b29b13a-f64a-462f-a87d-c2cd56d34106 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:36:41.841011 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b29b13a-f64a-462f-a87d-c2cd56d34106 None admin] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:36:41.841519 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b29b13a-f64a-462f-a87d-c2cd56d34106 None admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:36:41.842121 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b29b13a-f64a-462f-a87d-c2cd56d34106 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:36:41.851152 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e60bcf5-6658-42c2-baf4-d641b782f008 None admin] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:36:41.855158 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b29b13a-f64a-462f-a87d-c2cd56d34106 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:36:41.856660 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b29b13a-f64a-462f-a87d-c2cd56d34106 None admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:36:41.872701 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application [None req-5b29b13a-f64a-462f-a87d-c2cd56d34106 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 16:36:41.872701 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 16:36:41.872701 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 16:36:41.872701 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 16:36:41.872701 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 16:36:41.872701 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 16:36:41.872701 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 16:36:41.872701 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 16:36:41.872701 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 16:36:41.872701 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 16:36:41.872701 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 16:36:41.872701 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 16:36:41.872701 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 16:36:41.872701 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 16:36:41.872701 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 16:36:41.873580 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 16:36:41.873580 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 16:36:41.873580 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 16:36:41.873580 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 16:36:41.873580 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 16:36:41.873580 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 16:36:41.873580 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 16:36:41.873580 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 16:36:41.873580 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application with Lock( Aug 01 16:36:41.873580 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 16:36:41.873580 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self._enter() Aug 01 16:36:41.873580 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 16:36:41.873580 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 16:36:41.873580 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 16:36:41.873580 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self.creator() Aug 01 16:36:41.873580 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 16:36:41.873580 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application created_value = creator( Aug 01 16:36:41.874872 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 16:36:41.874872 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 16:36:41.874872 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 16:36:41.874872 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application Aug 01 16:36:41.874872 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 247/497] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Aug 1 16:36:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 16:36:41.874872 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b29b13a-f64a-462f-a87d-c2cd56d34106 None admin] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:36:41.892542 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a6433b-8ba1-4755-8091-b47aebbf568a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:36:41.893184 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a6433b-8ba1-4755-8091-b47aebbf568a None admin] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:36:41.893459 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a6433b-8ba1-4755-8091-b47aebbf568a None admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:36:41.893727 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a6433b-8ba1-4755-8091-b47aebbf568a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:36:41.908560 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34a6433b-8ba1-4755-8091-b47aebbf568a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:36:41.910485 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34a6433b-8ba1-4755-8091-b47aebbf568a None admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:36:41.926200 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application [None req-34a6433b-8ba1-4755-8091-b47aebbf568a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 16:36:41.926200 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 16:36:41.926200 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 16:36:41.926200 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 16:36:41.926200 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 16:36:41.926200 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 16:36:41.926200 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 16:36:41.926200 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 16:36:41.926200 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 16:36:41.926200 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 16:36:41.926200 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 16:36:41.926200 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 16:36:41.926200 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 16:36:41.926200 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 16:36:41.926200 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 16:36:41.927080 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 16:36:41.927080 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 16:36:41.927080 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 16:36:41.927080 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 16:36:41.927080 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 16:36:41.927080 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 16:36:41.927080 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 16:36:41.927080 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 16:36:41.927080 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application with Lock( Aug 01 16:36:41.927080 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 16:36:41.927080 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self._enter() Aug 01 16:36:41.927080 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 16:36:41.927080 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 16:36:41.927080 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 16:36:41.927080 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self.creator() Aug 01 16:36:41.927080 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 16:36:41.927080 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application created_value = creator( Aug 01 16:36:41.927930 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 16:36:41.927930 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 16:36:41.927930 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 16:36:41.927930 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application Aug 01 16:36:41.927930 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 251/498] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Aug 1 16:36:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 16:36:41.929298 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41e0b74-9a7d-42ab-b8cc-97d8fb36bf30 None None] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:36:41.929697 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41e0b74-9a7d-42ab-b8cc-97d8fb36bf30 None None] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:36:41.929973 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41e0b74-9a7d-42ab-b8cc-97d8fb36bf30 None None] PATH_INFO: `/` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:36:41.933068 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 252/499] 10.222.0.20 () {54 vars in 926 bytes} [Mon Aug 1 16:36:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 16:36:41.933068 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f41e0b74-9a7d-42ab-b8cc-97d8fb36bf30 None None] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:36:41.998747 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b29b13a-f64a-462f-a87d-c2cd56d34106 None admin] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:36:42.023070 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG dbcounter [-] [87058] Writing DB stats keystone:SELECT=100 {{(pid=87058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:36:42.024708 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG dbcounter [-] [87057] Writing DB stats keystone:SELECT=100 {{(pid=87057) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:36:42.037552 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b730bba-8bc6-4b38-993e-790a151df84c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:36:42.039033 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b730bba-8bc6-4b38-993e-790a151df84c None admin] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:36:42.039286 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f41e0b74-9a7d-42ab-b8cc-97d8fb36bf30 None None] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:36:42.039366 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b730bba-8bc6-4b38-993e-790a151df84c None admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:36:42.039578 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b730bba-8bc6-4b38-993e-790a151df84c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:36:42.074613 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77f0b8f9-afe0-4809-b4c6-4aa19edefe5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:36:42.075303 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f0b8f9-afe0-4809-b4c6-4aa19edefe5e None admin] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:36:42.075586 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f0b8f9-afe0-4809-b4c6-4aa19edefe5e None admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:36:42.075861 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f0b8f9-afe0-4809-b4c6-4aa19edefe5e None admin] PATH_INFO: `/v3/domains` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:36:42.076564 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77f0b8f9-afe0-4809-b4c6-4aa19edefe5e None admin] RBAC: Adding query filter params (name=Default) {{(pid=87058) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:36:42.079450 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77f0b8f9-afe0-4809-b4c6-4aa19edefe5e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:36:42.079450 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77f0b8f9-afe0-4809-b4c6-4aa19edefe5e None admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:36:42.095364 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 253/500] 10.222.0.20 () {56 vars in 1109 bytes} [Mon Aug 1 16:36:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:36:42.095364 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77f0b8f9-afe0-4809-b4c6-4aa19edefe5e None admin] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:36:42.113843 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b730bba-8bc6-4b38-993e-790a151df84c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:36:42.115511 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b730bba-8bc6-4b38-993e-790a151df84c None admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:36:42.201588 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77f0b8f9-afe0-4809-b4c6-4aa19edefe5e None admin] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:36:42.225246 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 248/501] 10.222.0.20 () {58 vars in 1282 bytes} [Mon Aug 1 16:36:41 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 346 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 01 16:36:42.225246 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77b244d-af76-4ae7-8687-299d0ec6d2d6 None None] REQUEST_METHOD: `POST` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:36:42.225246 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77b244d-af76-4ae7-8687-299d0ec6d2d6 None None] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:36:42.225246 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77b244d-af76-4ae7-8687-299d0ec6d2d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:36:42.241741 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0656e1d3-0299-4770-8174-ae6874ea6d25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:36:42.241976 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.auth.core [None req-c77b244d-af76-4ae7-8687-299d0ec6d2d6 None None] MFA Rules not processed for user `1007f8e379b74e169b98fb7c3c0ba5b4`. Rule list: `[]` (Enabled: `True`). {{(pid=87057) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:36:42.242418 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0656e1d3-0299-4770-8174-ae6874ea6d25 None admin] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:36:42.242650 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0656e1d3-0299-4770-8174-ae6874ea6d25 None admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:36:42.242913 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0656e1d3-0299-4770-8174-ae6874ea6d25 None admin] PATH_INFO: `/v3/domains` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:36:42.243089 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.fernet_utils [None req-c77b244d-af76-4ae7-8687-299d0ec6d2d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87057) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:36:42.243602 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0656e1d3-0299-4770-8174-ae6874ea6d25 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87058) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:36:42.243900 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0656e1d3-0299-4770-8174-ae6874ea6d25 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:36:42.245256 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0656e1d3-0299-4770-8174-ae6874ea6d25 None admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:36:42.258285 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 254/502] 10.222.0.20 () {56 vars in 1109 bytes} [Mon Aug 1 16:36:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:36:42.259061 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0656e1d3-0299-4770-8174-ae6874ea6d25 None admin] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:36:42.268453 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG dbcounter [-] [87058] Writing DB stats keystone:SELECT=100 {{(pid=87058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:36:42.269150 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG dbcounter [-] [87057] Writing DB stats keystone:SELECT=100 {{(pid=87057) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:36:42.334065 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 249/503] 10.222.0.20 () {58 vars in 1023 bytes} [Mon Aug 1 16:36:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 16:36:42.334065 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c77b244d-af76-4ae7-8687-299d0ec6d2d6 None None] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:36:42.374582 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0656e1d3-0299-4770-8174-ae6874ea6d25 None admin] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:36:42.408731 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d43d82-35cf-4c86-91d3-96d51c22ad2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:36:42.409370 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d43d82-35cf-4c86-91d3-96d51c22ad2f None admin] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:36:42.409650 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d43d82-35cf-4c86-91d3-96d51c22ad2f None admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:36:42.409924 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d43d82-35cf-4c86-91d3-96d51c22ad2f None admin] PATH_INFO: `/v3/users/glance` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:36:42.417608 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8d43d82-35cf-4c86-91d3-96d51c22ad2f None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:36:42.419086 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8d43d82-35cf-4c86-91d3-96d51c22ad2f None admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:36:42.432510 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application [None req-f8d43d82-35cf-4c86-91d3-96d51c22ad2f None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Aug 01 16:36:42.432510 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 16:36:42.432510 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 16:36:42.432510 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 16:36:42.432510 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 16:36:42.432510 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 16:36:42.432510 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 16:36:42.432510 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 16:36:42.432510 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 16:36:42.432510 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 16:36:42.432510 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 16:36:42.432510 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 16:36:42.432510 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 01 16:36:42.432510 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 01 16:36:42.432510 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 01 16:36:42.432510 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 01 16:36:42.433470 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 16:36:42.433470 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 16:36:42.433470 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 01 16:36:42.433470 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 16:36:42.433470 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 01 16:36:42.433470 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 16:36:42.433470 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 16:36:42.433470 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 16:36:42.433470 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 16:36:42.433470 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 16:36:42.433470 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 16:36:42.433470 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application with Lock( Aug 01 16:36:42.433470 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 16:36:42.433470 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self._enter() Aug 01 16:36:42.433470 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 16:36:42.433470 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 16:36:42.433470 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 16:36:42.433470 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self.creator() Aug 01 16:36:42.434392 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 16:36:42.434392 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application created_value = creator( Aug 01 16:36:42.434392 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 01 16:36:42.434392 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 01 16:36:42.434392 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 01 16:36:42.434392 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 01 16:36:42.434392 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 01 16:36:42.434392 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 01 16:36:42.434392 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Aug 01 16:36:42.434392 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application Aug 01 16:36:42.434392 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 255/504] 10.222.0.20 () {56 vars in 1099 bytes} [Mon Aug 1 16:36:42 2022] GET /identity/v3/users/glance => generated 84 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 16:36:42.434949 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d43d82-35cf-4c86-91d3-96d51c22ad2f None admin] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:36:42.440696 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c77b244d-af76-4ae7-8687-299d0ec6d2d6 None None] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:36:42.474249 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd90c566-7625-4648-bdb0-49cab5f5bae4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:36:42.474999 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd90c566-7625-4648-bdb0-49cab5f5bae4 None admin] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:36:42.475277 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd90c566-7625-4648-bdb0-49cab5f5bae4 None admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:36:42.476240 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd90c566-7625-4648-bdb0-49cab5f5bae4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:36:42.486205 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd90c566-7625-4648-bdb0-49cab5f5bae4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:36:42.487657 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd90c566-7625-4648-bdb0-49cab5f5bae4 None admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:36:42.498556 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG dbcounter [-] [87058] Writing DB stats keystone:SELECT=100 {{(pid=87058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:36:42.502849 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application [None req-cd90c566-7625-4648-bdb0-49cab5f5bae4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 16:36:42.502849 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 16:36:42.502849 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 16:36:42.502849 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 16:36:42.502849 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 16:36:42.502849 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 16:36:42.502849 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 16:36:42.502849 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 16:36:42.502849 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 16:36:42.502849 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 16:36:42.502849 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 16:36:42.502849 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 16:36:42.502849 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 16:36:42.502849 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 16:36:42.502849 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 16:36:42.509314 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 16:36:42.509314 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 16:36:42.509314 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 16:36:42.509314 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 16:36:42.509314 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 16:36:42.509314 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 16:36:42.509314 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 16:36:42.509314 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 16:36:42.509314 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application with Lock( Aug 01 16:36:42.509314 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 16:36:42.509314 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self._enter() Aug 01 16:36:42.509314 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 16:36:42.509314 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 16:36:42.509314 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 16:36:42.509314 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self.creator() Aug 01 16:36:42.509314 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 16:36:42.509314 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application created_value = creator( Aug 01 16:36:42.510225 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 16:36:42.510225 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 16:36:42.510225 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 16:36:42.510225 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application Aug 01 16:36:42.510225 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 250/505] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Aug 1 16:36:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 16:36:42.510225 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd90c566-7625-4648-bdb0-49cab5f5bae4 None admin] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:36:42.510225 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG dbcounter [-] [87057] Writing DB stats keystone:SELECT=100 {{(pid=87057) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:36:42.533627 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d43d82-35cf-4c86-91d3-96d51c22ad2f None admin] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:36:42.568088 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb86fd3-1292-40b2-b627-8a9b0d57eba5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:36:42.568730 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb86fd3-1292-40b2-b627-8a9b0d57eba5 None admin] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:36:42.569025 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb86fd3-1292-40b2-b627-8a9b0d57eba5 None admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:36:42.569972 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb86fd3-1292-40b2-b627-8a9b0d57eba5 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:36:42.577629 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bb86fd3-1292-40b2-b627-8a9b0d57eba5 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:36:42.579199 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bb86fd3-1292-40b2-b627-8a9b0d57eba5 None admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:36:42.590995 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application [None req-0bb86fd3-1292-40b2-b627-8a9b0d57eba5 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 01 16:36:42.590995 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 16:36:42.590995 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 16:36:42.590995 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 16:36:42.590995 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 16:36:42.590995 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 16:36:42.590995 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 16:36:42.590995 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 16:36:42.590995 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 16:36:42.590995 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 16:36:42.590995 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 16:36:42.590995 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 16:36:42.590995 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 01 16:36:42.590995 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 01 16:36:42.590995 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 01 16:36:42.590995 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 01 16:36:42.591985 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 16:36:42.591985 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 16:36:42.591985 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 01 16:36:42.591985 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 16:36:42.591985 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 01 16:36:42.591985 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 16:36:42.591985 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 16:36:42.591985 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 16:36:42.591985 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 16:36:42.591985 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 16:36:42.591985 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 16:36:42.591985 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application with Lock( Aug 01 16:36:42.591985 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 16:36:42.591985 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self._enter() Aug 01 16:36:42.591985 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 16:36:42.591985 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 16:36:42.591985 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 16:36:42.591985 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self.creator() Aug 01 16:36:42.592903 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 16:36:42.592903 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application created_value = creator( Aug 01 16:36:42.592903 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 01 16:36:42.592903 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 01 16:36:42.592903 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 01 16:36:42.592903 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 01 16:36:42.592903 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 01 16:36:42.592903 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 01 16:36:42.592903 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 01 16:36:42.592903 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application Aug 01 16:36:42.592903 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 256/506] 10.222.0.20 () {56 vars in 1099 bytes} [Mon Aug 1 16:36:42 2022] GET /identity/v3/users/cinder => generated 84 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 16:36:42.593534 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aca2fa7-d92a-40a7-8496-24c524faccfd None None] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:36:42.593662 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aca2fa7-d92a-40a7-8496-24c524faccfd None None] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:36:42.593937 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aca2fa7-d92a-40a7-8496-24c524faccfd None None] PATH_INFO: `/` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:36:42.595204 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 257/507] 10.222.0.20 () {54 vars in 926 bytes} [Mon Aug 1 16:36:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 16:36:42.595893 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aca2fa7-d92a-40a7-8496-24c524faccfd None None] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:36:42.609205 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd90c566-7625-4648-bdb0-49cab5f5bae4 None admin] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:36:42.643979 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45844f4d-ccd6-4046-b18a-5fde75d7acf2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:36:42.644611 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45844f4d-ccd6-4046-b18a-5fde75d7acf2 None admin] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:36:42.644878 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45844f4d-ccd6-4046-b18a-5fde75d7acf2 None admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:36:42.645148 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45844f4d-ccd6-4046-b18a-5fde75d7acf2 None admin] PATH_INFO: `/v3/roles/service` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:36:42.657922 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45844f4d-ccd6-4046-b18a-5fde75d7acf2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 16:36:42.662737 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application [None req-45844f4d-ccd6-4046-b18a-5fde75d7acf2 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 01 16:36:42.662737 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 16:36:42.662737 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 16:36:42.662737 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 16:36:42.662737 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 16:36:42.662737 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 16:36:42.662737 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 16:36:42.662737 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 16:36:42.662737 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 16:36:42.662737 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 16:36:42.662737 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 16:36:42.662737 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 16:36:42.662737 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 16:36:42.662737 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 16:36:42.662737 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 16:36:42.662737 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 16:36:42.664450 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 16:36:42.664450 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 16:36:42.664450 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 16:36:42.664450 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 16:36:42.664450 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 16:36:42.664450 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 16:36:42.664450 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 16:36:42.664450 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 16:36:42.664450 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 16:36:42.664450 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 16:36:42.664450 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 16:36:42.664450 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application with Lock( Aug 01 16:36:42.664450 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 16:36:42.664450 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self._enter() Aug 01 16:36:42.664450 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 16:36:42.664450 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 16:36:42.665241 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 16:36:42.665241 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self.creator() Aug 01 16:36:42.665241 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 16:36:42.665241 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application created_value = creator( Aug 01 16:36:42.665241 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 16:36:42.665241 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 16:36:42.665241 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 16:36:42.665241 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 16:36:42.665241 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 16:36:42.665241 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 16:36:42.665241 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 01 16:36:42.665241 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application Aug 01 16:36:42.665241 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 251/508] 10.222.0.20 () {56 vars in 1102 bytes} [Mon Aug 1 16:36:42 2022] GET /identity/v3/roles/service => generated 85 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 16:36:42.665905 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45844f4d-ccd6-4046-b18a-5fde75d7acf2 None admin] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:36:42.702251 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aca2fa7-d92a-40a7-8496-24c524faccfd None None] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:36:42.727990 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG dbcounter [-] [87058] Writing DB stats keystone:SELECT=100 {{(pid=87058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:36:42.728525 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG dbcounter [-] [87057] Writing DB stats keystone:SELECT=100 {{(pid=87057) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:36:42.736062 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a94b6282-6099-4a27-856d-9e9c14e72318 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:36:42.736666 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94b6282-6099-4a27-856d-9e9c14e72318 None admin] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:36:42.743020 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94b6282-6099-4a27-856d-9e9c14e72318 None admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:36:42.743020 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94b6282-6099-4a27-856d-9e9c14e72318 None admin] PATH_INFO: `/v3/users` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:36:42.743020 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a94b6282-6099-4a27-856d-9e9c14e72318 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=87058) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:36:42.743020 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a94b6282-6099-4a27-856d-9e9c14e72318 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:36:42.743020 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a94b6282-6099-4a27-856d-9e9c14e72318 None admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:36:42.756956 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 258/509] 10.222.0.20 () {56 vars in 1137 bytes} [Mon Aug 1 16:36:42 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:36:42.758232 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a94b6282-6099-4a27-856d-9e9c14e72318 None admin] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:36:42.766266 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45844f4d-ccd6-4046-b18a-5fde75d7acf2 None admin] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:36:42.803888 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bea72df-dc6a-4576-a01e-4c15beab165f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:36:42.804505 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bea72df-dc6a-4576-a01e-4c15beab165f None admin] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:36:42.804783 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bea72df-dc6a-4576-a01e-4c15beab165f None admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:36:42.805054 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bea72df-dc6a-4576-a01e-4c15beab165f None admin] PATH_INFO: `/v3/domains` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:36:42.805751 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bea72df-dc6a-4576-a01e-4c15beab165f None admin] RBAC: Adding query filter params (name=Default) {{(pid=87057) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:36:42.806088 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bea72df-dc6a-4576-a01e-4c15beab165f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:36:42.807461 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bea72df-dc6a-4576-a01e-4c15beab165f None admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:36:42.818931 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 252/510] 10.222.0.20 () {56 vars in 1109 bytes} [Mon Aug 1 16:36:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:36:42.820263 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60fcc5a-e6d6-4e41-95b6-ba120f1eb027 None None] REQUEST_METHOD: `POST` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:36:42.820544 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60fcc5a-e6d6-4e41-95b6-ba120f1eb027 None None] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:36:42.820813 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60fcc5a-e6d6-4e41-95b6-ba120f1eb027 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:36:42.838536 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.auth.core [None req-d60fcc5a-e6d6-4e41-95b6-ba120f1eb027 None None] MFA Rules not processed for user `1007f8e379b74e169b98fb7c3c0ba5b4`. Rule list: `[]` (Enabled: `True`). {{(pid=87057) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:36:42.839742 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.fernet_utils [None req-d60fcc5a-e6d6-4e41-95b6-ba120f1eb027 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87057) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:36:42.861441 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a94b6282-6099-4a27-856d-9e9c14e72318 None admin] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:36:42.895611 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11cfbcb9-2cf6-46bc-8c68-b91c11b9f302 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:36:42.896668 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11cfbcb9-2cf6-46bc-8c68-b91c11b9f302 None admin] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:36:42.896952 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11cfbcb9-2cf6-46bc-8c68-b91c11b9f302 None admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:36:42.897775 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11cfbcb9-2cf6-46bc-8c68-b91c11b9f302 None admin] PATH_INFO: `/v3/users` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:36:42.898570 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11cfbcb9-2cf6-46bc-8c68-b91c11b9f302 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=87058) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:36:42.898949 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11cfbcb9-2cf6-46bc-8c68-b91c11b9f302 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:36:42.900380 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11cfbcb9-2cf6-46bc-8c68-b91c11b9f302 None admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:36:42.909997 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 253/511] 10.222.0.20 () {58 vars in 1023 bytes} [Mon Aug 1 16:36:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 16:36:42.911120 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d60fcc5a-e6d6-4e41-95b6-ba120f1eb027 None None] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:36:42.929823 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 259/512] 10.222.0.20 () {56 vars in 1137 bytes} [Mon Aug 1 16:36:42 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:36:42.930540 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11cfbcb9-2cf6-46bc-8c68-b91c11b9f302 None admin] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:36:42.967012 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG dbcounter [-] [87057] Writing DB stats keystone:SELECT=100 {{(pid=87057) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:36:42.974628 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG dbcounter [-] [87058] Writing DB stats keystone:SELECT=100 {{(pid=87058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:36:43.011546 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d60fcc5a-e6d6-4e41-95b6-ba120f1eb027 None None] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:36:43.030623 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11cfbcb9-2cf6-46bc-8c68-b91c11b9f302 None admin] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:36:43.045488 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a14afc01-aea1-4e0e-b695-c0b6e80f3d26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:36:43.046127 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14afc01-aea1-4e0e-b695-c0b6e80f3d26 None admin] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:36:43.046406 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14afc01-aea1-4e0e-b695-c0b6e80f3d26 None admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:36:43.046676 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14afc01-aea1-4e0e-b695-c0b6e80f3d26 None admin] PATH_INFO: `/v3/roles` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:36:43.047386 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a14afc01-aea1-4e0e-b695-c0b6e80f3d26 None admin] RBAC: Adding query filter params (name=service) {{(pid=87057) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:36:43.047687 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a14afc01-aea1-4e0e-b695-c0b6e80f3d26 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:36:43.049081 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a14afc01-aea1-4e0e-b695-c0b6e80f3d26 None admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:36:43.056777 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 254/513] 10.222.0.20 () {56 vars in 1103 bytes} [Mon Aug 1 16:36:42 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:36:43.057553 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a14afc01-aea1-4e0e-b695-c0b6e80f3d26 None admin] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:36:43.064461 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-356b6df8-3d8e-471d-a0fd-30b75c5ad366 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:36:43.065089 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356b6df8-3d8e-471d-a0fd-30b75c5ad366 None admin] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:36:43.065368 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356b6df8-3d8e-471d-a0fd-30b75c5ad366 None admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:36:43.065640 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356b6df8-3d8e-471d-a0fd-30b75c5ad366 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:36:43.134337 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-356b6df8-3d8e-471d-a0fd-30b75c5ad366 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:36:43.135790 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-356b6df8-3d8e-471d-a0fd-30b75c5ad366 None admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:36:43.160876 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a14afc01-aea1-4e0e-b695-c0b6e80f3d26 None admin] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:36:43.176895 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG dbcounter [-] [87058] Writing DB stats keystone:SELECT=100 {{(pid=87058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:36:43.177269 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG dbcounter [-] [87057] Writing DB stats keystone:SELECT=100 {{(pid=87057) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:36:43.195002 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd5946d0-fe1c-450c-80ac-605d997602a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:36:43.195729 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5946d0-fe1c-450c-80ac-605d997602a3 None admin] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:36:43.196012 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5946d0-fe1c-450c-80ac-605d997602a3 None admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:36:43.196285 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5946d0-fe1c-450c-80ac-605d997602a3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:36:43.202761 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd5946d0-fe1c-450c-80ac-605d997602a3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:36:43.204231 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd5946d0-fe1c-450c-80ac-605d997602a3 None admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:36:43.217266 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application [None req-fd5946d0-fe1c-450c-80ac-605d997602a3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 16:36:43.217266 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 16:36:43.217266 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 16:36:43.217266 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 16:36:43.217266 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 16:36:43.217266 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 16:36:43.217266 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 16:36:43.217266 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 16:36:43.217266 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 16:36:43.217266 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 16:36:43.217266 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 16:36:43.217266 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 16:36:43.217266 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 01 16:36:43.217266 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 01 16:36:43.217266 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 01 16:36:43.218303 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 01 16:36:43.218303 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 16:36:43.218303 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 16:36:43.218303 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 16:36:43.218303 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 16:36:43.218303 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 16:36:43.218303 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 16:36:43.218303 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 16:36:43.218303 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application with Lock( Aug 01 16:36:43.218303 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 16:36:43.218303 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self._enter() Aug 01 16:36:43.218303 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 16:36:43.218303 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 16:36:43.218303 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 16:36:43.218303 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self.creator() Aug 01 16:36:43.218303 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 16:36:43.218303 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application created_value = creator( Aug 01 16:36:43.219260 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 01 16:36:43.219260 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 01 16:36:43.219260 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 01 16:36:43.219260 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 01 16:36:43.219260 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 01 16:36:43.219260 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 01 16:36:43.219260 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 16:36:43.219260 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application Aug 01 16:36:43.219260 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 255/514] 10.222.0.20 () {56 vars in 1111 bytes} [Mon Aug 1 16:36:43 2022] GET /identity/v3/projects/service => generated 88 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 16:36:43.219751 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd5946d0-fe1c-450c-80ac-605d997602a3 None admin] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:36:43.233621 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 260/515] 10.222.0.20 () {58 vars in 1282 bytes} [Mon Aug 1 16:36:42 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 303 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 01 16:36:43.238902 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-356b6df8-3d8e-471d-a0fd-30b75c5ad366 None admin] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:36:43.322182 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd5946d0-fe1c-450c-80ac-605d997602a3 None admin] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:36:43.335950 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-356b6df8-3d8e-471d-a0fd-30b75c5ad366 None admin] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:36:43.355177 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a95c24-b273-4b28-9008-0d44e50514e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:36:43.355816 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a95c24-b273-4b28-9008-0d44e50514e8 None admin] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:36:43.356096 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a95c24-b273-4b28-9008-0d44e50514e8 None admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:36:43.356369 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a95c24-b273-4b28-9008-0d44e50514e8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:36:43.366719 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68a95c24-b273-4b28-9008-0d44e50514e8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:36:43.368186 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68a95c24-b273-4b28-9008-0d44e50514e8 None admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:36:43.369702 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f527bb90-e585-4bcc-8f7f-ee24c7a9075f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:36:43.370343 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f527bb90-e585-4bcc-8f7f-ee24c7a9075f None admin] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:36:43.370620 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f527bb90-e585-4bcc-8f7f-ee24c7a9075f None admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:36:43.370890 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f527bb90-e585-4bcc-8f7f-ee24c7a9075f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:36:43.382760 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application [None req-68a95c24-b273-4b28-9008-0d44e50514e8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 16:36:43.382760 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 16:36:43.382760 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 16:36:43.382760 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 16:36:43.382760 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 16:36:43.382760 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 16:36:43.382760 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 16:36:43.382760 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 16:36:43.382760 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 16:36:43.382760 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 16:36:43.382760 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 16:36:43.382760 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 16:36:43.382760 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 16:36:43.382760 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 16:36:43.382760 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 16:36:43.383747 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 16:36:43.383747 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 16:36:43.383747 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 16:36:43.383747 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 16:36:43.383747 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 16:36:43.383747 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 16:36:43.383747 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 16:36:43.383747 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 16:36:43.383747 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application with Lock( Aug 01 16:36:43.383747 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 16:36:43.383747 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self._enter() Aug 01 16:36:43.383747 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 16:36:43.383747 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 16:36:43.383747 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 16:36:43.383747 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self.creator() Aug 01 16:36:43.383747 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 16:36:43.383747 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application created_value = creator( Aug 01 16:36:43.389256 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 16:36:43.389256 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 16:36:43.389256 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 16:36:43.389256 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application Aug 01 16:36:43.389256 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 256/516] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Aug 1 16:36:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 16:36:43.389256 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a95c24-b273-4b28-9008-0d44e50514e8 None admin] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:36:43.416051 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG dbcounter [-] [87057] Writing DB stats keystone:SELECT=100 {{(pid=87057) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:36:43.433877 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG dbcounter [-] [87058] Writing DB stats keystone:SELECT=100 {{(pid=87058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:36:43.486358 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a95c24-b273-4b28-9008-0d44e50514e8 None admin] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:36:43.489982 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f527bb90-e585-4bcc-8f7f-ee24c7a9075f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:36:43.492179 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f527bb90-e585-4bcc-8f7f-ee24c7a9075f None admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:36:43.521424 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2979b374-4a16-4c4e-b755-225cec0d8255 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:36:43.522079 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2979b374-4a16-4c4e-b755-225cec0d8255 None admin] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:36:43.522679 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2979b374-4a16-4c4e-b755-225cec0d8255 None admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:36:43.522949 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2979b374-4a16-4c4e-b755-225cec0d8255 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:36:43.532702 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2979b374-4a16-4c4e-b755-225cec0d8255 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:36:43.534154 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2979b374-4a16-4c4e-b755-225cec0d8255 None admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:36:43.548516 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application [None req-2979b374-4a16-4c4e-b755-225cec0d8255 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 16:36:43.548516 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 16:36:43.548516 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 16:36:43.548516 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 16:36:43.548516 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 16:36:43.548516 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 16:36:43.548516 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 16:36:43.548516 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 16:36:43.548516 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 16:36:43.548516 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 16:36:43.548516 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 16:36:43.548516 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 16:36:43.548516 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 16:36:43.548516 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 16:36:43.548516 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 16:36:43.549553 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 16:36:43.549553 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 16:36:43.549553 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 16:36:43.549553 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 16:36:43.549553 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 16:36:43.549553 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 16:36:43.549553 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 16:36:43.549553 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 16:36:43.549553 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application with Lock( Aug 01 16:36:43.549553 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 16:36:43.549553 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self._enter() Aug 01 16:36:43.549553 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 16:36:43.549553 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 16:36:43.549553 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 16:36:43.549553 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self.creator() Aug 01 16:36:43.549553 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 16:36:43.549553 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application created_value = creator( Aug 01 16:36:43.550609 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 16:36:43.550609 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 16:36:43.550609 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 16:36:43.550609 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application Aug 01 16:36:43.550609 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 257/517] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Aug 1 16:36:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 16:36:43.550946 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2979b374-4a16-4c4e-b755-225cec0d8255 None admin] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:36:43.595755 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 261/518] 10.222.0.20 () {58 vars in 1282 bytes} [Mon Aug 1 16:36:43 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 361 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 01 16:36:43.596368 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f527bb90-e585-4bcc-8f7f-ee24c7a9075f None admin] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:36:43.634357 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG dbcounter [-] [87057] Writing DB stats keystone:SELECT=100 {{(pid=87057) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:36:43.638895 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG dbcounter [-] [87058] Writing DB stats keystone:SELECT=100 {{(pid=87058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:36:43.652955 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2979b374-4a16-4c4e-b755-225cec0d8255 None admin] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:36:43.686874 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08654e87-0f31-4f95-aaf8-a020ae7ee2c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:36:43.687487 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08654e87-0f31-4f95-aaf8-a020ae7ee2c4 None admin] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:36:43.687767 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08654e87-0f31-4f95-aaf8-a020ae7ee2c4 None admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:36:43.688054 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08654e87-0f31-4f95-aaf8-a020ae7ee2c4 None admin] PATH_INFO: `/v3/projects` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:36:43.688804 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08654e87-0f31-4f95-aaf8-a020ae7ee2c4 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87057) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:36:43.689110 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08654e87-0f31-4f95-aaf8-a020ae7ee2c4 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:36:43.690592 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08654e87-0f31-4f95-aaf8-a020ae7ee2c4 None admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:36:43.693452 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f527bb90-e585-4bcc-8f7f-ee24c7a9075f None admin] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:36:43.708744 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 258/519] 10.222.0.20 () {56 vars in 1148 bytes} [Mon Aug 1 16:36:43 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:36:43.709566 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08654e87-0f31-4f95-aaf8-a020ae7ee2c4 None admin] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:36:43.732642 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b4f7f38-1ccc-4e78-a2da-982f9ec7df4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:36:43.733274 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4f7f38-1ccc-4e78-a2da-982f9ec7df4a None admin] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:36:43.733550 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4f7f38-1ccc-4e78-a2da-982f9ec7df4a None admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:36:43.733819 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4f7f38-1ccc-4e78-a2da-982f9ec7df4a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:36:43.744546 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b4f7f38-1ccc-4e78-a2da-982f9ec7df4a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:36:43.745994 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b4f7f38-1ccc-4e78-a2da-982f9ec7df4a None admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:36:43.762521 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application [None req-0b4f7f38-1ccc-4e78-a2da-982f9ec7df4a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 16:36:43.762521 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 16:36:43.762521 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 16:36:43.762521 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 16:36:43.762521 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 16:36:43.762521 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 16:36:43.762521 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 16:36:43.762521 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 16:36:43.762521 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 16:36:43.762521 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 16:36:43.762521 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 16:36:43.762521 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 16:36:43.762521 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 16:36:43.762521 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 16:36:43.762521 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 16:36:43.763537 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 16:36:43.763537 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 16:36:43.763537 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 16:36:43.763537 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 16:36:43.763537 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 16:36:43.763537 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 16:36:43.763537 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 16:36:43.763537 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 16:36:43.763537 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application with Lock( Aug 01 16:36:43.763537 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 16:36:43.763537 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self._enter() Aug 01 16:36:43.763537 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 16:36:43.763537 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 16:36:43.763537 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 16:36:43.763537 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self.creator() Aug 01 16:36:43.763537 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 16:36:43.763537 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application created_value = creator( Aug 01 16:36:43.764761 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 16:36:43.764761 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 16:36:43.764761 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 16:36:43.764761 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application Aug 01 16:36:43.764761 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 262/520] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Aug 1 16:36:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 16:36:43.765962 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b4f7f38-1ccc-4e78-a2da-982f9ec7df4a None admin] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:36:43.809944 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08654e87-0f31-4f95-aaf8-a020ae7ee2c4 None admin] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:36:43.844346 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbac1417-d261-4bb4-8153-3f355540c07a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:36:43.844956 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbac1417-d261-4bb4-8153-3f355540c07a None admin] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:36:43.845233 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbac1417-d261-4bb4-8153-3f355540c07a None admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:36:43.845499 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbac1417-d261-4bb4-8153-3f355540c07a None admin] PATH_INFO: `/v3/domains` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:36:43.846263 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbac1417-d261-4bb4-8153-3f355540c07a None admin] RBAC: Adding query filter params (name=Default) {{(pid=87057) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:36:43.846510 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbac1417-d261-4bb4-8153-3f355540c07a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:36:43.847905 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbac1417-d261-4bb4-8153-3f355540c07a None admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:36:43.857256 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG dbcounter [-] [87057] Writing DB stats keystone:SELECT=100 {{(pid=87057) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:36:43.859226 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG dbcounter [-] [87058] Writing DB stats keystone:SELECT=100 {{(pid=87058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:36:43.860626 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 259/521] 10.222.0.20 () {56 vars in 1109 bytes} [Mon Aug 1 16:36:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:36:43.861319 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbac1417-d261-4bb4-8153-3f355540c07a None admin] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:36:43.870692 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b4f7f38-1ccc-4e78-a2da-982f9ec7df4a None admin] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:36:43.906330 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a48679-217c-4b7a-9619-22e08dd45c70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:36:43.907153 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a48679-217c-4b7a-9619-22e08dd45c70 None admin] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:36:43.907604 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a48679-217c-4b7a-9619-22e08dd45c70 None admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:36:43.908228 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a48679-217c-4b7a-9619-22e08dd45c70 None admin] PATH_INFO: `/v3/domains` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:36:43.908932 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13a48679-217c-4b7a-9619-22e08dd45c70 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87058) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:36:43.909236 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13a48679-217c-4b7a-9619-22e08dd45c70 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:36:43.910646 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13a48679-217c-4b7a-9619-22e08dd45c70 None admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:36:43.922511 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 263/522] 10.222.0.20 () {56 vars in 1109 bytes} [Mon Aug 1 16:36:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:36:43.923303 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a48679-217c-4b7a-9619-22e08dd45c70 None admin] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:36:43.961326 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbac1417-d261-4bb4-8153-3f355540c07a None admin] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:36:43.996119 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d52f6f3-e4db-4d60-ad75-65f8a30a966d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:36:43.996754 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d52f6f3-e4db-4d60-ad75-65f8a30a966d None admin] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:36:43.997037 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d52f6f3-e4db-4d60-ad75-65f8a30a966d None admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:36:43.997307 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d52f6f3-e4db-4d60-ad75-65f8a30a966d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:36:44.007343 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d52f6f3-e4db-4d60-ad75-65f8a30a966d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:36:44.008800 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d52f6f3-e4db-4d60-ad75-65f8a30a966d None admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:36:44.021843 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a48679-217c-4b7a-9619-22e08dd45c70 None admin] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:36:44.024176 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application [None req-1d52f6f3-e4db-4d60-ad75-65f8a30a966d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 16:36:44.024176 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 16:36:44.024176 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 16:36:44.024176 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 16:36:44.024176 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 16:36:44.024176 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 16:36:44.024176 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 16:36:44.024176 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 16:36:44.024176 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 16:36:44.024176 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 16:36:44.024176 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 16:36:44.024176 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 16:36:44.024176 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 16:36:44.024176 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 16:36:44.024176 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 16:36:44.025183 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 16:36:44.025183 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 16:36:44.025183 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 16:36:44.025183 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 16:36:44.025183 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 16:36:44.025183 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 16:36:44.025183 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 16:36:44.025183 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 16:36:44.025183 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application with Lock( Aug 01 16:36:44.025183 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 16:36:44.025183 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self._enter() Aug 01 16:36:44.025183 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 16:36:44.025183 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 16:36:44.025183 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 16:36:44.025183 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self.creator() Aug 01 16:36:44.025183 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 16:36:44.025183 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application created_value = creator( Aug 01 16:36:44.026109 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 16:36:44.026109 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 16:36:44.026109 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 16:36:44.026109 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application Aug 01 16:36:44.026109 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 260/523] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Aug 1 16:36:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 16:36:44.026485 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d52f6f3-e4db-4d60-ad75-65f8a30a966d None admin] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:36:44.057922 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8836614-cac7-49d4-877d-9dfbc6279285 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:36:44.058572 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8836614-cac7-49d4-877d-9dfbc6279285 None admin] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:36:44.058849 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8836614-cac7-49d4-877d-9dfbc6279285 None admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:36:44.059121 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8836614-cac7-49d4-877d-9dfbc6279285 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:36:44.059898 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8836614-cac7-49d4-877d-9dfbc6279285 None admin] RBAC: Adding query filter params (role.id=51bb93e277c446b49bb80a32afcd3af7, scope.project.id=7b992d6a65df4bf9b27c13192d57c5f0, user.id=6533106d03eb4ca185a8839336d2aff9) {{(pid=87058) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:36:44.060204 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8836614-cac7-49d4-877d-9dfbc6279285 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:36:44.061641 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8836614-cac7-49d4-877d-9dfbc6279285 None admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:36:44.066922 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 264/524] 10.222.0.20 () {56 vars in 1374 bytes} [Mon Aug 1 16:36:43 2022] GET /identity/v3/role_assignments?user.id=6533106d03eb4ca185a8839336d2aff9&scope.project.id=7b992d6a65df4bf9b27c13192d57c5f0&role.id=51bb93e277c446b49bb80a32afcd3af7 => generated 259 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:36:44.067665 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8836614-cac7-49d4-877d-9dfbc6279285 None admin] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:36:44.080486 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG dbcounter [-] [87057] Writing DB stats keystone:SELECT=100 {{(pid=87057) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:36:44.089369 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG dbcounter [-] [87058] Writing DB stats keystone:SELECT=100 {{(pid=87058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:36:44.139581 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d52f6f3-e4db-4d60-ad75-65f8a30a966d None admin] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:36:44.177745 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa926b78-b187-460a-8e43-9e050e383a5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:36:44.179254 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa926b78-b187-460a-8e43-9e050e383a5a None admin] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:36:44.179601 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa926b78-b187-460a-8e43-9e050e383a5a None admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:36:44.180032 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa926b78-b187-460a-8e43-9e050e383a5a None admin] PATH_INFO: `/v3/domains` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:36:44.180824 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa926b78-b187-460a-8e43-9e050e383a5a None admin] RBAC: Adding query filter params (name=Default) {{(pid=87057) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:36:44.181203 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa926b78-b187-460a-8e43-9e050e383a5a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:36:44.182721 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8836614-cac7-49d4-877d-9dfbc6279285 None admin] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:36:44.182851 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa926b78-b187-460a-8e43-9e050e383a5a None admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:36:44.202869 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 261/525] 10.222.0.20 () {56 vars in 1109 bytes} [Mon Aug 1 16:36:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:36:44.202869 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa926b78-b187-460a-8e43-9e050e383a5a None admin] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:36:44.227917 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-380c033d-5c1f-474e-bef1-397b4823f838 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:36:44.228545 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380c033d-5c1f-474e-bef1-397b4823f838 None admin] REQUEST_METHOD: `POST` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:36:44.228824 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380c033d-5c1f-474e-bef1-397b4823f838 None admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:36:44.229103 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380c033d-5c1f-474e-bef1-397b4823f838 None admin] PATH_INFO: `/v3/users` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:36:44.229790 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-380c033d-5c1f-474e-bef1-397b4823f838 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:36:44.231531 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-380c033d-5c1f-474e-bef1-397b4823f838 None admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:36:44.308291 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa926b78-b187-460a-8e43-9e050e383a5a None admin] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:36:44.322059 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 265/526] 10.222.0.20 () {60 vars in 1132 bytes} [Mon Aug 1 16:36:44 2022] POST /identity/v3/users => generated 247 bytes in 246 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 16:36:44.322059 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-380c033d-5c1f-474e-bef1-397b4823f838 None admin] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:36:44.323681 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG dbcounter [-] [87057] Writing DB stats keystone:SELECT=100 {{(pid=87057) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:36:44.346479 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb953950-aa37-40d0-a62c-3465f6b98e53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:36:44.347197 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb953950-aa37-40d0-a62c-3465f6b98e53 None admin] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:36:44.347483 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb953950-aa37-40d0-a62c-3465f6b98e53 None admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:36:44.347759 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb953950-aa37-40d0-a62c-3465f6b98e53 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:36:44.356822 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb953950-aa37-40d0-a62c-3465f6b98e53 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:36:44.358476 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb953950-aa37-40d0-a62c-3465f6b98e53 None admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:36:44.371925 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application [None req-cb953950-aa37-40d0-a62c-3465f6b98e53 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Aug 01 16:36:44.371925 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 16:36:44.371925 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 16:36:44.371925 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 16:36:44.371925 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 16:36:44.371925 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 16:36:44.371925 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 16:36:44.371925 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 16:36:44.371925 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 16:36:44.371925 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 16:36:44.371925 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 16:36:44.371925 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 16:36:44.371925 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 01 16:36:44.371925 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 01 16:36:44.371925 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 01 16:36:44.371925 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 01 16:36:44.373004 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 16:36:44.373004 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 16:36:44.373004 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 01 16:36:44.373004 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 16:36:44.373004 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 01 16:36:44.373004 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 16:36:44.373004 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 16:36:44.373004 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 16:36:44.373004 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 16:36:44.373004 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 16:36:44.373004 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 16:36:44.373004 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application with Lock( Aug 01 16:36:44.373004 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 16:36:44.373004 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self._enter() Aug 01 16:36:44.373004 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 16:36:44.373004 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 16:36:44.373004 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 16:36:44.373004 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self.creator() Aug 01 16:36:44.374114 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 16:36:44.374114 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application created_value = creator( Aug 01 16:36:44.374114 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 01 16:36:44.374114 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 01 16:36:44.374114 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 01 16:36:44.374114 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 01 16:36:44.374114 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 01 16:36:44.374114 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 01 16:36:44.374114 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Aug 01 16:36:44.374114 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application Aug 01 16:36:44.377909 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 262/527] 10.222.0.20 () {56 vars in 1102 bytes} [Mon Aug 1 16:36:44 2022] GET /identity/v3/users/neutron => generated 85 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 16:36:44.378858 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb953950-aa37-40d0-a62c-3465f6b98e53 None admin] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:36:44.438954 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG dbcounter [-] [87058] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:36:44.482097 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-380c033d-5c1f-474e-bef1-397b4823f838 None admin] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:36:44.511229 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb953950-aa37-40d0-a62c-3465f6b98e53 None admin] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:36:44.525864 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5377e364-9672-49fa-b7f5-26c4a3a36a91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:36:44.526563 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5377e364-9672-49fa-b7f5-26c4a3a36a91 None admin] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:36:44.526846 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5377e364-9672-49fa-b7f5-26c4a3a36a91 None admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:36:44.527135 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5377e364-9672-49fa-b7f5-26c4a3a36a91 None admin] PATH_INFO: `/v3/domains` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:36:44.527831 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5377e364-9672-49fa-b7f5-26c4a3a36a91 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87058) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:36:44.528138 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5377e364-9672-49fa-b7f5-26c4a3a36a91 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:36:44.529516 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5377e364-9672-49fa-b7f5-26c4a3a36a91 None admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:36:44.542204 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 266/528] 10.222.0.20 () {56 vars in 1109 bytes} [Mon Aug 1 16:36:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:36:44.543047 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5377e364-9672-49fa-b7f5-26c4a3a36a91 None admin] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:36:44.555302 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50bd8010-d3e1-40c7-bd45-53554b49b056 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:36:44.556348 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50bd8010-d3e1-40c7-bd45-53554b49b056 None admin] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:36:44.556625 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50bd8010-d3e1-40c7-bd45-53554b49b056 None admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:36:44.556926 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50bd8010-d3e1-40c7-bd45-53554b49b056 None admin] PATH_INFO: `/v3/projects/service` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:36:44.565922 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50bd8010-d3e1-40c7-bd45-53554b49b056 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:36:44.567418 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50bd8010-d3e1-40c7-bd45-53554b49b056 None admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:36:44.581594 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application [None req-50bd8010-d3e1-40c7-bd45-53554b49b056 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 16:36:44.581594 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 16:36:44.581594 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 16:36:44.581594 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 16:36:44.581594 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 16:36:44.581594 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 16:36:44.581594 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 16:36:44.581594 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 16:36:44.581594 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 16:36:44.581594 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 16:36:44.581594 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 16:36:44.581594 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 16:36:44.581594 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 01 16:36:44.581594 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 01 16:36:44.581594 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 01 16:36:44.582584 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 01 16:36:44.582584 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 16:36:44.582584 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 16:36:44.582584 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 16:36:44.582584 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 16:36:44.582584 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 16:36:44.582584 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 16:36:44.582584 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 16:36:44.582584 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application with Lock( Aug 01 16:36:44.582584 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 16:36:44.582584 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self._enter() Aug 01 16:36:44.582584 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 16:36:44.582584 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 16:36:44.582584 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 16:36:44.582584 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self.creator() Aug 01 16:36:44.582584 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 16:36:44.582584 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application created_value = creator( Aug 01 16:36:44.583461 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 01 16:36:44.583461 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 01 16:36:44.583461 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 01 16:36:44.583461 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 01 16:36:44.583461 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 01 16:36:44.583461 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 01 16:36:44.583461 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 16:36:44.583461 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application Aug 01 16:36:44.583883 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 263/529] 10.222.0.20 () {56 vars in 1111 bytes} [Mon Aug 1 16:36:44 2022] GET /identity/v3/projects/service => generated 88 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 16:36:44.584620 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50bd8010-d3e1-40c7-bd45-53554b49b056 None admin] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:36:44.619484 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG dbcounter [-] [87057] Writing DB stats keystone:SELECT=100 {{(pid=87057) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:36:44.670599 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5377e364-9672-49fa-b7f5-26c4a3a36a91 None admin] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:36:44.703761 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50bd8010-d3e1-40c7-bd45-53554b49b056 None admin] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:36:44.719034 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-398cf028-38a7-4fb2-8750-28c0ca0e09ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:36:44.720027 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398cf028-38a7-4fb2-8750-28c0ca0e09ee None admin] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:36:44.720378 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398cf028-38a7-4fb2-8750-28c0ca0e09ee None admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:36:44.721187 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398cf028-38a7-4fb2-8750-28c0ca0e09ee None admin] PATH_INFO: `/v3/users` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:36:44.722353 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-398cf028-38a7-4fb2-8750-28c0ca0e09ee None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=87058) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:36:44.722743 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-398cf028-38a7-4fb2-8750-28c0ca0e09ee None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:36:44.724544 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-398cf028-38a7-4fb2-8750-28c0ca0e09ee None admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:36:44.727392 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG dbcounter [-] [87058] Writing DB stats keystone:SELECT=100 {{(pid=87058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:36:44.744072 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 267/530] 10.222.0.20 () {56 vars in 1139 bytes} [Mon Aug 1 16:36:44 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:36:44.745387 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-398cf028-38a7-4fb2-8750-28c0ca0e09ee None admin] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:36:44.749303 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19aaa07a-e0a1-491c-ad96-6e95070afc18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:36:44.749951 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19aaa07a-e0a1-491c-ad96-6e95070afc18 None admin] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:36:44.750262 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19aaa07a-e0a1-491c-ad96-6e95070afc18 None admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:36:44.750539 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19aaa07a-e0a1-491c-ad96-6e95070afc18 None admin] PATH_INFO: `/v3/projects/service` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:36:44.760142 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19aaa07a-e0a1-491c-ad96-6e95070afc18 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:36:44.762248 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19aaa07a-e0a1-491c-ad96-6e95070afc18 None admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:36:44.776900 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application [None req-19aaa07a-e0a1-491c-ad96-6e95070afc18 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 16:36:44.776900 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 16:36:44.776900 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 16:36:44.776900 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 16:36:44.776900 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 16:36:44.776900 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 16:36:44.776900 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 16:36:44.776900 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 16:36:44.776900 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 16:36:44.776900 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 16:36:44.776900 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 16:36:44.776900 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 16:36:44.776900 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 01 16:36:44.776900 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 01 16:36:44.776900 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 01 16:36:44.777823 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 01 16:36:44.777823 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 16:36:44.777823 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 16:36:44.777823 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 16:36:44.777823 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 16:36:44.777823 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 16:36:44.777823 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 16:36:44.777823 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 16:36:44.777823 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application with Lock( Aug 01 16:36:44.777823 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 16:36:44.777823 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self._enter() Aug 01 16:36:44.777823 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 16:36:44.777823 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 16:36:44.777823 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 16:36:44.777823 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self.creator() Aug 01 16:36:44.777823 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 16:36:44.777823 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application created_value = creator( Aug 01 16:36:44.779357 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 01 16:36:44.779357 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 01 16:36:44.779357 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 01 16:36:44.779357 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 01 16:36:44.779357 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 01 16:36:44.779357 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 01 16:36:44.779357 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 16:36:44.779357 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application Aug 01 16:36:44.779357 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 264/531] 10.222.0.20 () {56 vars in 1111 bytes} [Mon Aug 1 16:36:44 2022] GET /identity/v3/projects/service => generated 88 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 16:36:44.779357 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19aaa07a-e0a1-491c-ad96-6e95070afc18 None admin] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:36:44.863086 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-398cf028-38a7-4fb2-8750-28c0ca0e09ee None admin] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:36:44.899041 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG dbcounter [-] [87057] Writing DB stats keystone:SELECT=100 {{(pid=87057) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:36:44.905056 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcacbdb5-8fbf-4e79-a204-602f9ea68bc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:36:44.909041 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcacbdb5-8fbf-4e79-a204-602f9ea68bc4 None admin] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:36:44.909041 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcacbdb5-8fbf-4e79-a204-602f9ea68bc4 None admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:36:44.909041 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcacbdb5-8fbf-4e79-a204-602f9ea68bc4 None admin] PATH_INFO: `/v3/projects` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:36:44.909041 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcacbdb5-8fbf-4e79-a204-602f9ea68bc4 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87058) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:36:44.909041 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcacbdb5-8fbf-4e79-a204-602f9ea68bc4 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:36:44.909914 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcacbdb5-8fbf-4e79-a204-602f9ea68bc4 None admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:36:44.913768 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19aaa07a-e0a1-491c-ad96-6e95070afc18 None admin] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:36:44.933646 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 268/532] 10.222.0.20 () {56 vars in 1148 bytes} [Mon Aug 1 16:36:44 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:36:44.934607 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcacbdb5-8fbf-4e79-a204-602f9ea68bc4 None admin] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:36:44.953429 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3378199-fc91-4577-9e1c-677aa37fe590 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:36:44.954563 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3378199-fc91-4577-9e1c-677aa37fe590 None admin] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:36:44.955306 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3378199-fc91-4577-9e1c-677aa37fe590 None admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:36:44.955773 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3378199-fc91-4577-9e1c-677aa37fe590 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:36:45.006087 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG dbcounter [-] [87058] Writing DB stats keystone:SELECT=100 {{(pid=87058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:36:45.072178 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3378199-fc91-4577-9e1c-677aa37fe590 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:36:45.075504 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3378199-fc91-4577-9e1c-677aa37fe590 None admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:36:45.093257 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcacbdb5-8fbf-4e79-a204-602f9ea68bc4 None admin] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:36:45.151797 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b289a5-7a53-4429-bbf4-4e735f6335bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:36:45.153267 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b289a5-7a53-4429-bbf4-4e735f6335bf None admin] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:36:45.154176 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b289a5-7a53-4429-bbf4-4e735f6335bf None admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:36:45.154900 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b289a5-7a53-4429-bbf4-4e735f6335bf None admin] PATH_INFO: `/v3/projects` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:36:45.157070 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46b289a5-7a53-4429-bbf4-4e735f6335bf None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87058) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:36:45.157754 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46b289a5-7a53-4429-bbf4-4e735f6335bf None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:36:45.160638 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46b289a5-7a53-4429-bbf4-4e735f6335bf None admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:36:45.184413 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 269/533] 10.222.0.20 () {56 vars in 1148 bytes} [Mon Aug 1 16:36:44 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:36:45.186111 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b289a5-7a53-4429-bbf4-4e735f6335bf None admin] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:36:45.251449 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG dbcounter [-] [87057] Writing DB stats keystone:SELECT=100 {{(pid=87057) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:36:45.272594 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 265/534] 10.222.0.20 () {58 vars in 1282 bytes} [Mon Aug 1 16:36:44 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 489 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 01 16:36:45.272594 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3378199-fc91-4577-9e1c-677aa37fe590 None admin] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:36:45.347667 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b289a5-7a53-4429-bbf4-4e735f6335bf None admin] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:36:45.365235 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG dbcounter [-] [87058] Writing DB stats keystone:SELECT=100 {{(pid=87058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:36:45.392034 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6468ce93-4278-40db-9aa2-34dc446f1f0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:36:45.392648 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6468ce93-4278-40db-9aa2-34dc446f1f0c None admin] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:36:45.392928 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6468ce93-4278-40db-9aa2-34dc446f1f0c None admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:36:45.393228 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6468ce93-4278-40db-9aa2-34dc446f1f0c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:36:45.394172 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6468ce93-4278-40db-9aa2-34dc446f1f0c None admin] RBAC: Adding query filter params (role.id=51bb93e277c446b49bb80a32afcd3af7, scope.project.id=7b992d6a65df4bf9b27c13192d57c5f0, user.id=d80694e439e2401d9491712666409016) {{(pid=87058) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:36:45.394473 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6468ce93-4278-40db-9aa2-34dc446f1f0c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:36:45.396558 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6468ce93-4278-40db-9aa2-34dc446f1f0c None admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:36:45.401831 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3378199-fc91-4577-9e1c-677aa37fe590 None admin] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:36:45.403495 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 270/535] 10.222.0.20 () {56 vars in 1374 bytes} [Mon Aug 1 16:36:45 2022] GET /identity/v3/role_assignments?user.id=d80694e439e2401d9491712666409016&scope.project.id=7b992d6a65df4bf9b27c13192d57c5f0&role.id=51bb93e277c446b49bb80a32afcd3af7 => generated 259 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:36:45.404464 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6468ce93-4278-40db-9aa2-34dc446f1f0c None admin] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:36:45.444533 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a19c41-b1db-45f1-8d6d-7c3413a5bd50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:36:45.445662 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a19c41-b1db-45f1-8d6d-7c3413a5bd50 None admin] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:36:45.446221 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a19c41-b1db-45f1-8d6d-7c3413a5bd50 None admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:36:45.446686 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a19c41-b1db-45f1-8d6d-7c3413a5bd50 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:36:45.447805 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1a19c41-b1db-45f1-8d6d-7c3413a5bd50 None admin] RBAC: Adding query filter params (role.id=51bb93e277c446b49bb80a32afcd3af7, scope.project.id=7b992d6a65df4bf9b27c13192d57c5f0, user.id=25970b52ca15448fa1abb1f974d64038) {{(pid=87057) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:36:45.448517 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1a19c41-b1db-45f1-8d6d-7c3413a5bd50 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:36:45.450217 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1a19c41-b1db-45f1-8d6d-7c3413a5bd50 None admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:36:45.457410 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 266/536] 10.222.0.20 () {56 vars in 1374 bytes} [Mon Aug 1 16:36:45 2022] GET /identity/v3/role_assignments?user.id=25970b52ca15448fa1abb1f974d64038&scope.project.id=7b992d6a65df4bf9b27c13192d57c5f0&role.id=51bb93e277c446b49bb80a32afcd3af7 => generated 259 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:36:45.530429 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6468ce93-4278-40db-9aa2-34dc446f1f0c None admin] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:36:45.579275 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4602aaf9-d5d2-434d-b1dc-79158d71a619 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:36:45.579940 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4602aaf9-d5d2-434d-b1dc-79158d71a619 None admin] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:36:45.580221 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4602aaf9-d5d2-434d-b1dc-79158d71a619 None admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:36:45.580494 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4602aaf9-d5d2-434d-b1dc-79158d71a619 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:36:45.593379 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4602aaf9-d5d2-434d-b1dc-79158d71a619 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:36:45.594945 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4602aaf9-d5d2-434d-b1dc-79158d71a619 None admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:36:45.615556 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application [None req-4602aaf9-d5d2-434d-b1dc-79158d71a619 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 16:36:45.615556 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 16:36:45.615556 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 16:36:45.615556 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 16:36:45.615556 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 16:36:45.615556 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 16:36:45.615556 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 16:36:45.615556 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 16:36:45.615556 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 16:36:45.615556 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 16:36:45.615556 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 16:36:45.615556 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 16:36:45.615556 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 16:36:45.615556 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 16:36:45.615556 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 16:36:45.617544 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 16:36:45.617544 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 16:36:45.617544 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 16:36:45.617544 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 16:36:45.617544 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 16:36:45.617544 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 16:36:45.617544 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 16:36:45.617544 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 16:36:45.617544 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application with Lock( Aug 01 16:36:45.617544 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 16:36:45.617544 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self._enter() Aug 01 16:36:45.617544 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 16:36:45.617544 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 16:36:45.617544 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 16:36:45.617544 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self.creator() Aug 01 16:36:45.617544 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 16:36:45.617544 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application created_value = creator( Aug 01 16:36:45.618485 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 16:36:45.618485 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 16:36:45.618485 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 16:36:45.618485 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application Aug 01 16:36:45.618485 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 271/537] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Aug 1 16:36:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 16:36:45.623313 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a19c41-b1db-45f1-8d6d-7c3413a5bd50 None admin] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:36:45.686182 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG dbcounter [-] [87057] Writing DB stats keystone:SELECT=100 {{(pid=87057) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:36:45.743489 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a19c41-b1db-45f1-8d6d-7c3413a5bd50 None admin] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:36:45.780127 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1481ec49-3738-431d-958e-ca1d4056b1b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:36:45.780801 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1481ec49-3738-431d-958e-ca1d4056b1b9 None admin] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:36:45.781051 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1481ec49-3738-431d-958e-ca1d4056b1b9 None admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:36:45.781269 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1481ec49-3738-431d-958e-ca1d4056b1b9 None admin] PATH_INFO: `/v3/domains` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:36:45.781968 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1481ec49-3738-431d-958e-ca1d4056b1b9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87057) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:36:45.782298 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1481ec49-3738-431d-958e-ca1d4056b1b9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:36:45.783661 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1481ec49-3738-431d-958e-ca1d4056b1b9 None admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:36:45.801308 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 267/538] 10.222.0.20 () {56 vars in 1109 bytes} [Mon Aug 1 16:36:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:36:45.802682 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4602aaf9-d5d2-434d-b1dc-79158d71a619 None admin] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:36:45.844902 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG dbcounter [-] [87058] Writing DB stats keystone:SELECT=100 {{(pid=87058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:36:45.940517 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4602aaf9-d5d2-434d-b1dc-79158d71a619 None admin] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:36:45.981069 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbefb479-32ee-4d73-aea0-a2bcd873d7bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:36:45.981719 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbefb479-32ee-4d73-aea0-a2bcd873d7bc None admin] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:36:45.982003 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbefb479-32ee-4d73-aea0-a2bcd873d7bc None admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:36:45.982299 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbefb479-32ee-4d73-aea0-a2bcd873d7bc None admin] PATH_INFO: `/v3/projects/service` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:36:45.989390 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbefb479-32ee-4d73-aea0-a2bcd873d7bc None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:36:45.991439 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbefb479-32ee-4d73-aea0-a2bcd873d7bc None admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:36:46.010595 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application [None req-dbefb479-32ee-4d73-aea0-a2bcd873d7bc None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 16:36:46.010595 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 16:36:46.010595 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 16:36:46.010595 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 16:36:46.010595 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 16:36:46.010595 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 16:36:46.010595 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 16:36:46.010595 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 16:36:46.010595 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 16:36:46.010595 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 16:36:46.010595 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 16:36:46.010595 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 16:36:46.010595 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 01 16:36:46.010595 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 01 16:36:46.010595 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 01 16:36:46.011485 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 01 16:36:46.011485 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 16:36:46.011485 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 16:36:46.011485 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 16:36:46.011485 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 16:36:46.011485 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 16:36:46.011485 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 16:36:46.011485 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 16:36:46.011485 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application with Lock( Aug 01 16:36:46.011485 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 16:36:46.011485 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self._enter() Aug 01 16:36:46.011485 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 16:36:46.011485 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 16:36:46.011485 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 16:36:46.011485 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self.creator() Aug 01 16:36:46.011485 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 16:36:46.011485 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application created_value = creator( Aug 01 16:36:46.012332 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 01 16:36:46.012332 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 01 16:36:46.012332 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 01 16:36:46.012332 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 01 16:36:46.012332 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 01 16:36:46.012332 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 01 16:36:46.012332 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 16:36:46.012332 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application Aug 01 16:36:46.012332 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 272/539] 10.222.0.20 () {56 vars in 1111 bytes} [Mon Aug 1 16:36:45 2022] GET /identity/v3/projects/service => generated 88 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 16:36:46.017908 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1481ec49-3738-431d-958e-ca1d4056b1b9 None admin] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:36:46.171218 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1481ec49-3738-431d-958e-ca1d4056b1b9 None admin] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:36:46.193584 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG dbcounter [-] [87057] Writing DB stats keystone:SELECT=100 {{(pid=87057) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:36:46.219739 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f829226-f17c-4a55-996f-235550f9de43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:36:46.220362 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f829226-f17c-4a55-996f-235550f9de43 None admin] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:36:46.220642 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f829226-f17c-4a55-996f-235550f9de43 None admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:36:46.220905 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f829226-f17c-4a55-996f-235550f9de43 None admin] PATH_INFO: `/v3/projects` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:36:46.221658 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f829226-f17c-4a55-996f-235550f9de43 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87057) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:36:46.221957 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f829226-f17c-4a55-996f-235550f9de43 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:36:46.223408 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f829226-f17c-4a55-996f-235550f9de43 None admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:36:46.238221 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 268/540] 10.222.0.20 () {56 vars in 1148 bytes} [Mon Aug 1 16:36:46 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:36:46.244970 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbefb479-32ee-4d73-aea0-a2bcd873d7bc None admin] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:36:46.360953 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG dbcounter [-] [87058] Writing DB stats keystone:SELECT=100 {{(pid=87058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:36:46.376919 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbefb479-32ee-4d73-aea0-a2bcd873d7bc None admin] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:36:46.424891 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22b3c187-5bef-4fd0-a2a9-fd3298b8abc3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:36:46.425423 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b3c187-5bef-4fd0-a2a9-fd3298b8abc3 None admin] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:36:46.425704 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b3c187-5bef-4fd0-a2a9-fd3298b8abc3 None admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:36:46.425976 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b3c187-5bef-4fd0-a2a9-fd3298b8abc3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:36:46.426788 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22b3c187-5bef-4fd0-a2a9-fd3298b8abc3 None admin] RBAC: Adding query filter params (role.id=51bb93e277c446b49bb80a32afcd3af7, scope.project.id=7b992d6a65df4bf9b27c13192d57c5f0, user.id=3d9bd8d649154c979be3744bfd016f95) {{(pid=87058) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:36:46.427096 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22b3c187-5bef-4fd0-a2a9-fd3298b8abc3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:36:46.429086 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22b3c187-5bef-4fd0-a2a9-fd3298b8abc3 None admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:36:46.435472 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 273/541] 10.222.0.20 () {56 vars in 1374 bytes} [Mon Aug 1 16:36:46 2022] GET /identity/v3/role_assignments?user.id=3d9bd8d649154c979be3744bfd016f95&scope.project.id=7b992d6a65df4bf9b27c13192d57c5f0&role.id=51bb93e277c446b49bb80a32afcd3af7 => generated 259 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:36:46.442395 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b030653-5edf-40b4-a7e9-32bdd7f9213c None None] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:36:46.442900 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b030653-5edf-40b4-a7e9-32bdd7f9213c None None] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:36:46.443001 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b030653-5edf-40b4-a7e9-32bdd7f9213c None None] PATH_INFO: `/` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:36:46.445594 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 269/542] 10.222.0.20 () {54 vars in 926 bytes} [Mon Aug 1 16:36:46 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 16:36:46.450131 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a6a104-e3e1-4bc2-957c-b3bf71655939 None None] REQUEST_METHOD: `POST` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:36:46.450417 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a6a104-e3e1-4bc2-957c-b3bf71655939 None None] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:36:46.450705 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a6a104-e3e1-4bc2-957c-b3bf71655939 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:36:46.483797 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.auth.core [None req-23a6a104-e3e1-4bc2-957c-b3bf71655939 None None] MFA Rules not processed for user `1007f8e379b74e169b98fb7c3c0ba5b4`. Rule list: `[]` (Enabled: `True`). {{(pid=87058) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:36:46.485064 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.fernet_utils [None req-23a6a104-e3e1-4bc2-957c-b3bf71655939 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:36:46.562339 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 274/543] 10.222.0.20 () {58 vars in 1023 bytes} [Mon Aug 1 16:36:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 16:36:46.596797 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b030653-5edf-40b4-a7e9-32bdd7f9213c None None] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:36:46.740586 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b030653-5edf-40b4-a7e9-32bdd7f9213c None None] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:36:46.780778 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e1e610-82dd-42f0-a106-0edd5c86a2dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:36:46.781418 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e1e610-82dd-42f0-a106-0edd5c86a2dc None admin] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:36:46.781694 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e1e610-82dd-42f0-a106-0edd5c86a2dc None admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:36:46.781964 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e1e610-82dd-42f0-a106-0edd5c86a2dc None admin] PATH_INFO: `/v3/roles/service` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:36:46.792589 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53e1e610-82dd-42f0-a106-0edd5c86a2dc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 16:36:46.797526 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application [None req-53e1e610-82dd-42f0-a106-0edd5c86a2dc None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 01 16:36:46.797526 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 16:36:46.797526 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 16:36:46.797526 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 16:36:46.797526 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 16:36:46.797526 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 16:36:46.797526 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 16:36:46.797526 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 16:36:46.797526 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 16:36:46.797526 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 16:36:46.797526 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 16:36:46.797526 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 16:36:46.797526 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 16:36:46.797526 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 16:36:46.797526 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 16:36:46.797526 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 16:36:46.798478 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 16:36:46.798478 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 16:36:46.798478 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 16:36:46.798478 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 16:36:46.798478 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 16:36:46.798478 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 16:36:46.798478 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 16:36:46.798478 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 16:36:46.798478 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 16:36:46.798478 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 16:36:46.798478 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 16:36:46.798478 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application with Lock( Aug 01 16:36:46.798478 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 16:36:46.798478 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self._enter() Aug 01 16:36:46.798478 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 16:36:46.798478 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 16:36:46.799282 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 16:36:46.799282 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self.creator() Aug 01 16:36:46.799282 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 16:36:46.799282 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application created_value = creator( Aug 01 16:36:46.799282 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 16:36:46.799282 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 16:36:46.799282 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 16:36:46.799282 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 16:36:46.799282 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 16:36:46.799282 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 16:36:46.799282 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 01 16:36:46.799282 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application Aug 01 16:36:46.799282 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 270/544] 10.222.0.20 () {56 vars in 1102 bytes} [Mon Aug 1 16:36:46 2022] GET /identity/v3/roles/service => generated 85 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 16:36:46.805007 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a6a104-e3e1-4bc2-957c-b3bf71655939 None None] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:36:46.949661 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG dbcounter [-] [87058] Writing DB stats keystone:SELECT=100 {{(pid=87058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:36:46.974994 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a6a104-e3e1-4bc2-957c-b3bf71655939 None None] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:36:46.995545 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9e5a1e-dee3-42f3-bd6a-cb267a128931 None None] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:36:46.995784 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9e5a1e-dee3-42f3-bd6a-cb267a128931 None None] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:36:46.996122 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9e5a1e-dee3-42f3-bd6a-cb267a128931 None None] PATH_INFO: `/` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:36:47.003596 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 271/545] 10.222.0.20 () {54 vars in 926 bytes} [Mon Aug 1 16:36:46 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 16:36:47.011255 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741a054d-fa20-4352-801f-42c2cdf900a1 None None] REQUEST_METHOD: `POST` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:36:47.011543 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741a054d-fa20-4352-801f-42c2cdf900a1 None None] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:36:47.011804 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741a054d-fa20-4352-801f-42c2cdf900a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:36:47.029694 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3968ea5b-576f-421e-ba6f-65ae6867cb44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:36:47.032178 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3968ea5b-576f-421e-ba6f-65ae6867cb44 None admin] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:36:47.032458 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3968ea5b-576f-421e-ba6f-65ae6867cb44 None admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:36:47.032733 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3968ea5b-576f-421e-ba6f-65ae6867cb44 None admin] PATH_INFO: `/v3/roles` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:36:47.033471 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3968ea5b-576f-421e-ba6f-65ae6867cb44 None admin] RBAC: Adding query filter params (name=service) {{(pid=87058) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:36:47.033780 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3968ea5b-576f-421e-ba6f-65ae6867cb44 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:36:47.035190 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3968ea5b-576f-421e-ba6f-65ae6867cb44 None admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:36:47.037474 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.auth.core [None req-741a054d-fa20-4352-801f-42c2cdf900a1 None None] MFA Rules not processed for user `1007f8e379b74e169b98fb7c3c0ba5b4`. Rule list: `[]` (Enabled: `True`). {{(pid=87057) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:36:47.039627 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.fernet_utils [None req-741a054d-fa20-4352-801f-42c2cdf900a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87057) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:36:47.050323 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 275/546] 10.222.0.20 () {56 vars in 1103 bytes} [Mon Aug 1 16:36:46 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:36:47.057066 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3968ea5b-576f-421e-ba6f-65ae6867cb44 None admin] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:36:47.090404 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG dbcounter [-] [87057] Writing DB stats keystone:SELECT=100 {{(pid=87057) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:36:47.169894 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 272/547] 10.222.0.20 () {58 vars in 1023 bytes} [Mon Aug 1 16:36:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 16:36:47.210497 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3968ea5b-576f-421e-ba6f-65ae6867cb44 None admin] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:36:47.232175 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-741a054d-fa20-4352-801f-42c2cdf900a1 None None] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:36:47.252212 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb92bef-0f7e-4d17-9705-9a056a8a0a8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:36:47.252784 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb92bef-0f7e-4d17-9705-9a056a8a0a8d None admin] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:36:47.253065 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb92bef-0f7e-4d17-9705-9a056a8a0a8d None admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:36:47.253337 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb92bef-0f7e-4d17-9705-9a056a8a0a8d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:36:47.266713 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bb92bef-0f7e-4d17-9705-9a056a8a0a8d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:36:47.268183 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bb92bef-0f7e-4d17-9705-9a056a8a0a8d None admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:36:47.270865 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG dbcounter [-] [87058] Writing DB stats keystone:SELECT=100 {{(pid=87058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:36:47.284041 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application [None req-0bb92bef-0f7e-4d17-9705-9a056a8a0a8d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 16:36:47.284041 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 16:36:47.284041 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 16:36:47.284041 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 16:36:47.284041 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 16:36:47.284041 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 16:36:47.284041 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 16:36:47.284041 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 16:36:47.284041 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 16:36:47.284041 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 16:36:47.284041 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 16:36:47.284041 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 16:36:47.284041 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 16:36:47.284041 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 16:36:47.284041 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 16:36:47.285045 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 16:36:47.285045 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 16:36:47.285045 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 16:36:47.285045 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 16:36:47.285045 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 16:36:47.285045 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 16:36:47.285045 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 16:36:47.285045 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 16:36:47.285045 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application with Lock( Aug 01 16:36:47.285045 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 16:36:47.285045 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self._enter() Aug 01 16:36:47.285045 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 16:36:47.285045 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 16:36:47.285045 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 16:36:47.285045 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self.creator() Aug 01 16:36:47.285045 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 16:36:47.285045 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application created_value = creator( Aug 01 16:36:47.285891 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 16:36:47.285891 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 16:36:47.285891 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 16:36:47.285891 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application Aug 01 16:36:47.285891 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 276/548] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Aug 1 16:36:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 16:36:47.292080 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb92bef-0f7e-4d17-9705-9a056a8a0a8d None admin] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:36:47.416812 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-741a054d-fa20-4352-801f-42c2cdf900a1 None None] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:36:47.453798 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a05005d1-4e85-49af-8240-8ffb521408b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:36:47.454485 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05005d1-4e85-49af-8240-8ffb521408b9 None admin] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:36:47.454977 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05005d1-4e85-49af-8240-8ffb521408b9 None admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:36:47.454977 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05005d1-4e85-49af-8240-8ffb521408b9 None admin] PATH_INFO: `/v3/roles/service` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:36:47.461903 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb92bef-0f7e-4d17-9705-9a056a8a0a8d None admin] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:36:47.488007 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a05005d1-4e85-49af-8240-8ffb521408b9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 16:36:47.493542 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application [None req-a05005d1-4e85-49af-8240-8ffb521408b9 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 01 16:36:47.493542 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 16:36:47.493542 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 16:36:47.493542 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 16:36:47.493542 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 16:36:47.493542 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 16:36:47.493542 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 16:36:47.493542 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 16:36:47.493542 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 16:36:47.493542 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 16:36:47.493542 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 16:36:47.493542 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 16:36:47.493542 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 16:36:47.493542 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 16:36:47.493542 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 16:36:47.493542 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 16:36:47.494677 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 16:36:47.494677 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 16:36:47.494677 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 16:36:47.494677 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 16:36:47.494677 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 16:36:47.494677 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 16:36:47.494677 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 16:36:47.494677 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 16:36:47.494677 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 16:36:47.494677 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 16:36:47.494677 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 16:36:47.494677 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application with Lock( Aug 01 16:36:47.494677 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 16:36:47.494677 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self._enter() Aug 01 16:36:47.494677 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 16:36:47.494677 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 16:36:47.495488 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 16:36:47.495488 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self.creator() Aug 01 16:36:47.495488 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 16:36:47.495488 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application created_value = creator( Aug 01 16:36:47.495488 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 16:36:47.495488 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 16:36:47.495488 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 16:36:47.495488 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 16:36:47.495488 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 16:36:47.495488 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 16:36:47.495488 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 01 16:36:47.495488 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application Aug 01 16:36:47.495488 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 273/549] 10.222.0.20 () {56 vars in 1102 bytes} [Mon Aug 1 16:36:47 2022] GET /identity/v3/roles/service => generated 85 bytes in 263 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 01 16:36:47.500629 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a05005d1-4e85-49af-8240-8ffb521408b9 None admin] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:36:47.501521 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e3a5800-7e81-465c-be4c-912acdb3eca8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:36:47.502434 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3a5800-7e81-465c-be4c-912acdb3eca8 None admin] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:36:47.502898 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3a5800-7e81-465c-be4c-912acdb3eca8 None admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:36:47.503348 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3a5800-7e81-465c-be4c-912acdb3eca8 None admin] PATH_INFO: `/v3/domains` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:36:47.504250 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e3a5800-7e81-465c-be4c-912acdb3eca8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87058) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:36:47.504736 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e3a5800-7e81-465c-be4c-912acdb3eca8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:36:47.506318 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e3a5800-7e81-465c-be4c-912acdb3eca8 None admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:36:47.520543 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 277/550] 10.222.0.20 () {56 vars in 1109 bytes} [Mon Aug 1 16:36:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:36:47.526170 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e3a5800-7e81-465c-be4c-912acdb3eca8 None admin] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:36:47.555147 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG dbcounter [-] [87057] Writing DB stats keystone:SELECT=100 {{(pid=87057) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:36:47.599037 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG dbcounter [-] [87058] Writing DB stats keystone:SELECT=100 {{(pid=87058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:36:47.647334 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e3a5800-7e81-465c-be4c-912acdb3eca8 None admin] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:36:47.703305 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a05005d1-4e85-49af-8240-8ffb521408b9 None admin] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:36:47.706765 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-766716fc-1b42-43de-81fd-674dafec13d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:36:47.710532 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766716fc-1b42-43de-81fd-674dafec13d0 None admin] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:36:47.711045 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766716fc-1b42-43de-81fd-674dafec13d0 None admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:36:47.711505 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766716fc-1b42-43de-81fd-674dafec13d0 None admin] PATH_INFO: `/v3/users/nova` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:36:47.720761 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-766716fc-1b42-43de-81fd-674dafec13d0 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:36:47.722502 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-766716fc-1b42-43de-81fd-674dafec13d0 None admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:36:47.735839 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application [None req-766716fc-1b42-43de-81fd-674dafec13d0 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 01 16:36:47.735839 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 16:36:47.735839 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 16:36:47.735839 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 16:36:47.735839 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 16:36:47.735839 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 16:36:47.735839 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 16:36:47.735839 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 16:36:47.735839 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 16:36:47.735839 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 16:36:47.735839 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 16:36:47.735839 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 16:36:47.735839 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 01 16:36:47.735839 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 01 16:36:47.735839 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 01 16:36:47.735839 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 01 16:36:47.737006 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 16:36:47.737006 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 16:36:47.737006 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 01 16:36:47.737006 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 16:36:47.737006 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 01 16:36:47.737006 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 16:36:47.737006 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 16:36:47.737006 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 16:36:47.737006 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 16:36:47.737006 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 16:36:47.737006 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 16:36:47.737006 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application with Lock( Aug 01 16:36:47.737006 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 16:36:47.737006 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self._enter() Aug 01 16:36:47.737006 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 16:36:47.737006 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 16:36:47.737006 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 16:36:47.737006 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self.creator() Aug 01 16:36:47.737927 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 16:36:47.737927 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application created_value = creator( Aug 01 16:36:47.737927 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 01 16:36:47.737927 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 01 16:36:47.737927 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 01 16:36:47.737927 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 01 16:36:47.737927 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 01 16:36:47.737927 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 01 16:36:47.737927 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 01 16:36:47.737927 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application Aug 01 16:36:47.740326 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 278/551] 10.222.0.20 () {56 vars in 1093 bytes} [Mon Aug 1 16:36:47 2022] GET /identity/v3/users/nova => generated 82 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 16:36:47.746072 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-766716fc-1b42-43de-81fd-674dafec13d0 None admin] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:36:47.767078 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3b254a3-ccf0-493b-989d-cc6f8dad160e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:36:47.768217 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b254a3-ccf0-493b-989d-cc6f8dad160e None admin] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:36:47.768665 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b254a3-ccf0-493b-989d-cc6f8dad160e None admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:36:47.769367 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b254a3-ccf0-493b-989d-cc6f8dad160e None admin] PATH_INFO: `/v3/roles` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:36:47.770704 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3b254a3-ccf0-493b-989d-cc6f8dad160e None admin] RBAC: Adding query filter params (name=service) {{(pid=87057) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:36:47.771162 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3b254a3-ccf0-493b-989d-cc6f8dad160e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:36:47.773343 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3b254a3-ccf0-493b-989d-cc6f8dad160e None admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:36:47.786062 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 274/552] 10.222.0.20 () {56 vars in 1103 bytes} [Mon Aug 1 16:36:47 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 286 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 16:36:47.788321 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a390ef-b9cd-4822-8bde-c0bc358d755e None None] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:36:47.788754 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a390ef-b9cd-4822-8bde-c0bc358d755e None None] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:36:47.790172 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a390ef-b9cd-4822-8bde-c0bc358d755e None None] PATH_INFO: `/` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:36:47.791823 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 275/553] 10.222.0.20 () {54 vars in 926 bytes} [Mon Aug 1 16:36:47 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 16:36:47.792702 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a390ef-b9cd-4822-8bde-c0bc358d755e None None] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:36:47.875041 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-766716fc-1b42-43de-81fd-674dafec13d0 None admin] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:36:47.901287 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG dbcounter [-] [87058] Writing DB stats keystone:SELECT=100 {{(pid=87058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:36:47.911560 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG dbcounter [-] [87057] Writing DB stats keystone:SELECT=100 {{(pid=87057) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:36:47.925205 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f68f3be-e19e-4378-8208-362903fd7ea0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:36:47.925824 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f68f3be-e19e-4378-8208-362903fd7ea0 None admin] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:36:47.926131 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f68f3be-e19e-4378-8208-362903fd7ea0 None admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:36:47.926405 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f68f3be-e19e-4378-8208-362903fd7ea0 None admin] PATH_INFO: `/v3/users` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:36:47.926830 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a390ef-b9cd-4822-8bde-c0bc358d755e None None] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:36:47.927201 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f68f3be-e19e-4378-8208-362903fd7ea0 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=87058) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:36:47.927504 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f68f3be-e19e-4378-8208-362903fd7ea0 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:36:47.929709 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f68f3be-e19e-4378-8208-362903fd7ea0 None admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:36:47.952737 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 279/554] 10.222.0.20 () {56 vars in 1133 bytes} [Mon Aug 1 16:36:47 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:36:47.954444 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a217033-1c71-4c50-ad01-b0f9e65b8d02 None None] REQUEST_METHOD: `POST` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:36:47.954727 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a217033-1c71-4c50-ad01-b0f9e65b8d02 None None] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:36:47.956632 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a217033-1c71-4c50-ad01-b0f9e65b8d02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:36:47.965210 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2124500-4561-4bae-b661-e331877edd99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:36:47.965851 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2124500-4561-4bae-b661-e331877edd99 None admin] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:36:47.966161 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2124500-4561-4bae-b661-e331877edd99 None admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:36:47.966480 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2124500-4561-4bae-b661-e331877edd99 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:36:47.977881 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.auth.core [None req-1a217033-1c71-4c50-ad01-b0f9e65b8d02 None None] MFA Rules not processed for user `1007f8e379b74e169b98fb7c3c0ba5b4`. Rule list: `[]` (Enabled: `True`). {{(pid=87058) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:36:47.978288 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2124500-4561-4bae-b661-e331877edd99 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:36:47.979110 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.fernet_utils [None req-1a217033-1c71-4c50-ad01-b0f9e65b8d02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:36:47.979852 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2124500-4561-4bae-b661-e331877edd99 None admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:36:47.997503 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application [None req-f2124500-4561-4bae-b661-e331877edd99 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 16:36:47.997503 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 16:36:47.997503 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 16:36:47.997503 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 16:36:47.997503 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 16:36:47.997503 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 16:36:47.997503 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 16:36:47.997503 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 16:36:47.997503 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 16:36:47.997503 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 16:36:47.997503 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 16:36:47.997503 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 16:36:47.997503 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 16:36:47.997503 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 16:36:47.997503 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 16:36:47.998468 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 16:36:47.998468 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 16:36:47.998468 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 16:36:47.998468 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 16:36:47.998468 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 16:36:47.998468 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 16:36:47.998468 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 16:36:47.998468 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 16:36:47.998468 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application with Lock( Aug 01 16:36:47.998468 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 16:36:47.998468 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self._enter() Aug 01 16:36:47.998468 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 16:36:47.998468 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 16:36:47.998468 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 16:36:47.998468 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self.creator() Aug 01 16:36:47.998468 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 16:36:47.998468 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application created_value = creator( Aug 01 16:36:47.999954 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 16:36:47.999954 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 16:36:47.999954 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 16:36:47.999954 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application Aug 01 16:36:47.999954 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 276/555] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Aug 1 16:36:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 16:36:48.000225 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2124500-4561-4bae-b661-e331877edd99 None admin] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:36:48.063782 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 280/556] 10.222.0.20 () {58 vars in 1023 bytes} [Mon Aug 1 16:36:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 16:36:48.064386 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a217033-1c71-4c50-ad01-b0f9e65b8d02 None None] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:36:48.113770 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2124500-4561-4bae-b661-e331877edd99 None admin] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:36:48.149512 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-673a5132-909c-4994-a326-d1d0afde366b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:36:48.150168 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673a5132-909c-4994-a326-d1d0afde366b None admin] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:36:48.150448 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673a5132-909c-4994-a326-d1d0afde366b None admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:36:48.150717 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673a5132-909c-4994-a326-d1d0afde366b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:36:48.161425 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG dbcounter [-] [87057] Writing DB stats keystone:SELECT=100 {{(pid=87057) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:36:48.164153 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-673a5132-909c-4994-a326-d1d0afde366b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:36:48.165870 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-673a5132-909c-4994-a326-d1d0afde366b None admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:36:48.181600 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application [None req-673a5132-909c-4994-a326-d1d0afde366b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 16:36:48.181600 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 16:36:48.181600 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 16:36:48.181600 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 16:36:48.181600 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 16:36:48.181600 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 16:36:48.181600 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 16:36:48.181600 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 16:36:48.181600 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 16:36:48.181600 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 16:36:48.181600 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 16:36:48.181600 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 16:36:48.181600 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 16:36:48.181600 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 16:36:48.181600 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 16:36:48.182765 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 16:36:48.182765 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 16:36:48.182765 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 16:36:48.182765 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 16:36:48.182765 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 16:36:48.182765 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 16:36:48.182765 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 16:36:48.182765 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 16:36:48.182765 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application with Lock( Aug 01 16:36:48.182765 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 16:36:48.182765 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self._enter() Aug 01 16:36:48.182765 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 16:36:48.182765 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 16:36:48.182765 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 16:36:48.182765 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self.creator() Aug 01 16:36:48.182765 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 16:36:48.182765 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application created_value = creator( Aug 01 16:36:48.184328 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 16:36:48.184328 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 16:36:48.184328 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 16:36:48.184328 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application Aug 01 16:36:48.185863 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a217033-1c71-4c50-ad01-b0f9e65b8d02 None None] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:36:48.185979 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 277/557] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Aug 1 16:36:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 16:36:48.187674 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd6bf0a-953d-43b2-95d7-d7612c2efa70 None None] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:36:48.187962 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd6bf0a-953d-43b2-95d7-d7612c2efa70 None None] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:36:48.188238 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd6bf0a-953d-43b2-95d7-d7612c2efa70 None None] PATH_INFO: `/` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:36:48.189688 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 278/558] 10.222.0.20 () {54 vars in 926 bytes} [Mon Aug 1 16:36:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 16:36:48.192865 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG dbcounter [-] [87058] Writing DB stats keystone:SELECT=100 {{(pid=87058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:36:48.195253 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd6bf0a-953d-43b2-95d7-d7612c2efa70 None None] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:36:48.230349 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-989edc41-c06f-4070-bb6e-01cb24a75db9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:36:48.230967 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989edc41-c06f-4070-bb6e-01cb24a75db9 None admin] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:36:48.231242 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989edc41-c06f-4070-bb6e-01cb24a75db9 None admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:36:48.231565 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989edc41-c06f-4070-bb6e-01cb24a75db9 None admin] PATH_INFO: `/v3/domains` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:36:48.233776 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-989edc41-c06f-4070-bb6e-01cb24a75db9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87058) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:36:48.234656 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-989edc41-c06f-4070-bb6e-01cb24a75db9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:36:48.236019 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-989edc41-c06f-4070-bb6e-01cb24a75db9 None admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:36:48.254196 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 281/559] 10.222.0.20 () {56 vars in 1109 bytes} [Mon Aug 1 16:36:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:36:48.255139 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-989edc41-c06f-4070-bb6e-01cb24a75db9 None admin] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:36:48.310117 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd6bf0a-953d-43b2-95d7-d7612c2efa70 None None] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:36:48.349227 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-863f3023-c7e9-4e42-81df-4b3291930995 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:36:48.349227 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863f3023-c7e9-4e42-81df-4b3291930995 None admin] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:36:48.349227 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863f3023-c7e9-4e42-81df-4b3291930995 None admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:36:48.349227 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863f3023-c7e9-4e42-81df-4b3291930995 None admin] PATH_INFO: `/v3/roles/service` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:36:48.358901 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-863f3023-c7e9-4e42-81df-4b3291930995 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 16:36:48.363107 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-989edc41-c06f-4070-bb6e-01cb24a75db9 None admin] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:36:48.363852 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application [None req-863f3023-c7e9-4e42-81df-4b3291930995 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 01 16:36:48.363852 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 16:36:48.363852 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 16:36:48.363852 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 16:36:48.363852 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 16:36:48.363852 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 16:36:48.363852 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 16:36:48.363852 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 16:36:48.363852 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 16:36:48.363852 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 16:36:48.363852 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 16:36:48.363852 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 16:36:48.363852 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 16:36:48.363852 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 16:36:48.363852 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 16:36:48.363852 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 16:36:48.364781 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 16:36:48.364781 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 16:36:48.364781 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 16:36:48.364781 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 16:36:48.364781 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 16:36:48.364781 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 16:36:48.364781 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 16:36:48.364781 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 16:36:48.364781 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 16:36:48.364781 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 16:36:48.364781 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 16:36:48.364781 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application with Lock( Aug 01 16:36:48.364781 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 16:36:48.364781 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self._enter() Aug 01 16:36:48.364781 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 16:36:48.364781 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 16:36:48.365738 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 16:36:48.365738 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self.creator() Aug 01 16:36:48.365738 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 16:36:48.365738 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application created_value = creator( Aug 01 16:36:48.365738 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 16:36:48.365738 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 16:36:48.365738 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 16:36:48.365738 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 16:36:48.365738 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 16:36:48.365738 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 16:36:48.365738 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 01 16:36:48.365738 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application Aug 01 16:36:48.365738 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 279/560] 10.222.0.20 () {56 vars in 1102 bytes} [Mon Aug 1 16:36:48 2022] GET /identity/v3/roles/service => generated 85 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 16:36:48.367265 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba88edb3-42b9-4016-a272-b6e7d4441ae2 None None] REQUEST_METHOD: `POST` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:36:48.367265 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba88edb3-42b9-4016-a272-b6e7d4441ae2 None None] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:36:48.367265 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba88edb3-42b9-4016-a272-b6e7d4441ae2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:36:48.385702 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.auth.core [None req-ba88edb3-42b9-4016-a272-b6e7d4441ae2 None None] MFA Rules not processed for user `1007f8e379b74e169b98fb7c3c0ba5b4`. Rule list: `[]` (Enabled: `True`). {{(pid=87057) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:36:48.386941 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.fernet_utils [None req-ba88edb3-42b9-4016-a272-b6e7d4441ae2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87057) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:36:48.405773 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-055ab2cb-3de9-4bd5-a861-22ab610bd0d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:36:48.406433 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055ab2cb-3de9-4bd5-a861-22ab610bd0d9 None admin] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:36:48.406706 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055ab2cb-3de9-4bd5-a861-22ab610bd0d9 None admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:36:48.406981 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055ab2cb-3de9-4bd5-a861-22ab610bd0d9 None admin] PATH_INFO: `/v3/domains` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:36:48.407675 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-055ab2cb-3de9-4bd5-a861-22ab610bd0d9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87058) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:36:48.407985 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-055ab2cb-3de9-4bd5-a861-22ab610bd0d9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:36:48.409389 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-055ab2cb-3de9-4bd5-a861-22ab610bd0d9 None admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:36:48.426652 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 282/561] 10.222.0.20 () {56 vars in 1109 bytes} [Mon Aug 1 16:36:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:36:48.426652 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-055ab2cb-3de9-4bd5-a861-22ab610bd0d9 None admin] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:36:48.440423 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG dbcounter [-] [87057] Writing DB stats keystone:SELECT=100 {{(pid=87057) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:36:48.455017 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG dbcounter [-] [87058] Writing DB stats keystone:SELECT=100 {{(pid=87058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:36:48.460912 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 280/562] 10.222.0.20 () {58 vars in 1023 bytes} [Mon Aug 1 16:36:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 16:36:48.461584 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba88edb3-42b9-4016-a272-b6e7d4441ae2 None None] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:36:48.543150 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-055ab2cb-3de9-4bd5-a861-22ab610bd0d9 None admin] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:36:48.570831 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba88edb3-42b9-4016-a272-b6e7d4441ae2 None None] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:36:48.580524 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd994db5-d874-4a88-998f-51ed0ac01ea0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:36:48.581170 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd994db5-d874-4a88-998f-51ed0ac01ea0 None admin] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:36:48.581446 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd994db5-d874-4a88-998f-51ed0ac01ea0 None admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:36:48.581729 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd994db5-d874-4a88-998f-51ed0ac01ea0 None admin] PATH_INFO: `/v3/users/swift` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:36:48.589597 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd994db5-d874-4a88-998f-51ed0ac01ea0 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:36:48.591198 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd994db5-d874-4a88-998f-51ed0ac01ea0 None admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:36:48.604251 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application [None req-fd994db5-d874-4a88-998f-51ed0ac01ea0 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 01 16:36:48.604251 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 16:36:48.604251 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 16:36:48.604251 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 16:36:48.604251 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 16:36:48.604251 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 16:36:48.604251 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 16:36:48.604251 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 16:36:48.604251 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 16:36:48.604251 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 16:36:48.604251 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 16:36:48.604251 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 16:36:48.604251 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 01 16:36:48.604251 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 01 16:36:48.604251 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 01 16:36:48.604251 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 01 16:36:48.605337 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 16:36:48.605337 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 16:36:48.605337 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 01 16:36:48.605337 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 16:36:48.605337 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 01 16:36:48.605337 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 16:36:48.605337 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 16:36:48.605337 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 16:36:48.605337 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 16:36:48.605337 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 16:36:48.605337 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 16:36:48.605337 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application with Lock( Aug 01 16:36:48.605337 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 16:36:48.605337 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self._enter() Aug 01 16:36:48.605337 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 16:36:48.605337 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 16:36:48.605337 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 16:36:48.605337 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self.creator() Aug 01 16:36:48.606911 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 16:36:48.606911 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application created_value = creator( Aug 01 16:36:48.606911 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 01 16:36:48.606911 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 01 16:36:48.606911 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 01 16:36:48.606911 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 01 16:36:48.606911 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 01 16:36:48.606911 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 01 16:36:48.606911 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 01 16:36:48.606911 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application Aug 01 16:36:48.606911 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 283/563] 10.222.0.20 () {56 vars in 1096 bytes} [Mon Aug 1 16:36:48 2022] GET /identity/v3/users/swift => generated 83 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 16:36:48.606911 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd994db5-d874-4a88-998f-51ed0ac01ea0 None admin] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:36:48.612050 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d877461-a7b7-4180-bd83-e30bddc1a933 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:36:48.612675 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d877461-a7b7-4180-bd83-e30bddc1a933 None admin] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:36:48.612950 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d877461-a7b7-4180-bd83-e30bddc1a933 None admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:36:48.613220 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d877461-a7b7-4180-bd83-e30bddc1a933 None admin] PATH_INFO: `/v3/roles` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:36:48.614185 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d877461-a7b7-4180-bd83-e30bddc1a933 None admin] RBAC: Adding query filter params (name=service) {{(pid=87057) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:36:48.614486 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d877461-a7b7-4180-bd83-e30bddc1a933 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:36:48.616303 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d877461-a7b7-4180-bd83-e30bddc1a933 None admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:36:48.629124 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 281/564] 10.222.0.20 () {56 vars in 1103 bytes} [Mon Aug 1 16:36:48 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:36:48.629124 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d877461-a7b7-4180-bd83-e30bddc1a933 None admin] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:36:48.674259 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG dbcounter [-] [87057] Writing DB stats keystone:SELECT=100 {{(pid=87057) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:36:48.701334 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG dbcounter [-] [87058] Writing DB stats keystone:SELECT=100 {{(pid=87058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:36:48.720312 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd994db5-d874-4a88-998f-51ed0ac01ea0 None admin] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:36:48.733359 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d877461-a7b7-4180-bd83-e30bddc1a933 None admin] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:36:48.757221 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e153d370-bcc7-40f9-b51e-d8076054f95a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:36:48.757860 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e153d370-bcc7-40f9-b51e-d8076054f95a None admin] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:36:48.758157 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e153d370-bcc7-40f9-b51e-d8076054f95a None admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:36:48.758420 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e153d370-bcc7-40f9-b51e-d8076054f95a None admin] PATH_INFO: `/v3/projects/service` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:36:48.765033 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e153d370-bcc7-40f9-b51e-d8076054f95a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:36:48.766772 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e153d370-bcc7-40f9-b51e-d8076054f95a None admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:36:48.769216 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-976147ad-1996-4a69-8d3c-df8aeef86d74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:36:48.769849 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976147ad-1996-4a69-8d3c-df8aeef86d74 None admin] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:36:48.770153 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976147ad-1996-4a69-8d3c-df8aeef86d74 None admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:36:48.770429 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976147ad-1996-4a69-8d3c-df8aeef86d74 None admin] PATH_INFO: `/v3/roles/service` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:36:48.777837 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application [None req-e153d370-bcc7-40f9-b51e-d8076054f95a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 16:36:48.777837 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 16:36:48.777837 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 16:36:48.777837 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 16:36:48.777837 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 16:36:48.777837 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 16:36:48.777837 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 16:36:48.777837 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 16:36:48.777837 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 16:36:48.777837 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 16:36:48.777837 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 16:36:48.777837 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 16:36:48.777837 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 01 16:36:48.777837 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 01 16:36:48.777837 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 01 16:36:48.778759 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 01 16:36:48.778759 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 16:36:48.778759 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 16:36:48.778759 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 16:36:48.778759 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 16:36:48.778759 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 16:36:48.778759 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 16:36:48.778759 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 16:36:48.778759 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application with Lock( Aug 01 16:36:48.778759 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 16:36:48.778759 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self._enter() Aug 01 16:36:48.778759 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 16:36:48.778759 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 16:36:48.778759 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 16:36:48.778759 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self.creator() Aug 01 16:36:48.778759 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 16:36:48.778759 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application created_value = creator( Aug 01 16:36:48.779668 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 01 16:36:48.779668 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 01 16:36:48.779668 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 01 16:36:48.779668 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 01 16:36:48.779668 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 01 16:36:48.779668 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 01 16:36:48.779668 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 16:36:48.779668 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application Aug 01 16:36:48.779668 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 284/565] 10.222.0.20 () {56 vars in 1111 bytes} [Mon Aug 1 16:36:48 2022] GET /identity/v3/projects/service => generated 88 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 16:36:48.780168 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e153d370-bcc7-40f9-b51e-d8076054f95a None admin] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:36:48.784626 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-976147ad-1996-4a69-8d3c-df8aeef86d74 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 16:36:48.790755 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application [None req-976147ad-1996-4a69-8d3c-df8aeef86d74 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 01 16:36:48.790755 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 16:36:48.790755 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 16:36:48.790755 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 16:36:48.790755 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 16:36:48.790755 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 16:36:48.790755 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 16:36:48.790755 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 16:36:48.790755 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 16:36:48.790755 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 16:36:48.790755 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 16:36:48.790755 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 16:36:48.790755 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 16:36:48.790755 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 16:36:48.790755 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 16:36:48.790755 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 16:36:48.791799 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 16:36:48.791799 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 16:36:48.791799 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 16:36:48.791799 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 16:36:48.791799 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 16:36:48.791799 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 16:36:48.791799 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 16:36:48.791799 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 16:36:48.791799 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 16:36:48.791799 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 16:36:48.791799 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 16:36:48.791799 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application with Lock( Aug 01 16:36:48.791799 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 16:36:48.791799 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self._enter() Aug 01 16:36:48.791799 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 16:36:48.791799 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 16:36:48.792637 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 16:36:48.792637 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self.creator() Aug 01 16:36:48.792637 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 16:36:48.792637 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application created_value = creator( Aug 01 16:36:48.792637 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 16:36:48.792637 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 16:36:48.792637 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 16:36:48.792637 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 16:36:48.792637 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 16:36:48.792637 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 16:36:48.792637 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 01 16:36:48.792637 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application Aug 01 16:36:48.792637 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 282/566] 10.222.0.20 () {56 vars in 1102 bytes} [Mon Aug 1 16:36:48 2022] GET /identity/v3/roles/service => generated 85 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 16:36:48.793322 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-976147ad-1996-4a69-8d3c-df8aeef86d74 None admin] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:36:48.880787 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e153d370-bcc7-40f9-b51e-d8076054f95a None admin] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:36:48.893395 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-976147ad-1996-4a69-8d3c-df8aeef86d74 None admin] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:36:48.896472 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG dbcounter [-] [87057] Writing DB stats keystone:SELECT=100 {{(pid=87057) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:36:48.915430 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba838caa-3fb9-4d65-bed8-3cb2cbf8f846 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:36:48.916050 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba838caa-3fb9-4d65-bed8-3cb2cbf8f846 None admin] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:36:48.916330 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba838caa-3fb9-4d65-bed8-3cb2cbf8f846 None admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:36:48.916601 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba838caa-3fb9-4d65-bed8-3cb2cbf8f846 None admin] PATH_INFO: `/v3/users` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:36:48.917429 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba838caa-3fb9-4d65-bed8-3cb2cbf8f846 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=87058) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:36:48.917731 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba838caa-3fb9-4d65-bed8-3cb2cbf8f846 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:36:48.919209 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba838caa-3fb9-4d65-bed8-3cb2cbf8f846 None admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:36:48.930878 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af3a9336-5141-4973-8544-4d7abf3eaab4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:36:48.931513 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3a9336-5141-4973-8544-4d7abf3eaab4 None admin] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:36:48.931792 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3a9336-5141-4973-8544-4d7abf3eaab4 None admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:36:48.932068 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3a9336-5141-4973-8544-4d7abf3eaab4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:36:48.936451 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 285/567] 10.222.0.20 () {56 vars in 1135 bytes} [Mon Aug 1 16:36:48 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:36:48.940992 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aecaffdc-3da3-4423-8e45-fe472f620667 None None] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:36:48.940992 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aecaffdc-3da3-4423-8e45-fe472f620667 None None] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:36:48.940992 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aecaffdc-3da3-4423-8e45-fe472f620667 None None] PATH_INFO: `/` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:36:48.940992 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 286/568] 10.222.0.20 () {54 vars in 926 bytes} [Mon Aug 1 16:36:48 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 16:36:48.940992 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aecaffdc-3da3-4423-8e45-fe472f620667 None None] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:36:48.943817 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG dbcounter [-] [87058] Writing DB stats keystone:SELECT=100 {{(pid=87058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:36:48.945852 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af3a9336-5141-4973-8544-4d7abf3eaab4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:36:48.947358 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af3a9336-5141-4973-8544-4d7abf3eaab4 None admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:36:48.963323 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application [None req-af3a9336-5141-4973-8544-4d7abf3eaab4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 16:36:48.963323 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 16:36:48.963323 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 16:36:48.963323 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 16:36:48.963323 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 16:36:48.963323 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 16:36:48.963323 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 16:36:48.963323 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 16:36:48.963323 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 16:36:48.963323 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 16:36:48.963323 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 16:36:48.963323 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 16:36:48.963323 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 16:36:48.963323 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 16:36:48.963323 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 16:36:48.964266 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 16:36:48.964266 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 16:36:48.964266 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 16:36:48.964266 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 16:36:48.964266 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 16:36:48.964266 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 16:36:48.964266 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 16:36:48.964266 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 16:36:48.964266 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application with Lock( Aug 01 16:36:48.964266 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 16:36:48.964266 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self._enter() Aug 01 16:36:48.964266 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 16:36:48.964266 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 16:36:48.964266 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 16:36:48.964266 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self.creator() Aug 01 16:36:48.964266 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 16:36:48.964266 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application created_value = creator( Aug 01 16:36:48.965169 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 16:36:48.965169 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 16:36:48.965169 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 16:36:48.965169 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application Aug 01 16:36:48.965169 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 283/569] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Aug 1 16:36:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 16:36:48.965448 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af3a9336-5141-4973-8544-4d7abf3eaab4 None admin] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:36:49.054948 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aecaffdc-3da3-4423-8e45-fe472f620667 None None] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:36:49.063031 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af3a9336-5141-4973-8544-4d7abf3eaab4 None admin] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:36:49.089048 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de5ecb42-9b87-4796-9f10-946cd5eb7fec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:36:49.089663 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5ecb42-9b87-4796-9f10-946cd5eb7fec None admin] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:36:49.089943 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5ecb42-9b87-4796-9f10-946cd5eb7fec None admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:36:49.090247 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5ecb42-9b87-4796-9f10-946cd5eb7fec None admin] PATH_INFO: `/v3/projects` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:36:49.090998 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de5ecb42-9b87-4796-9f10-946cd5eb7fec None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87058) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:36:49.091306 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de5ecb42-9b87-4796-9f10-946cd5eb7fec None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:36:49.092742 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de5ecb42-9b87-4796-9f10-946cd5eb7fec None admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:36:49.104562 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-136692a4-2ea8-46b9-a904-eb94172e2187 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:36:49.104765 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 287/570] 10.222.0.20 () {56 vars in 1148 bytes} [Mon Aug 1 16:36:48 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:36:49.105170 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136692a4-2ea8-46b9-a904-eb94172e2187 None admin] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:36:49.105450 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136692a4-2ea8-46b9-a904-eb94172e2187 None admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:36:49.105546 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de5ecb42-9b87-4796-9f10-946cd5eb7fec None admin] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:36:49.105731 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136692a4-2ea8-46b9-a904-eb94172e2187 None admin] PATH_INFO: `/v3/roles` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:36:49.106441 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-136692a4-2ea8-46b9-a904-eb94172e2187 None admin] RBAC: Adding query filter params (name=service) {{(pid=87057) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:36:49.106740 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-136692a4-2ea8-46b9-a904-eb94172e2187 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:36:49.108130 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-136692a4-2ea8-46b9-a904-eb94172e2187 None admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:36:49.117662 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 284/571] 10.222.0.20 () {56 vars in 1103 bytes} [Mon Aug 1 16:36:48 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:36:49.118924 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb82e4a-29d8-4a43-b581-a8b8d029cb37 None None] REQUEST_METHOD: `POST` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:36:49.119208 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb82e4a-29d8-4a43-b581-a8b8d029cb37 None None] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:36:49.119483 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb82e4a-29d8-4a43-b581-a8b8d029cb37 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:36:49.137543 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.auth.core [None req-ceb82e4a-29d8-4a43-b581-a8b8d029cb37 None None] MFA Rules not processed for user `1007f8e379b74e169b98fb7c3c0ba5b4`. Rule list: `[]` (Enabled: `True`). {{(pid=87057) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:36:49.138698 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.fernet_utils [None req-ceb82e4a-29d8-4a43-b581-a8b8d029cb37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87057) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:36:49.153140 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG dbcounter [-] [87057] Writing DB stats keystone:SELECT=100 {{(pid=87057) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:36:49.175642 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG dbcounter [-] [87058] Writing DB stats keystone:SELECT=100 {{(pid=87058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:36:49.207778 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 285/572] 10.222.0.20 () {58 vars in 1023 bytes} [Mon Aug 1 16:36:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 16:36:49.208817 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb82e4a-29d8-4a43-b581-a8b8d029cb37 None None] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:36:49.213766 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de5ecb42-9b87-4796-9f10-946cd5eb7fec None admin] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:36:49.248533 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cbee198-7e79-473e-a0ab-4de0d2c05931 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:36:49.249140 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cbee198-7e79-473e-a0ab-4de0d2c05931 None admin] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:36:49.249414 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cbee198-7e79-473e-a0ab-4de0d2c05931 None admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:36:49.249679 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cbee198-7e79-473e-a0ab-4de0d2c05931 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:36:49.310075 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb82e4a-29d8-4a43-b581-a8b8d029cb37 None None] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:36:49.316043 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cbee198-7e79-473e-a0ab-4de0d2c05931 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:36:49.317501 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cbee198-7e79-473e-a0ab-4de0d2c05931 None admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:36:49.343516 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85e772e8-bbf1-48c3-89db-a800a7da7661 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:36:49.344134 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e772e8-bbf1-48c3-89db-a800a7da7661 None admin] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:36:49.344415 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e772e8-bbf1-48c3-89db-a800a7da7661 None admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:36:49.344687 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e772e8-bbf1-48c3-89db-a800a7da7661 None admin] PATH_INFO: `/v3/domains` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:36:49.345378 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85e772e8-bbf1-48c3-89db-a800a7da7661 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87057) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:36:49.345681 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85e772e8-bbf1-48c3-89db-a800a7da7661 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:36:49.347083 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85e772e8-bbf1-48c3-89db-a800a7da7661 None admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:36:49.358560 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 286/573] 10.222.0.20 () {56 vars in 1109 bytes} [Mon Aug 1 16:36:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:36:49.359329 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85e772e8-bbf1-48c3-89db-a800a7da7661 None admin] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:36:49.371028 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG dbcounter [-] [87057] Writing DB stats keystone:SELECT=100 {{(pid=87057) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:36:49.377131 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG dbcounter [-] [87058] Writing DB stats keystone:SELECT=100 {{(pid=87058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:36:49.438068 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 288/574] 10.222.0.20 () {58 vars in 1282 bytes} [Mon Aug 1 16:36:49 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 332 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 01 16:36:49.439110 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cbee198-7e79-473e-a0ab-4de0d2c05931 None admin] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:36:49.478778 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85e772e8-bbf1-48c3-89db-a800a7da7661 None admin] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:36:49.511015 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62e7a563-6ff7-465c-8bd2-2435c8d1f96f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:36:49.511669 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e7a563-6ff7-465c-8bd2-2435c8d1f96f None admin] REQUEST_METHOD: `PUT` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:36:49.511941 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e7a563-6ff7-465c-8bd2-2435c8d1f96f None admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:36:49.512210 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e7a563-6ff7-465c-8bd2-2435c8d1f96f None admin] PATH_INFO: `/v3/projects/7b992d6a65df4bf9b27c13192d57c5f0/users/6533106d03eb4ca185a8839336d2aff9/roles/51bb93e277c446b49bb80a32afcd3af7` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:36:49.526593 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62e7a563-6ff7-465c-8bd2-2435c8d1f96f None admin] RBAC: Authorizing `identity:create_grant(project_id=7b992d6a65df4bf9b27c13192d57c5f0, user_id=6533106d03eb4ca185a8839336d2aff9, role_id=51bb93e277c446b49bb80a32afcd3af7)` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:36:49.528034 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62e7a563-6ff7-465c-8bd2-2435c8d1f96f None admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:36:49.539026 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cbee198-7e79-473e-a0ab-4de0d2c05931 None admin] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:36:49.571502 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9690722-29ee-4038-afca-51ee4f565b3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:36:49.572132 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9690722-29ee-4038-afca-51ee4f565b3c None admin] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:36:49.572411 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9690722-29ee-4038-afca-51ee4f565b3c None admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:36:49.572685 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9690722-29ee-4038-afca-51ee4f565b3c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:36:49.582424 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9690722-29ee-4038-afca-51ee4f565b3c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:36:49.583864 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9690722-29ee-4038-afca-51ee4f565b3c None admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:36:49.598693 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application [None req-c9690722-29ee-4038-afca-51ee4f565b3c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 16:36:49.598693 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 16:36:49.598693 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 16:36:49.598693 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 16:36:49.598693 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 16:36:49.598693 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 16:36:49.598693 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 16:36:49.598693 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 16:36:49.598693 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 16:36:49.598693 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 16:36:49.598693 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 16:36:49.598693 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 16:36:49.598693 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 16:36:49.598693 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 16:36:49.598693 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 16:36:49.599569 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 16:36:49.599569 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 16:36:49.599569 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 16:36:49.599569 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 16:36:49.599569 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 16:36:49.599569 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 16:36:49.599569 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 16:36:49.599569 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 16:36:49.599569 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application with Lock( Aug 01 16:36:49.599569 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 16:36:49.599569 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self._enter() Aug 01 16:36:49.599569 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 16:36:49.599569 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 16:36:49.599569 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 16:36:49.599569 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self.creator() Aug 01 16:36:49.599569 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 16:36:49.599569 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application created_value = creator( Aug 01 16:36:49.601056 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 16:36:49.601056 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 16:36:49.601056 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 16:36:49.601056 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application Aug 01 16:36:49.601056 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 289/575] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Aug 1 16:36:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 16:36:49.601340 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9690722-29ee-4038-afca-51ee4f565b3c None admin] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:36:49.602986 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 287/576] 10.222.0.20 () {58 vars in 1439 bytes} [Mon Aug 1 16:36:49 2022] PUT /identity/v3/projects/7b992d6a65df4bf9b27c13192d57c5f0/users/6533106d03eb4ca185a8839336d2aff9/roles/51bb93e277c446b49bb80a32afcd3af7 => generated 0 bytes in 244 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 16:36:49.603844 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62e7a563-6ff7-465c-8bd2-2435c8d1f96f None admin] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:36:49.623627 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG dbcounter [-] [87058] Writing DB stats keystone:SELECT=100 {{(pid=87058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:36:49.670865 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG dbcounter [-] [87057] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87057) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:36:49.718453 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62e7a563-6ff7-465c-8bd2-2435c8d1f96f None admin] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:36:49.718970 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9690722-29ee-4038-afca-51ee4f565b3c None admin] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:36:49.756922 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40c0409b-450c-4b2f-a022-3709dc85b17f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:36:49.757629 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c0409b-450c-4b2f-a022-3709dc85b17f None admin] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:36:49.757914 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c0409b-450c-4b2f-a022-3709dc85b17f None admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:36:49.758220 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c0409b-450c-4b2f-a022-3709dc85b17f None admin] PATH_INFO: `/v3/users/glance` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:36:49.759817 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca56d526-940d-4b96-a56b-edd161e81491 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:36:49.760481 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca56d526-940d-4b96-a56b-edd161e81491 None admin] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:36:49.760760 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca56d526-940d-4b96-a56b-edd161e81491 None admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:36:49.761034 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca56d526-940d-4b96-a56b-edd161e81491 None admin] PATH_INFO: `/v3/roles/service` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:36:49.768656 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40c0409b-450c-4b2f-a022-3709dc85b17f None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:36:49.770461 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40c0409b-450c-4b2f-a022-3709dc85b17f None admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:36:49.773695 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca56d526-940d-4b96-a56b-edd161e81491 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 16:36:49.778817 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application [None req-ca56d526-940d-4b96-a56b-edd161e81491 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 01 16:36:49.778817 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 16:36:49.778817 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 16:36:49.778817 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 16:36:49.778817 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 16:36:49.778817 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 16:36:49.778817 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 16:36:49.778817 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 16:36:49.778817 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 16:36:49.778817 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 16:36:49.778817 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 16:36:49.778817 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 16:36:49.778817 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 16:36:49.778817 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 16:36:49.778817 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 16:36:49.778817 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 16:36:49.779764 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 16:36:49.779764 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 16:36:49.779764 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 16:36:49.779764 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 16:36:49.779764 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 16:36:49.779764 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 16:36:49.779764 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 16:36:49.779764 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 16:36:49.779764 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 16:36:49.779764 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 16:36:49.779764 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 16:36:49.779764 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application with Lock( Aug 01 16:36:49.779764 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 16:36:49.779764 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self._enter() Aug 01 16:36:49.779764 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 16:36:49.779764 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 16:36:49.780620 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 16:36:49.780620 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self.creator() Aug 01 16:36:49.780620 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 16:36:49.780620 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application created_value = creator( Aug 01 16:36:49.780620 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 16:36:49.780620 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 16:36:49.780620 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 16:36:49.780620 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 16:36:49.780620 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 16:36:49.780620 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 16:36:49.780620 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 01 16:36:49.780620 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application Aug 01 16:36:49.780620 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 290/577] 10.222.0.20 () {56 vars in 1102 bytes} [Mon Aug 1 16:36:49 2022] GET /identity/v3/roles/service => generated 85 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 16:36:49.781340 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca56d526-940d-4b96-a56b-edd161e81491 None admin] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:36:49.784412 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application [None req-40c0409b-450c-4b2f-a022-3709dc85b17f None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Aug 01 16:36:49.784412 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 16:36:49.784412 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 16:36:49.784412 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 16:36:49.784412 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 16:36:49.784412 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 16:36:49.784412 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 16:36:49.784412 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 16:36:49.784412 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 16:36:49.784412 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 16:36:49.784412 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 16:36:49.784412 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 16:36:49.784412 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 01 16:36:49.784412 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 01 16:36:49.784412 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 01 16:36:49.784412 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 01 16:36:49.785364 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 16:36:49.785364 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 16:36:49.785364 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 01 16:36:49.785364 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 16:36:49.785364 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 01 16:36:49.785364 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 16:36:49.785364 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 16:36:49.785364 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 16:36:49.785364 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 16:36:49.785364 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 16:36:49.785364 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 16:36:49.785364 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application with Lock( Aug 01 16:36:49.785364 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 16:36:49.785364 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self._enter() Aug 01 16:36:49.785364 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 16:36:49.785364 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 16:36:49.785364 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 16:36:49.785364 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self.creator() Aug 01 16:36:49.786421 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 16:36:49.786421 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application created_value = creator( Aug 01 16:36:49.786421 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 01 16:36:49.786421 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 01 16:36:49.786421 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 01 16:36:49.786421 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 01 16:36:49.786421 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 01 16:36:49.786421 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 01 16:36:49.786421 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Aug 01 16:36:49.786421 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application Aug 01 16:36:49.786421 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 288/578] 10.222.0.20 () {56 vars in 1099 bytes} [Mon Aug 1 16:36:49 2022] GET /identity/v3/users/glance => generated 84 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 16:36:49.786421 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40c0409b-450c-4b2f-a022-3709dc85b17f None admin] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:36:49.868046 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG dbcounter [-] [87058] Writing DB stats keystone:SELECT=100 {{(pid=87058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:36:49.894881 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca56d526-940d-4b96-a56b-edd161e81491 None admin] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:36:49.896147 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40c0409b-450c-4b2f-a022-3709dc85b17f None admin] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:36:49.917709 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG dbcounter [-] [87057] Writing DB stats keystone:SELECT=100 {{(pid=87057) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:36:49.936764 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30250a6f-8deb-4cf6-ab88-ded5a8f41bd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:36:49.936951 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b93a42cf-1996-4826-8cb7-024494125edf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:36:49.937397 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93a42cf-1996-4826-8cb7-024494125edf None admin] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:36:49.937512 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30250a6f-8deb-4cf6-ab88-ded5a8f41bd5 None admin] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:36:49.937671 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30250a6f-8deb-4cf6-ab88-ded5a8f41bd5 None admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:36:49.937775 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93a42cf-1996-4826-8cb7-024494125edf None admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:36:49.937938 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30250a6f-8deb-4cf6-ab88-ded5a8f41bd5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:36:49.938049 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93a42cf-1996-4826-8cb7-024494125edf None admin] PATH_INFO: `/v3/domains` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:36:49.938690 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b93a42cf-1996-4826-8cb7-024494125edf None admin] RBAC: Adding query filter params (name=Default) {{(pid=87057) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:36:49.938965 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b93a42cf-1996-4826-8cb7-024494125edf None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:36:49.940526 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b93a42cf-1996-4826-8cb7-024494125edf None admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:36:49.948282 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30250a6f-8deb-4cf6-ab88-ded5a8f41bd5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:36:49.949735 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30250a6f-8deb-4cf6-ab88-ded5a8f41bd5 None admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:36:49.958332 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 289/579] 10.222.0.20 () {56 vars in 1109 bytes} [Mon Aug 1 16:36:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:36:49.958332 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b93a42cf-1996-4826-8cb7-024494125edf None admin] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:36:49.967331 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application [None req-30250a6f-8deb-4cf6-ab88-ded5a8f41bd5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 16:36:49.967331 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 16:36:49.967331 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 16:36:49.967331 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 16:36:49.967331 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 16:36:49.967331 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 16:36:49.967331 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 16:36:49.967331 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 16:36:49.967331 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 16:36:49.967331 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 16:36:49.967331 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 16:36:49.967331 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 16:36:49.967331 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 16:36:49.967331 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 16:36:49.967331 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 16:36:49.968242 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 16:36:49.968242 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 16:36:49.968242 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 16:36:49.968242 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 16:36:49.968242 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 16:36:49.968242 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 16:36:49.968242 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 16:36:49.968242 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 16:36:49.968242 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application with Lock( Aug 01 16:36:49.968242 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 16:36:49.968242 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self._enter() Aug 01 16:36:49.968242 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 16:36:49.968242 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 16:36:49.968242 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 16:36:49.968242 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self.creator() Aug 01 16:36:49.968242 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 16:36:49.968242 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application created_value = creator( Aug 01 16:36:49.969191 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 16:36:49.969191 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 16:36:49.969191 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 16:36:49.969191 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application Aug 01 16:36:49.973125 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 291/580] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Aug 1 16:36:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 16:36:49.973846 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30250a6f-8deb-4cf6-ab88-ded5a8f41bd5 None admin] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:36:50.071629 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b93a42cf-1996-4826-8cb7-024494125edf None admin] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:36:50.076741 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30250a6f-8deb-4cf6-ab88-ded5a8f41bd5 None admin] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:36:50.102431 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG dbcounter [-] [87058] Writing DB stats keystone:SELECT=100 {{(pid=87058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:36:50.107934 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa7ff1f-3ea5-484e-9ab5-ab5ddb942305 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:36:50.108561 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa7ff1f-3ea5-484e-9ab5-ab5ddb942305 None admin] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:36:50.108844 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa7ff1f-3ea5-484e-9ab5-ab5ddb942305 None admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:36:50.109119 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa7ff1f-3ea5-484e-9ab5-ab5ddb942305 None admin] PATH_INFO: `/v3/roles` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:36:50.109844 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fa7ff1f-3ea5-484e-9ab5-ab5ddb942305 None admin] RBAC: Adding query filter params (name=service) {{(pid=87057) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:36:50.110183 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fa7ff1f-3ea5-484e-9ab5-ab5ddb942305 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:36:50.116663 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fa7ff1f-3ea5-484e-9ab5-ab5ddb942305 None admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:36:50.116815 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d863649-d98a-4c1e-a25e-839b5fb047d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:36:50.116815 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d863649-d98a-4c1e-a25e-839b5fb047d8 None admin] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:36:50.116815 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d863649-d98a-4c1e-a25e-839b5fb047d8 None admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:36:50.116815 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d863649-d98a-4c1e-a25e-839b5fb047d8 None admin] PATH_INFO: `/v3/users` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:36:50.117114 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d863649-d98a-4c1e-a25e-839b5fb047d8 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=87058) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:36:50.117114 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d863649-d98a-4c1e-a25e-839b5fb047d8 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:36:50.117114 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d863649-d98a-4c1e-a25e-839b5fb047d8 None admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:36:50.121040 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 290/581] 10.222.0.20 () {56 vars in 1103 bytes} [Mon Aug 1 16:36:49 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:36:50.122953 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa7ff1f-3ea5-484e-9ab5-ab5ddb942305 None admin] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:36:50.138136 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 292/582] 10.222.0.20 () {56 vars in 1137 bytes} [Mon Aug 1 16:36:49 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:36:50.139435 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d863649-d98a-4c1e-a25e-839b5fb047d8 None admin] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:36:50.170377 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG dbcounter [-] [87057] Writing DB stats keystone:SELECT=100 {{(pid=87057) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:36:50.230411 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa7ff1f-3ea5-484e-9ab5-ab5ddb942305 None admin] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:36:50.254282 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d863649-d98a-4c1e-a25e-839b5fb047d8 None admin] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:36:50.265236 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8da2ba6-1eda-47d6-9e38-31334e7a63da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:36:50.265879 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8da2ba6-1eda-47d6-9e38-31334e7a63da None admin] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:36:50.266191 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8da2ba6-1eda-47d6-9e38-31334e7a63da None admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:36:50.266723 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8da2ba6-1eda-47d6-9e38-31334e7a63da None admin] PATH_INFO: `/v3/users/cinder` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:36:50.275287 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8da2ba6-1eda-47d6-9e38-31334e7a63da None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:36:50.276731 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8da2ba6-1eda-47d6-9e38-31334e7a63da None admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:36:50.288056 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application [None req-c8da2ba6-1eda-47d6-9e38-31334e7a63da None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 01 16:36:50.288056 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 16:36:50.288056 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 16:36:50.288056 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 16:36:50.288056 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 16:36:50.288056 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 16:36:50.288056 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 16:36:50.288056 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 16:36:50.288056 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 16:36:50.288056 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 16:36:50.288056 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 16:36:50.288056 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 16:36:50.288056 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 01 16:36:50.288056 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 01 16:36:50.288056 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 01 16:36:50.288056 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 01 16:36:50.288988 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 16:36:50.288988 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 16:36:50.288988 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 01 16:36:50.288988 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 16:36:50.288988 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 01 16:36:50.288988 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 16:36:50.288988 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 16:36:50.288988 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 16:36:50.288988 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 16:36:50.288988 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 16:36:50.288988 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 16:36:50.288988 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application with Lock( Aug 01 16:36:50.288988 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 16:36:50.288988 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self._enter() Aug 01 16:36:50.288988 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 16:36:50.288988 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 16:36:50.288988 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 16:36:50.288988 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self.creator() Aug 01 16:36:50.289960 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 16:36:50.289960 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application created_value = creator( Aug 01 16:36:50.289960 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 01 16:36:50.289960 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 01 16:36:50.289960 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 01 16:36:50.289960 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 01 16:36:50.289960 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 01 16:36:50.289960 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 01 16:36:50.289960 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 01 16:36:50.289960 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application Aug 01 16:36:50.289960 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 291/583] 10.222.0.20 () {56 vars in 1099 bytes} [Mon Aug 1 16:36:50 2022] GET /identity/v3/users/cinder => generated 84 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 16:36:50.291041 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8da2ba6-1eda-47d6-9e38-31334e7a63da None admin] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:36:50.300160 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c18e9f-bc72-4764-8910-75d367663708 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:36:50.300781 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c18e9f-bc72-4764-8910-75d367663708 None admin] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:36:50.301060 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c18e9f-bc72-4764-8910-75d367663708 None admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:36:50.301333 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c18e9f-bc72-4764-8910-75d367663708 None admin] PATH_INFO: `/v3/domains` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:36:50.302046 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20c18e9f-bc72-4764-8910-75d367663708 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87058) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:36:50.302351 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20c18e9f-bc72-4764-8910-75d367663708 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:36:50.304500 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20c18e9f-bc72-4764-8910-75d367663708 None admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:36:50.316872 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 293/584] 10.222.0.20 () {56 vars in 1109 bytes} [Mon Aug 1 16:36:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:36:50.317676 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c18e9f-bc72-4764-8910-75d367663708 None admin] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:36:50.371685 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG dbcounter [-] [87058] Writing DB stats keystone:SELECT=100 {{(pid=87058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:36:50.400745 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG dbcounter [-] [87057] Writing DB stats keystone:SELECT=100 {{(pid=87057) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:36:50.403843 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8da2ba6-1eda-47d6-9e38-31334e7a63da None admin] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:36:50.442935 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c18e9f-bc72-4764-8910-75d367663708 None admin] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:36:50.444297 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad44abc-1e10-4655-aab2-16bb2a789c97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:36:50.444923 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad44abc-1e10-4655-aab2-16bb2a789c97 None admin] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:36:50.445200 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad44abc-1e10-4655-aab2-16bb2a789c97 None admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:36:50.445469 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad44abc-1e10-4655-aab2-16bb2a789c97 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:36:50.455669 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ad44abc-1e10-4655-aab2-16bb2a789c97 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:36:50.457115 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ad44abc-1e10-4655-aab2-16bb2a789c97 None admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:36:50.472185 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application [None req-4ad44abc-1e10-4655-aab2-16bb2a789c97 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 16:36:50.472185 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 16:36:50.472185 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 16:36:50.472185 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 16:36:50.472185 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 16:36:50.472185 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 16:36:50.472185 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 16:36:50.472185 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 16:36:50.472185 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 16:36:50.472185 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 16:36:50.472185 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 16:36:50.472185 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 16:36:50.472185 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 16:36:50.472185 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 16:36:50.472185 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 16:36:50.473238 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 16:36:50.473238 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 16:36:50.473238 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 16:36:50.473238 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 16:36:50.473238 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 16:36:50.473238 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 16:36:50.473238 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 16:36:50.473238 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 16:36:50.473238 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application with Lock( Aug 01 16:36:50.473238 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 16:36:50.473238 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self._enter() Aug 01 16:36:50.473238 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 16:36:50.473238 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 16:36:50.473238 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 16:36:50.473238 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self.creator() Aug 01 16:36:50.473238 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 16:36:50.473238 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application created_value = creator( Aug 01 16:36:50.474285 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 16:36:50.474285 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 16:36:50.474285 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 16:36:50.474285 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application Aug 01 16:36:50.474285 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 292/585] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Aug 1 16:36:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 16:36:50.474583 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad44abc-1e10-4655-aab2-16bb2a789c97 None admin] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:36:50.484911 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d302b22-4703-44ff-b35e-99dfa2840326 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:36:50.485739 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d302b22-4703-44ff-b35e-99dfa2840326 None admin] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:36:50.486252 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d302b22-4703-44ff-b35e-99dfa2840326 None admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:36:50.486689 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d302b22-4703-44ff-b35e-99dfa2840326 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:36:50.497804 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d302b22-4703-44ff-b35e-99dfa2840326 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:36:50.499727 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d302b22-4703-44ff-b35e-99dfa2840326 None admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:36:50.514339 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application [None req-8d302b22-4703-44ff-b35e-99dfa2840326 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 16:36:50.514339 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 16:36:50.514339 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 16:36:50.514339 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 16:36:50.514339 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 16:36:50.514339 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 16:36:50.514339 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 16:36:50.514339 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 16:36:50.514339 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 16:36:50.514339 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 16:36:50.514339 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 16:36:50.514339 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 16:36:50.514339 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 16:36:50.514339 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 16:36:50.514339 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 16:36:50.515410 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 16:36:50.515410 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 16:36:50.515410 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 16:36:50.515410 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 16:36:50.515410 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 16:36:50.515410 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 16:36:50.515410 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 16:36:50.515410 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 16:36:50.515410 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application with Lock( Aug 01 16:36:50.515410 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 16:36:50.515410 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self._enter() Aug 01 16:36:50.515410 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 16:36:50.515410 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 16:36:50.515410 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 16:36:50.515410 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self.creator() Aug 01 16:36:50.515410 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 16:36:50.515410 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application created_value = creator( Aug 01 16:36:50.516778 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 16:36:50.516778 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 16:36:50.516778 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 16:36:50.516778 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application Aug 01 16:36:50.518265 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 294/586] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Aug 1 16:36:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 16:36:50.519165 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d302b22-4703-44ff-b35e-99dfa2840326 None admin] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:36:50.586654 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad44abc-1e10-4655-aab2-16bb2a789c97 None admin] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:36:50.619186 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG dbcounter [-] [87058] Writing DB stats keystone:SELECT=100 {{(pid=87058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:36:50.621881 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-910483cc-fadb-499d-95db-4b618687b520 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:36:50.622526 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910483cc-fadb-499d-95db-4b618687b520 None admin] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:36:50.622841 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910483cc-fadb-499d-95db-4b618687b520 None admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:36:50.623080 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910483cc-fadb-499d-95db-4b618687b520 None admin] PATH_INFO: `/v3/users` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:36:50.627387 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-910483cc-fadb-499d-95db-4b618687b520 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=87057) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:36:50.627387 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-910483cc-fadb-499d-95db-4b618687b520 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:36:50.627387 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-910483cc-fadb-499d-95db-4b618687b520 None admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:36:50.627675 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d302b22-4703-44ff-b35e-99dfa2840326 None admin] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:36:50.666068 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 293/587] 10.222.0.20 () {56 vars in 1137 bytes} [Mon Aug 1 16:36:50 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:36:50.667415 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-910483cc-fadb-499d-95db-4b618687b520 None admin] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:36:50.670101 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a113572-9d52-4e6d-8fe8-4198b292225f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:36:50.670759 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a113572-9d52-4e6d-8fe8-4198b292225f None admin] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:36:50.671037 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a113572-9d52-4e6d-8fe8-4198b292225f None admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:36:50.671308 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a113572-9d52-4e6d-8fe8-4198b292225f None admin] PATH_INFO: `/v3/projects/service` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:36:50.680261 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a113572-9d52-4e6d-8fe8-4198b292225f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:36:50.682104 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a113572-9d52-4e6d-8fe8-4198b292225f None admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:36:50.683228 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG dbcounter [-] [87057] Writing DB stats keystone:SELECT=100 {{(pid=87057) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:36:50.694572 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application [None req-7a113572-9d52-4e6d-8fe8-4198b292225f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 16:36:50.694572 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 16:36:50.694572 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 16:36:50.694572 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 16:36:50.694572 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 16:36:50.694572 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 16:36:50.694572 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 16:36:50.694572 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 16:36:50.694572 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 16:36:50.694572 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 16:36:50.694572 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 16:36:50.694572 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 16:36:50.694572 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 01 16:36:50.694572 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 01 16:36:50.694572 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 01 16:36:50.696345 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 01 16:36:50.696345 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 16:36:50.696345 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 16:36:50.696345 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 16:36:50.696345 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 16:36:50.696345 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 16:36:50.696345 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 16:36:50.696345 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 16:36:50.696345 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application with Lock( Aug 01 16:36:50.696345 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 16:36:50.696345 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self._enter() Aug 01 16:36:50.696345 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 16:36:50.696345 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 16:36:50.696345 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 16:36:50.696345 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self.creator() Aug 01 16:36:50.696345 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 16:36:50.696345 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application created_value = creator( Aug 01 16:36:50.697332 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 01 16:36:50.697332 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 01 16:36:50.697332 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 01 16:36:50.697332 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 01 16:36:50.697332 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 01 16:36:50.697332 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 01 16:36:50.697332 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 16:36:50.697332 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application Aug 01 16:36:50.697332 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 295/588] 10.222.0.20 () {56 vars in 1111 bytes} [Mon Aug 1 16:36:50 2022] GET /identity/v3/projects/service => generated 88 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 16:36:50.697332 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a113572-9d52-4e6d-8fe8-4198b292225f None admin] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:36:50.787312 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-910483cc-fadb-499d-95db-4b618687b520 None admin] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:36:50.808563 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a113572-9d52-4e6d-8fe8-4198b292225f None admin] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:36:50.824199 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a2ec188-16e0-40ea-8e9b-f5ce6ec3aabc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:36:50.824818 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2ec188-16e0-40ea-8e9b-f5ce6ec3aabc None admin] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:36:50.825093 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2ec188-16e0-40ea-8e9b-f5ce6ec3aabc None admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:36:50.825368 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2ec188-16e0-40ea-8e9b-f5ce6ec3aabc None admin] PATH_INFO: `/v3/domains` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:36:50.826082 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a2ec188-16e0-40ea-8e9b-f5ce6ec3aabc None admin] RBAC: Adding query filter params (name=Default) {{(pid=87057) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:36:50.826412 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a2ec188-16e0-40ea-8e9b-f5ce6ec3aabc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:36:50.827767 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a2ec188-16e0-40ea-8e9b-f5ce6ec3aabc None admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:36:50.840069 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 294/589] 10.222.0.20 () {56 vars in 1109 bytes} [Mon Aug 1 16:36:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:36:50.843211 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a2ec188-16e0-40ea-8e9b-f5ce6ec3aabc None admin] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:36:50.844641 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-255808fc-a2d3-41d7-bd9f-93c162bc9cbc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:36:50.845263 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255808fc-a2d3-41d7-bd9f-93c162bc9cbc None admin] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:36:50.845538 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255808fc-a2d3-41d7-bd9f-93c162bc9cbc None admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:36:50.845807 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255808fc-a2d3-41d7-bd9f-93c162bc9cbc None admin] PATH_INFO: `/v3/domains` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:36:50.846526 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-255808fc-a2d3-41d7-bd9f-93c162bc9cbc None admin] RBAC: Adding query filter params (name=Default) {{(pid=87058) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:36:50.846847 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-255808fc-a2d3-41d7-bd9f-93c162bc9cbc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:36:50.848631 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-255808fc-a2d3-41d7-bd9f-93c162bc9cbc None admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:36:50.862545 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 296/590] 10.222.0.20 () {56 vars in 1109 bytes} [Mon Aug 1 16:36:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:36:50.863384 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-255808fc-a2d3-41d7-bd9f-93c162bc9cbc None admin] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:36:50.881113 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG dbcounter [-] [87058] Writing DB stats keystone:SELECT=100 {{(pid=87058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:36:50.935875 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG dbcounter [-] [87057] Writing DB stats keystone:SELECT=100 {{(pid=87057) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:36:50.966899 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a2ec188-16e0-40ea-8e9b-f5ce6ec3aabc None admin] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:36:50.976401 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-255808fc-a2d3-41d7-bd9f-93c162bc9cbc None admin] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:36:51.002746 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d4bf6c-2b43-4b15-8656-20a1b5c51688 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:36:51.003375 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d4bf6c-2b43-4b15-8656-20a1b5c51688 None admin] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:36:51.003654 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d4bf6c-2b43-4b15-8656-20a1b5c51688 None admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:36:51.003924 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d4bf6c-2b43-4b15-8656-20a1b5c51688 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:36:51.014731 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61d4bf6c-2b43-4b15-8656-20a1b5c51688 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:36:51.016174 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61d4bf6c-2b43-4b15-8656-20a1b5c51688 None admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:36:51.020489 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a3f5b9-f79b-4396-b3b3-2eeb0c6b66c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:36:51.021123 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a3f5b9-f79b-4396-b3b3-2eeb0c6b66c6 None admin] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:36:51.021405 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a3f5b9-f79b-4396-b3b3-2eeb0c6b66c6 None admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:36:51.021679 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a3f5b9-f79b-4396-b3b3-2eeb0c6b66c6 None admin] PATH_INFO: `/v3/projects` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:36:51.022504 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8a3f5b9-f79b-4396-b3b3-2eeb0c6b66c6 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87058) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:36:51.022814 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8a3f5b9-f79b-4396-b3b3-2eeb0c6b66c6 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:36:51.024241 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8a3f5b9-f79b-4396-b3b3-2eeb0c6b66c6 None admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:36:51.031412 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application [None req-61d4bf6c-2b43-4b15-8656-20a1b5c51688 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 16:36:51.031412 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 16:36:51.031412 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 16:36:51.031412 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 16:36:51.031412 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 16:36:51.031412 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 16:36:51.031412 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 16:36:51.031412 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 16:36:51.031412 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 16:36:51.031412 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 16:36:51.031412 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 16:36:51.031412 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 16:36:51.031412 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 16:36:51.031412 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 16:36:51.031412 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 16:36:51.032478 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 16:36:51.032478 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 16:36:51.032478 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 16:36:51.032478 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 16:36:51.032478 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 16:36:51.032478 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 16:36:51.032478 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 16:36:51.032478 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 16:36:51.032478 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application with Lock( Aug 01 16:36:51.032478 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 16:36:51.032478 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self._enter() Aug 01 16:36:51.032478 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 16:36:51.032478 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 16:36:51.032478 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 16:36:51.032478 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self.creator() Aug 01 16:36:51.032478 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 16:36:51.032478 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application created_value = creator( Aug 01 16:36:51.033642 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 16:36:51.033642 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 16:36:51.033642 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 16:36:51.033642 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application Aug 01 16:36:51.033642 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 295/591] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Aug 1 16:36:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 16:36:51.033642 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d4bf6c-2b43-4b15-8656-20a1b5c51688 None admin] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:36:51.042402 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 297/592] 10.222.0.20 () {56 vars in 1148 bytes} [Mon Aug 1 16:36:50 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:36:51.045921 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a3f5b9-f79b-4396-b3b3-2eeb0c6b66c6 None admin] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:36:51.136598 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG dbcounter [-] [87058] Writing DB stats keystone:SELECT=100 {{(pid=87058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:36:51.137027 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d4bf6c-2b43-4b15-8656-20a1b5c51688 None admin] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:36:51.161504 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG dbcounter [-] [87057] Writing DB stats keystone:SELECT=100 {{(pid=87057) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:36:51.162054 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a3f5b9-f79b-4396-b3b3-2eeb0c6b66c6 None admin] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:36:51.173711 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb12e92-9716-4e8d-9169-2671a9b757e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:36:51.174377 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb12e92-9716-4e8d-9169-2671a9b757e3 None admin] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:36:51.174673 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb12e92-9716-4e8d-9169-2671a9b757e3 None admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:36:51.174945 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb12e92-9716-4e8d-9169-2671a9b757e3 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:36:51.182598 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fb12e92-9716-4e8d-9169-2671a9b757e3 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:36:51.184035 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fb12e92-9716-4e8d-9169-2671a9b757e3 None admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:36:51.196853 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe31db1-5aac-4616-81ee-4738dc44f220 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:36:51.197502 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe31db1-5aac-4616-81ee-4738dc44f220 None admin] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:36:51.197833 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application [None req-6fb12e92-9716-4e8d-9169-2671a9b757e3 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Aug 01 16:36:51.197833 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 16:36:51.197833 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 16:36:51.197833 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 16:36:51.197833 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 16:36:51.197833 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 16:36:51.197833 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 16:36:51.197833 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 16:36:51.197833 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 16:36:51.197833 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 16:36:51.197833 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 16:36:51.197833 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 16:36:51.197833 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 01 16:36:51.197833 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 01 16:36:51.197833 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 01 16:36:51.197833 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 01 16:36:51.198933 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 16:36:51.198933 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 16:36:51.198933 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 01 16:36:51.198933 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 16:36:51.198933 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 01 16:36:51.198933 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 16:36:51.198933 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 16:36:51.198933 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 16:36:51.198933 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 16:36:51.198933 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 16:36:51.198933 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 16:36:51.198933 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application with Lock( Aug 01 16:36:51.198933 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 16:36:51.198933 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self._enter() Aug 01 16:36:51.198933 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 16:36:51.198933 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 16:36:51.198933 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 16:36:51.198933 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self.creator() Aug 01 16:36:51.200099 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 16:36:51.200099 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application created_value = creator( Aug 01 16:36:51.200099 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 01 16:36:51.200099 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 01 16:36:51.200099 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 01 16:36:51.200099 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 01 16:36:51.200099 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 01 16:36:51.200099 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 01 16:36:51.200099 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Aug 01 16:36:51.200099 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application Aug 01 16:36:51.200743 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe31db1-5aac-4616-81ee-4738dc44f220 None admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:36:51.200743 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe31db1-5aac-4616-81ee-4738dc44f220 None admin] PATH_INFO: `/v3/projects/service` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:36:51.202868 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 296/593] 10.222.0.20 () {56 vars in 1102 bytes} [Mon Aug 1 16:36:51 2022] GET /identity/v3/users/neutron => generated 85 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 16:36:51.203806 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb12e92-9716-4e8d-9169-2671a9b757e3 None admin] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:36:51.206289 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fe31db1-5aac-4616-81ee-4738dc44f220 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:36:51.217193 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fe31db1-5aac-4616-81ee-4738dc44f220 None admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:36:51.219678 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application [None req-7fe31db1-5aac-4616-81ee-4738dc44f220 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 16:36:51.219678 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 16:36:51.219678 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 16:36:51.219678 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 16:36:51.219678 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 16:36:51.219678 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 16:36:51.219678 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 16:36:51.219678 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 16:36:51.219678 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 16:36:51.219678 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 16:36:51.219678 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 16:36:51.219678 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 16:36:51.219678 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 01 16:36:51.219678 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 01 16:36:51.219678 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 01 16:36:51.220740 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 01 16:36:51.220740 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 16:36:51.220740 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 16:36:51.220740 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 16:36:51.220740 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 16:36:51.220740 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 16:36:51.220740 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 16:36:51.220740 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 16:36:51.220740 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application with Lock( Aug 01 16:36:51.220740 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 16:36:51.220740 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self._enter() Aug 01 16:36:51.220740 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 16:36:51.220740 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 16:36:51.220740 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 16:36:51.220740 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self.creator() Aug 01 16:36:51.220740 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 16:36:51.220740 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application created_value = creator( Aug 01 16:36:51.221657 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 01 16:36:51.221657 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 01 16:36:51.221657 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 01 16:36:51.221657 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 01 16:36:51.221657 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 01 16:36:51.221657 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 01 16:36:51.221657 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 16:36:51.221657 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application Aug 01 16:36:51.221657 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 298/594] 10.222.0.20 () {56 vars in 1111 bytes} [Mon Aug 1 16:36:51 2022] GET /identity/v3/projects/service => generated 88 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 16:36:51.222255 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe31db1-5aac-4616-81ee-4738dc44f220 None admin] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:36:51.324384 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb12e92-9716-4e8d-9169-2671a9b757e3 None admin] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:36:51.330096 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe31db1-5aac-4616-81ee-4738dc44f220 None admin] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:36:51.361007 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d424cf9-57dd-433c-8c2d-eaee8f8cc174 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:36:51.361662 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d424cf9-57dd-433c-8c2d-eaee8f8cc174 None admin] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:36:51.362004 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d424cf9-57dd-433c-8c2d-eaee8f8cc174 None admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:36:51.362299 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d424cf9-57dd-433c-8c2d-eaee8f8cc174 None admin] PATH_INFO: `/v3/domains` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:36:51.363025 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d424cf9-57dd-433c-8c2d-eaee8f8cc174 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87057) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:36:51.363274 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d424cf9-57dd-433c-8c2d-eaee8f8cc174 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:36:51.364637 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d424cf9-57dd-433c-8c2d-eaee8f8cc174 None admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:36:51.368922 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4cc7f2c-cc5d-4ff1-a05c-d9ebf0d4cabb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:36:51.369533 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4cc7f2c-cc5d-4ff1-a05c-d9ebf0d4cabb None admin] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:36:51.369811 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4cc7f2c-cc5d-4ff1-a05c-d9ebf0d4cabb None admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:36:51.370115 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4cc7f2c-cc5d-4ff1-a05c-d9ebf0d4cabb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:36:51.370903 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4cc7f2c-cc5d-4ff1-a05c-d9ebf0d4cabb None admin] RBAC: Adding query filter params (role.id=51bb93e277c446b49bb80a32afcd3af7, scope.project.id=7b992d6a65df4bf9b27c13192d57c5f0, user.id=9ab4add1faa04faa811ef7a9341cc35c) {{(pid=87058) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:36:51.371214 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4cc7f2c-cc5d-4ff1-a05c-d9ebf0d4cabb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:36:51.372646 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4cc7f2c-cc5d-4ff1-a05c-d9ebf0d4cabb None admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:36:51.375993 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG dbcounter [-] [87058] Writing DB stats keystone:SELECT=100 {{(pid=87058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:36:51.377032 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 297/595] 10.222.0.20 () {56 vars in 1109 bytes} [Mon Aug 1 16:36:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:36:51.378918 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d424cf9-57dd-433c-8c2d-eaee8f8cc174 None admin] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:36:51.379550 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 299/596] 10.222.0.20 () {56 vars in 1374 bytes} [Mon Aug 1 16:36:51 2022] GET /identity/v3/role_assignments?user.id=9ab4add1faa04faa811ef7a9341cc35c&scope.project.id=7b992d6a65df4bf9b27c13192d57c5f0&role.id=51bb93e277c446b49bb80a32afcd3af7 => generated 259 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:36:51.380514 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4cc7f2c-cc5d-4ff1-a05c-d9ebf0d4cabb None admin] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:36:51.451841 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG dbcounter [-] [87057] Writing DB stats keystone:SELECT=100 {{(pid=87057) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:36:51.508519 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4cc7f2c-cc5d-4ff1-a05c-d9ebf0d4cabb None admin] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:36:51.536245 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d424cf9-57dd-433c-8c2d-eaee8f8cc174 None admin] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:36:51.549339 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f34fde00-178f-4ef9-86cb-6f1ccfbb8796 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:36:51.549968 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34fde00-178f-4ef9-86cb-6f1ccfbb8796 None admin] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:36:51.550822 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34fde00-178f-4ef9-86cb-6f1ccfbb8796 None admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:36:51.550822 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34fde00-178f-4ef9-86cb-6f1ccfbb8796 None admin] PATH_INFO: `/v3/projects` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:36:51.551311 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f34fde00-178f-4ef9-86cb-6f1ccfbb8796 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87058) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:36:51.551579 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f34fde00-178f-4ef9-86cb-6f1ccfbb8796 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:36:51.553016 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f34fde00-178f-4ef9-86cb-6f1ccfbb8796 None admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:36:51.570548 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 300/597] 10.222.0.20 () {56 vars in 1148 bytes} [Mon Aug 1 16:36:51 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:36:51.571417 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f34fde00-178f-4ef9-86cb-6f1ccfbb8796 None admin] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:36:51.580736 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d0f20fd-93e0-4036-8a78-37d3bd4eff2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:36:51.582089 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0f20fd-93e0-4036-8a78-37d3bd4eff2c None admin] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:36:51.582604 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0f20fd-93e0-4036-8a78-37d3bd4eff2c None admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:36:51.583083 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0f20fd-93e0-4036-8a78-37d3bd4eff2c None admin] PATH_INFO: `/v3/users` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:36:51.584555 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d0f20fd-93e0-4036-8a78-37d3bd4eff2c None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=87057) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:36:51.585576 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d0f20fd-93e0-4036-8a78-37d3bd4eff2c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:36:51.587910 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d0f20fd-93e0-4036-8a78-37d3bd4eff2c None admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:36:51.610983 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 298/598] 10.222.0.20 () {56 vars in 1139 bytes} [Mon Aug 1 16:36:51 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:36:51.613070 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d0f20fd-93e0-4036-8a78-37d3bd4eff2c None admin] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:36:51.642592 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG dbcounter [-] [87058] Writing DB stats keystone:SELECT=100 {{(pid=87058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:36:51.690848 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f34fde00-178f-4ef9-86cb-6f1ccfbb8796 None admin] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:36:51.724894 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG dbcounter [-] [87057] Writing DB stats keystone:SELECT=100 {{(pid=87057) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:36:51.728282 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d0f20fd-93e0-4036-8a78-37d3bd4eff2c None admin] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:36:51.729739 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ac7503-061a-4bbb-acc3-682950982567 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:36:51.730381 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ac7503-061a-4bbb-acc3-682950982567 None admin] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:36:51.730668 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ac7503-061a-4bbb-acc3-682950982567 None admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:36:51.730942 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ac7503-061a-4bbb-acc3-682950982567 None admin] PATH_INFO: `/v3/projects/service` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:36:51.738127 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3ac7503-061a-4bbb-acc3-682950982567 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:36:51.739745 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3ac7503-061a-4bbb-acc3-682950982567 None admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:36:51.751765 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application [None req-a3ac7503-061a-4bbb-acc3-682950982567 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 16:36:51.751765 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 16:36:51.751765 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 16:36:51.751765 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 16:36:51.751765 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 16:36:51.751765 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 16:36:51.751765 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 16:36:51.751765 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 16:36:51.751765 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 16:36:51.751765 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 16:36:51.751765 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 16:36:51.751765 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 16:36:51.751765 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 01 16:36:51.751765 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 01 16:36:51.751765 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 01 16:36:51.752752 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 01 16:36:51.752752 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 16:36:51.752752 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 16:36:51.752752 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 16:36:51.752752 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 16:36:51.752752 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 16:36:51.752752 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 16:36:51.752752 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 16:36:51.752752 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application with Lock( Aug 01 16:36:51.752752 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 16:36:51.752752 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self._enter() Aug 01 16:36:51.752752 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 16:36:51.752752 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 16:36:51.752752 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 16:36:51.752752 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self.creator() Aug 01 16:36:51.752752 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 16:36:51.752752 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application created_value = creator( Aug 01 16:36:51.753668 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 01 16:36:51.753668 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 01 16:36:51.753668 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 01 16:36:51.753668 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 01 16:36:51.753668 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 01 16:36:51.753668 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 01 16:36:51.753668 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 16:36:51.753668 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application Aug 01 16:36:51.753668 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 301/599] 10.222.0.20 () {56 vars in 1111 bytes} [Mon Aug 1 16:36:51 2022] GET /identity/v3/projects/service => generated 88 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 16:36:51.753668 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ac7503-061a-4bbb-acc3-682950982567 None admin] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:36:51.771713 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a32c4027-cc73-44e3-b4b2-c6da29b4da47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:36:51.773098 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a32c4027-cc73-44e3-b4b2-c6da29b4da47 None admin] REQUEST_METHOD: `PUT` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:36:51.773556 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a32c4027-cc73-44e3-b4b2-c6da29b4da47 None admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:36:51.773974 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a32c4027-cc73-44e3-b4b2-c6da29b4da47 None admin] PATH_INFO: `/v3/projects/7b992d6a65df4bf9b27c13192d57c5f0/users/d80694e439e2401d9491712666409016/roles/51bb93e277c446b49bb80a32afcd3af7` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:36:51.790001 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a32c4027-cc73-44e3-b4b2-c6da29b4da47 None admin] RBAC: Authorizing `identity:create_grant(project_id=7b992d6a65df4bf9b27c13192d57c5f0, user_id=d80694e439e2401d9491712666409016, role_id=51bb93e277c446b49bb80a32afcd3af7)` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:36:51.791664 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a32c4027-cc73-44e3-b4b2-c6da29b4da47 None admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:36:51.860714 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ac7503-061a-4bbb-acc3-682950982567 None admin] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:36:51.862635 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 299/600] 10.222.0.20 () {58 vars in 1439 bytes} [Mon Aug 1 16:36:51 2022] PUT /identity/v3/projects/7b992d6a65df4bf9b27c13192d57c5f0/users/d80694e439e2401d9491712666409016/roles/51bb93e277c446b49bb80a32afcd3af7 => generated 0 bytes in 250 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 16:36:51.863619 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712f34d1-0ff7-4686-8a9c-8a8563f04d0d None None] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:36:51.863619 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712f34d1-0ff7-4686-8a9c-8a8563f04d0d None None] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:36:51.863619 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712f34d1-0ff7-4686-8a9c-8a8563f04d0d None None] PATH_INFO: `/` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:36:51.865684 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 300/601] 10.222.0.20 () {54 vars in 926 bytes} [Mon Aug 1 16:36:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 16:36:51.865894 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-712f34d1-0ff7-4686-8a9c-8a8563f04d0d None None] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:36:51.888998 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG dbcounter [-] [87058] Writing DB stats keystone:SELECT=100 {{(pid=87058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:36:51.917602 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-950e81e1-082e-450e-8891-9074619f4a30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:36:51.918501 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950e81e1-082e-450e-8891-9074619f4a30 None admin] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:36:51.918790 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950e81e1-082e-450e-8891-9074619f4a30 None admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:36:51.919943 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950e81e1-082e-450e-8891-9074619f4a30 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:36:51.935252 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-950e81e1-082e-450e-8891-9074619f4a30 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:36:51.937421 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-950e81e1-082e-450e-8891-9074619f4a30 None admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:36:51.959431 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application [None req-950e81e1-082e-450e-8891-9074619f4a30 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 16:36:51.959431 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 16:36:51.959431 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 16:36:51.959431 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 16:36:51.959431 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 16:36:51.959431 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 16:36:51.959431 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 16:36:51.959431 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 16:36:51.959431 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 16:36:51.959431 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 16:36:51.959431 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 16:36:51.959431 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 16:36:51.959431 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 16:36:51.959431 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 16:36:51.959431 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 16:36:51.960405 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 16:36:51.960405 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 16:36:51.960405 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 16:36:51.960405 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 16:36:51.960405 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 16:36:51.960405 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 16:36:51.960405 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 16:36:51.960405 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 16:36:51.960405 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application with Lock( Aug 01 16:36:51.960405 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 16:36:51.960405 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self._enter() Aug 01 16:36:51.960405 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 16:36:51.960405 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 16:36:51.960405 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 16:36:51.960405 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self.creator() Aug 01 16:36:51.960405 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 16:36:51.960405 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application created_value = creator( Aug 01 16:36:51.961552 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 16:36:51.961552 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 16:36:51.961552 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 16:36:51.961552 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application Aug 01 16:36:51.961552 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 302/602] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Aug 1 16:36:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 16:36:51.962607 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6073ba82-d325-429d-9ab8-6a300dda1aad None None] REQUEST_METHOD: `POST` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:36:51.963747 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6073ba82-d325-429d-9ab8-6a300dda1aad None None] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:36:51.964027 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6073ba82-d325-429d-9ab8-6a300dda1aad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:36:51.989015 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.auth.core [None req-6073ba82-d325-429d-9ab8-6a300dda1aad None None] MFA Rules not processed for user `1007f8e379b74e169b98fb7c3c0ba5b4`. Rule list: `[]` (Enabled: `True`). {{(pid=87058) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:36:51.991300 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.fernet_utils [None req-6073ba82-d325-429d-9ab8-6a300dda1aad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:36:52.001353 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-712f34d1-0ff7-4686-8a9c-8a8563f04d0d None None] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:36:52.041051 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6443bd40-b90d-4a20-a525-73a2b0d2c718 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:36:52.041823 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6443bd40-b90d-4a20-a525-73a2b0d2c718 None admin] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:36:52.042252 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6443bd40-b90d-4a20-a525-73a2b0d2c718 None admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:36:52.042622 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6443bd40-b90d-4a20-a525-73a2b0d2c718 None admin] PATH_INFO: `/v3/projects` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:36:52.043571 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6443bd40-b90d-4a20-a525-73a2b0d2c718 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87057) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:36:52.044597 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6443bd40-b90d-4a20-a525-73a2b0d2c718 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:36:52.046129 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6443bd40-b90d-4a20-a525-73a2b0d2c718 None admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:36:52.060828 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 301/603] 10.222.0.20 () {56 vars in 1148 bytes} [Mon Aug 1 16:36:51 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:36:52.061349 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6443bd40-b90d-4a20-a525-73a2b0d2c718 None admin] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:36:52.074903 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG dbcounter [-] [87057] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87057) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:36:52.080602 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 303/604] 10.222.0.20 () {58 vars in 1023 bytes} [Mon Aug 1 16:36:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 16:36:52.093915 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6073ba82-d325-429d-9ab8-6a300dda1aad None None] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:36:52.195355 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6443bd40-b90d-4a20-a525-73a2b0d2c718 None admin] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:36:52.196364 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6073ba82-d325-429d-9ab8-6a300dda1aad None None] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:36:52.198287 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG dbcounter [-] [87058] Writing DB stats keystone:SELECT=100 {{(pid=87058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:36:52.240925 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-901248fc-d922-44f9-8a36-5376dc3089b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:36:52.241676 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901248fc-d922-44f9-8a36-5376dc3089b3 None admin] REQUEST_METHOD: `PUT` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:36:52.241960 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901248fc-d922-44f9-8a36-5376dc3089b3 None admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:36:52.242704 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f2efb4c-8dec-47e0-a6e6-53337ebc0acf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:36:52.243377 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2efb4c-8dec-47e0-a6e6-53337ebc0acf None admin] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:36:52.243659 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2efb4c-8dec-47e0-a6e6-53337ebc0acf None admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:36:52.243930 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2efb4c-8dec-47e0-a6e6-53337ebc0acf None admin] PATH_INFO: `/v3/domains` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:36:52.244626 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f2efb4c-8dec-47e0-a6e6-53337ebc0acf None admin] RBAC: Adding query filter params (name=Default) {{(pid=87057) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:36:52.244928 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f2efb4c-8dec-47e0-a6e6-53337ebc0acf None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:36:52.246343 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f2efb4c-8dec-47e0-a6e6-53337ebc0acf None admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:36:52.246459 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901248fc-d922-44f9-8a36-5376dc3089b3 None admin] PATH_INFO: `/v3/projects/7b992d6a65df4bf9b27c13192d57c5f0/users/25970b52ca15448fa1abb1f974d64038/roles/51bb93e277c446b49bb80a32afcd3af7` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:36:52.259714 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 302/605] 10.222.0.20 () {56 vars in 1109 bytes} [Mon Aug 1 16:36:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:36:52.260870 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f2efb4c-8dec-47e0-a6e6-53337ebc0acf None admin] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:36:52.264877 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-901248fc-d922-44f9-8a36-5376dc3089b3 None admin] RBAC: Authorizing `identity:create_grant(project_id=7b992d6a65df4bf9b27c13192d57c5f0, user_id=25970b52ca15448fa1abb1f974d64038, role_id=51bb93e277c446b49bb80a32afcd3af7)` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:36:52.267035 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-901248fc-d922-44f9-8a36-5376dc3089b3 None admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:36:52.330443 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 304/606] 10.222.0.20 () {58 vars in 1439 bytes} [Mon Aug 1 16:36:52 2022] PUT /identity/v3/projects/7b992d6a65df4bf9b27c13192d57c5f0/users/25970b52ca15448fa1abb1f974d64038/roles/51bb93e277c446b49bb80a32afcd3af7 => generated 0 bytes in 244 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 16:36:52.331966 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-901248fc-d922-44f9-8a36-5376dc3089b3 None admin] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:36:52.338527 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG dbcounter [-] [87057] Writing DB stats keystone:SELECT=100 {{(pid=87057) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:36:52.389056 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f2efb4c-8dec-47e0-a6e6-53337ebc0acf None admin] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:36:52.456783 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aad13e4e-cb1f-44ef-bbae-ddb1a2de6219 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:36:52.457359 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad13e4e-cb1f-44ef-bbae-ddb1a2de6219 None admin] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:36:52.457643 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad13e4e-cb1f-44ef-bbae-ddb1a2de6219 None admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:36:52.457917 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad13e4e-cb1f-44ef-bbae-ddb1a2de6219 None admin] PATH_INFO: `/v3/roles/service` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:36:52.481925 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aad13e4e-cb1f-44ef-bbae-ddb1a2de6219 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 16:36:52.489813 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application [None req-aad13e4e-cb1f-44ef-bbae-ddb1a2de6219 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 01 16:36:52.489813 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 16:36:52.489813 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 16:36:52.489813 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 16:36:52.489813 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 16:36:52.489813 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 16:36:52.489813 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 16:36:52.489813 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 16:36:52.489813 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 16:36:52.489813 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 16:36:52.489813 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 16:36:52.489813 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 16:36:52.489813 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 16:36:52.489813 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 16:36:52.489813 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 16:36:52.489813 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 16:36:52.491437 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 16:36:52.491437 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 16:36:52.491437 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 16:36:52.491437 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 16:36:52.491437 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 16:36:52.491437 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 16:36:52.491437 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 16:36:52.491437 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 16:36:52.491437 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 16:36:52.491437 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 16:36:52.491437 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 16:36:52.491437 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application with Lock( Aug 01 16:36:52.491437 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 16:36:52.491437 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self._enter() Aug 01 16:36:52.491437 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 16:36:52.491437 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 16:36:52.492729 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 16:36:52.492729 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self.creator() Aug 01 16:36:52.492729 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 16:36:52.492729 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application created_value = creator( Aug 01 16:36:52.492729 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 16:36:52.492729 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 16:36:52.492729 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 16:36:52.492729 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 16:36:52.492729 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 16:36:52.492729 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 16:36:52.492729 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 01 16:36:52.492729 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application Aug 01 16:36:52.493374 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 303/607] 10.222.0.20 () {56 vars in 1102 bytes} [Mon Aug 1 16:36:52 2022] GET /identity/v3/roles/service => generated 85 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 16:36:52.499977 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aad13e4e-cb1f-44ef-bbae-ddb1a2de6219 None admin] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:36:52.544562 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-901248fc-d922-44f9-8a36-5376dc3089b3 None admin] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:36:52.585296 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf3a680-802c-4f47-9b25-c43927f82220 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:36:52.586471 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf3a680-802c-4f47-9b25-c43927f82220 None admin] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:36:52.586955 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf3a680-802c-4f47-9b25-c43927f82220 None admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:36:52.587499 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf3a680-802c-4f47-9b25-c43927f82220 None admin] PATH_INFO: `/v3/projects/service` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:36:52.595316 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acf3a680-802c-4f47-9b25-c43927f82220 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:36:52.597137 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acf3a680-802c-4f47-9b25-c43927f82220 None admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:36:52.610573 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application [None req-acf3a680-802c-4f47-9b25-c43927f82220 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 16:36:52.610573 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 16:36:52.610573 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 16:36:52.610573 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 16:36:52.610573 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 16:36:52.610573 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 16:36:52.610573 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 16:36:52.610573 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 16:36:52.610573 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 16:36:52.610573 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 16:36:52.610573 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 16:36:52.610573 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 16:36:52.610573 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 01 16:36:52.610573 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 01 16:36:52.610573 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 01 16:36:52.611531 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 01 16:36:52.611531 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 16:36:52.611531 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 16:36:52.611531 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 16:36:52.611531 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 16:36:52.611531 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 16:36:52.611531 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 16:36:52.611531 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 16:36:52.611531 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application with Lock( Aug 01 16:36:52.611531 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 16:36:52.611531 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self._enter() Aug 01 16:36:52.611531 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 16:36:52.611531 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 16:36:52.611531 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 16:36:52.611531 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self.creator() Aug 01 16:36:52.611531 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 16:36:52.611531 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application created_value = creator( Aug 01 16:36:52.612502 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 01 16:36:52.612502 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 01 16:36:52.612502 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 01 16:36:52.612502 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 01 16:36:52.612502 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 01 16:36:52.612502 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 01 16:36:52.612502 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 16:36:52.612502 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application Aug 01 16:36:52.615138 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 305/608] 10.222.0.20 () {56 vars in 1111 bytes} [Mon Aug 1 16:36:52 2022] GET /identity/v3/projects/service => generated 88 bytes in 284 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 16:36:52.623436 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf3a680-802c-4f47-9b25-c43927f82220 None admin] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:36:52.627419 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aad13e4e-cb1f-44ef-bbae-ddb1a2de6219 None admin] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:36:52.640396 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG dbcounter [-] [87058] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:36:52.660424 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG dbcounter [-] [87057] Writing DB stats keystone:SELECT=100 {{(pid=87057) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:36:52.667375 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5fcf715-5e86-4de8-8c8f-3b76d2e310f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:36:52.668004 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5fcf715-5e86-4de8-8c8f-3b76d2e310f8 None admin] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:36:52.668284 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5fcf715-5e86-4de8-8c8f-3b76d2e310f8 None admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:36:52.668561 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5fcf715-5e86-4de8-8c8f-3b76d2e310f8 None admin] PATH_INFO: `/v3/roles` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:36:52.669286 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5fcf715-5e86-4de8-8c8f-3b76d2e310f8 None admin] RBAC: Adding query filter params (name=service) {{(pid=87057) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:36:52.669589 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5fcf715-5e86-4de8-8c8f-3b76d2e310f8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:36:52.671069 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5fcf715-5e86-4de8-8c8f-3b76d2e310f8 None admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:36:52.680768 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 304/609] 10.222.0.20 () {56 vars in 1103 bytes} [Mon Aug 1 16:36:52 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:36:52.686686 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5fcf715-5e86-4de8-8c8f-3b76d2e310f8 None admin] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:36:52.778457 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf3a680-802c-4f47-9b25-c43927f82220 None admin] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:36:52.807774 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5fcf715-5e86-4de8-8c8f-3b76d2e310f8 None admin] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:36:52.848130 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c47abd-1e83-48b4-abe9-74011215f5f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:36:52.848773 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c47abd-1e83-48b4-abe9-74011215f5f2 None admin] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:36:52.849054 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c47abd-1e83-48b4-abe9-74011215f5f2 None admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:36:52.849328 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c47abd-1e83-48b4-abe9-74011215f5f2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:36:52.861420 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71c47abd-1e83-48b4-abe9-74011215f5f2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:36:52.863102 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71c47abd-1e83-48b4-abe9-74011215f5f2 None admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:36:52.882085 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application [None req-71c47abd-1e83-48b4-abe9-74011215f5f2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 16:36:52.882085 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 16:36:52.882085 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 16:36:52.882085 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 16:36:52.882085 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 16:36:52.882085 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 16:36:52.882085 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 16:36:52.882085 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 16:36:52.882085 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 16:36:52.882085 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 16:36:52.882085 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 16:36:52.882085 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 16:36:52.882085 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 16:36:52.882085 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 16:36:52.882085 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 16:36:52.883025 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 16:36:52.883025 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 16:36:52.883025 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 16:36:52.883025 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 16:36:52.883025 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 16:36:52.883025 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 16:36:52.883025 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 16:36:52.883025 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 16:36:52.883025 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application with Lock( Aug 01 16:36:52.883025 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 16:36:52.883025 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self._enter() Aug 01 16:36:52.883025 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 16:36:52.883025 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 16:36:52.883025 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 16:36:52.883025 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self.creator() Aug 01 16:36:52.883025 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 16:36:52.883025 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application created_value = creator( Aug 01 16:36:52.883906 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 16:36:52.883906 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 16:36:52.883906 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 16:36:52.883906 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application Aug 01 16:36:52.885757 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-748ebc70-18be-4f1e-a36b-be12c5dcfa81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:36:52.886409 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748ebc70-18be-4f1e-a36b-be12c5dcfa81 None admin] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:36:52.886689 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748ebc70-18be-4f1e-a36b-be12c5dcfa81 None admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:36:52.886964 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748ebc70-18be-4f1e-a36b-be12c5dcfa81 None admin] PATH_INFO: `/v3/projects` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:36:52.887728 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-748ebc70-18be-4f1e-a36b-be12c5dcfa81 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87058) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:36:52.888036 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-748ebc70-18be-4f1e-a36b-be12c5dcfa81 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:36:52.889483 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-748ebc70-18be-4f1e-a36b-be12c5dcfa81 None admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:36:52.892177 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 305/610] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Aug 1 16:36:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 16:36:52.911456 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c47abd-1e83-48b4-abe9-74011215f5f2 None admin] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:36:52.911616 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 306/611] 10.222.0.20 () {56 vars in 1148 bytes} [Mon Aug 1 16:36:52 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 288 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 16:36:52.923241 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-748ebc70-18be-4f1e-a36b-be12c5dcfa81 None admin] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:36:53.006511 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG dbcounter [-] [87057] Writing DB stats keystone:SELECT=100 {{(pid=87057) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:36:53.078528 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG dbcounter [-] [87058] Writing DB stats keystone:SELECT=100 {{(pid=87058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:36:53.120200 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c47abd-1e83-48b4-abe9-74011215f5f2 None admin] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:36:53.141621 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-748ebc70-18be-4f1e-a36b-be12c5dcfa81 None admin] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:36:53.189722 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e71289-78bf-48fc-b1ef-bd2013b4ba9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:36:53.190444 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e71289-78bf-48fc-b1ef-bd2013b4ba9a None admin] REQUEST_METHOD: `PUT` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:36:53.190722 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e71289-78bf-48fc-b1ef-bd2013b4ba9a None admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:36:53.191002 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e71289-78bf-48fc-b1ef-bd2013b4ba9a None admin] PATH_INFO: `/v3/projects/7b992d6a65df4bf9b27c13192d57c5f0/users/3d9bd8d649154c979be3744bfd016f95/roles/51bb93e277c446b49bb80a32afcd3af7` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:36:53.195768 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eaeb07e-3d8e-4e58-9292-75811beab164 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:36:53.196395 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eaeb07e-3d8e-4e58-9292-75811beab164 None admin] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:36:53.196675 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eaeb07e-3d8e-4e58-9292-75811beab164 None admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:36:53.196947 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eaeb07e-3d8e-4e58-9292-75811beab164 None admin] PATH_INFO: `/v3/domains` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:36:53.197652 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eaeb07e-3d8e-4e58-9292-75811beab164 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87057) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:36:53.197956 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eaeb07e-3d8e-4e58-9292-75811beab164 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:36:53.199368 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eaeb07e-3d8e-4e58-9292-75811beab164 None admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:36:53.213511 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 306/612] 10.222.0.20 () {56 vars in 1109 bytes} [Mon Aug 1 16:36:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 303 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 16:36:53.219121 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eaeb07e-3d8e-4e58-9292-75811beab164 None admin] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:36:53.227937 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64e71289-78bf-48fc-b1ef-bd2013b4ba9a None admin] RBAC: Authorizing `identity:create_grant(project_id=7b992d6a65df4bf9b27c13192d57c5f0, user_id=3d9bd8d649154c979be3744bfd016f95, role_id=51bb93e277c446b49bb80a32afcd3af7)` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:36:53.229951 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64e71289-78bf-48fc-b1ef-bd2013b4ba9a None admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:36:53.319532 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 307/613] 10.222.0.20 () {58 vars in 1439 bytes} [Mon Aug 1 16:36:52 2022] PUT /identity/v3/projects/7b992d6a65df4bf9b27c13192d57c5f0/users/3d9bd8d649154c979be3744bfd016f95/roles/51bb93e277c446b49bb80a32afcd3af7 => generated 0 bytes in 397 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 16:36:53.353737 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eaeb07e-3d8e-4e58-9292-75811beab164 None admin] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:36:53.356574 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG dbcounter [-] [87057] Writing DB stats keystone:SELECT=100 {{(pid=87057) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:36:53.406169 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b928fb-bde6-40a4-b403-b9fb256467e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:36:53.406838 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b928fb-bde6-40a4-b403-b9fb256467e0 None admin] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:36:53.407117 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b928fb-bde6-40a4-b403-b9fb256467e0 None admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:36:53.407388 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b928fb-bde6-40a4-b403-b9fb256467e0 None admin] PATH_INFO: `/v3/users/nova` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:36:53.421724 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81b928fb-bde6-40a4-b403-b9fb256467e0 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:36:53.423329 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81b928fb-bde6-40a4-b403-b9fb256467e0 None admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:36:53.442947 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application [None req-81b928fb-bde6-40a4-b403-b9fb256467e0 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 01 16:36:53.442947 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 16:36:53.442947 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 16:36:53.442947 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 16:36:53.442947 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 16:36:53.442947 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 16:36:53.442947 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 16:36:53.442947 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 16:36:53.442947 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 16:36:53.442947 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 16:36:53.442947 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 16:36:53.442947 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 16:36:53.442947 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 01 16:36:53.442947 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 01 16:36:53.442947 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 01 16:36:53.442947 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 01 16:36:53.443975 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 16:36:53.443975 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 16:36:53.443975 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 01 16:36:53.443975 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 16:36:53.443975 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 01 16:36:53.443975 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 16:36:53.443975 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 16:36:53.443975 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 16:36:53.443975 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 16:36:53.443975 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 16:36:53.443975 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 16:36:53.443975 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application with Lock( Aug 01 16:36:53.443975 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 16:36:53.443975 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self._enter() Aug 01 16:36:53.443975 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 16:36:53.443975 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 16:36:53.443975 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 16:36:53.443975 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self.creator() Aug 01 16:36:53.445506 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 16:36:53.445506 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application created_value = creator( Aug 01 16:36:53.445506 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 01 16:36:53.445506 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 01 16:36:53.445506 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 01 16:36:53.445506 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 01 16:36:53.445506 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 01 16:36:53.445506 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 01 16:36:53.445506 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 01 16:36:53.445506 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application Aug 01 16:36:53.445506 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 307/614] 10.222.0.20 () {56 vars in 1093 bytes} [Mon Aug 1 16:36:53 2022] GET /identity/v3/users/nova => generated 82 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 16:36:53.449978 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e71289-78bf-48fc-b1ef-bd2013b4ba9a None admin] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:36:53.524492 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa9df3c-9829-4bf9-a9b6-bf82180abbfd None None] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:36:53.524827 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa9df3c-9829-4bf9-a9b6-bf82180abbfd None None] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:36:53.525057 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa9df3c-9829-4bf9-a9b6-bf82180abbfd None None] PATH_INFO: `/` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:36:53.526441 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 308/615] 10.222.0.20 () {54 vars in 926 bytes} [Mon Aug 1 16:36:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 01 16:36:53.532636 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f8c3c5-336b-42a8-931b-ac5ff6097f81 None None] REQUEST_METHOD: `POST` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:36:53.532931 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f8c3c5-336b-42a8-931b-ac5ff6097f81 None None] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:36:53.533207 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f8c3c5-336b-42a8-931b-ac5ff6097f81 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:36:53.557311 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.auth.core [None req-95f8c3c5-336b-42a8-931b-ac5ff6097f81 None None] MFA Rules not processed for user `1007f8e379b74e169b98fb7c3c0ba5b4`. Rule list: `[]` (Enabled: `True`). {{(pid=87057) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:36:53.558658 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.fernet_utils [None req-95f8c3c5-336b-42a8-931b-ac5ff6097f81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87057) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:36:53.573287 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e71289-78bf-48fc-b1ef-bd2013b4ba9a None admin] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:36:53.598648 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG dbcounter [-] [87058] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:36:53.619870 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec7ac729-3581-4f7b-b908-6d05dd74ca70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:36:53.619870 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7ac729-3581-4f7b-b908-6d05dd74ca70 None admin] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:36:53.619870 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7ac729-3581-4f7b-b908-6d05dd74ca70 None admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:36:53.619870 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7ac729-3581-4f7b-b908-6d05dd74ca70 None admin] PATH_INFO: `/v3/users` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:36:53.620230 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec7ac729-3581-4f7b-b908-6d05dd74ca70 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=87058) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:36:53.620230 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec7ac729-3581-4f7b-b908-6d05dd74ca70 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:36:53.620230 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec7ac729-3581-4f7b-b908-6d05dd74ca70 None admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:36:53.641611 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 308/616] 10.222.0.20 () {56 vars in 1133 bytes} [Mon Aug 1 16:36:53 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:36:53.646346 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec7ac729-3581-4f7b-b908-6d05dd74ca70 None admin] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:36:53.661008 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 309/617] 10.222.0.20 () {58 vars in 1023 bytes} [Mon Aug 1 16:36:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 01 16:36:53.690055 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95f8c3c5-336b-42a8-931b-ac5ff6097f81 None None] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:36:53.807457 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec7ac729-3581-4f7b-b908-6d05dd74ca70 None admin] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:36:53.831046 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG dbcounter [-] [87057] Writing DB stats keystone:SELECT=100 {{(pid=87057) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:36:53.842185 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95f8c3c5-336b-42a8-931b-ac5ff6097f81 None None] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:36:53.863488 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f04b7ea-ac86-4b2c-9f86-58d341d7ba5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:36:53.864495 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f04b7ea-ac86-4b2c-9f86-58d341d7ba5f None admin] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:36:53.864999 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f04b7ea-ac86-4b2c-9f86-58d341d7ba5f None admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:36:53.865475 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f04b7ea-ac86-4b2c-9f86-58d341d7ba5f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:36:53.875928 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b58c275-0b8f-4274-8765-532695348c30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:36:53.876497 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b58c275-0b8f-4274-8765-532695348c30 None admin] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:36:53.876772 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b58c275-0b8f-4274-8765-532695348c30 None admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:36:53.877043 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b58c275-0b8f-4274-8765-532695348c30 None admin] PATH_INFO: `/v3/roles/service` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:36:53.888286 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b58c275-0b8f-4274-8765-532695348c30 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 16:36:53.893791 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application [None req-9b58c275-0b8f-4274-8765-532695348c30 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 01 16:36:53.893791 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 16:36:53.893791 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 16:36:53.893791 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 16:36:53.893791 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 16:36:53.893791 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 16:36:53.893791 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 16:36:53.893791 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 16:36:53.893791 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 16:36:53.893791 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 16:36:53.893791 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 16:36:53.893791 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 16:36:53.893791 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 16:36:53.893791 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 16:36:53.893791 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 16:36:53.893791 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 16:36:53.894818 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 16:36:53.894818 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 16:36:53.894818 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 16:36:53.894818 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 16:36:53.894818 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 16:36:53.894818 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 16:36:53.894818 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 16:36:53.894818 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 16:36:53.894818 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 16:36:53.894818 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 16:36:53.894818 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 16:36:53.894818 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application with Lock( Aug 01 16:36:53.894818 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 16:36:53.894818 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self._enter() Aug 01 16:36:53.894818 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 16:36:53.894818 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 16:36:53.895671 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 16:36:53.895671 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self.creator() Aug 01 16:36:53.895671 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 16:36:53.895671 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application created_value = creator( Aug 01 16:36:53.895671 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 16:36:53.895671 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 16:36:53.895671 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 16:36:53.895671 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 16:36:53.895671 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 16:36:53.895671 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 16:36:53.895671 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 01 16:36:53.895671 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application Aug 01 16:36:53.895671 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 310/618] 10.222.0.20 () {56 vars in 1102 bytes} [Mon Aug 1 16:36:53 2022] GET /identity/v3/roles/service => generated 85 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 01 16:36:53.915274 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b58c275-0b8f-4274-8765-532695348c30 None admin] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:36:53.950946 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG dbcounter [-] [87058] Writing DB stats keystone:SELECT=100 {{(pid=87058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:36:53.973688 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f04b7ea-ac86-4b2c-9f86-58d341d7ba5f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:36:53.976091 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f04b7ea-ac86-4b2c-9f86-58d341d7ba5f None admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:36:54.099730 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b58c275-0b8f-4274-8765-532695348c30 None admin] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:36:54.145999 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-593b67c9-281e-44fd-866d-64af25ed84e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:36:54.147906 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593b67c9-281e-44fd-866d-64af25ed84e0 None admin] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:36:54.148183 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593b67c9-281e-44fd-866d-64af25ed84e0 None admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:36:54.148459 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593b67c9-281e-44fd-866d-64af25ed84e0 None admin] PATH_INFO: `/v3/roles` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:36:54.149190 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-593b67c9-281e-44fd-866d-64af25ed84e0 None admin] RBAC: Adding query filter params (name=service) {{(pid=87057) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:36:54.149491 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-593b67c9-281e-44fd-866d-64af25ed84e0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:36:54.153525 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-593b67c9-281e-44fd-866d-64af25ed84e0 None admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:36:54.157684 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 309/619] 10.222.0.20 () {58 vars in 1282 bytes} [Mon Aug 1 16:36:53 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 512 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 01 16:36:54.166223 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f04b7ea-ac86-4b2c-9f86-58d341d7ba5f None admin] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:36:54.176874 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 311/620] 10.222.0.20 () {56 vars in 1103 bytes} [Mon Aug 1 16:36:53 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:36:54.182907 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-593b67c9-281e-44fd-866d-64af25ed84e0 None admin] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:36:54.208747 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG dbcounter [-] [87057] Writing DB stats keystone:SELECT=100 {{(pid=87057) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:36:54.269801 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG dbcounter [-] [87058] Writing DB stats keystone:SELECT=100 {{(pid=87058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:36:54.290183 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f04b7ea-ac86-4b2c-9f86-58d341d7ba5f None admin] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:36:54.301319 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-593b67c9-281e-44fd-866d-64af25ed84e0 None admin] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:36:54.326236 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-421d59c9-319a-4744-8d42-cc368e4cccff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:36:54.326889 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421d59c9-319a-4744-8d42-cc368e4cccff None admin] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:36:54.327165 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421d59c9-319a-4744-8d42-cc368e4cccff None admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:36:54.327441 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421d59c9-319a-4744-8d42-cc368e4cccff None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:36:54.338116 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-421d59c9-319a-4744-8d42-cc368e4cccff None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:36:54.339855 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-421d59c9-319a-4744-8d42-cc368e4cccff None admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:36:54.341352 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed55faac-4fdb-436d-81cd-4e9d8ff4db52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:36:54.342341 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed55faac-4fdb-436d-81cd-4e9d8ff4db52 None admin] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:36:54.342822 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed55faac-4fdb-436d-81cd-4e9d8ff4db52 None admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:36:54.343263 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed55faac-4fdb-436d-81cd-4e9d8ff4db52 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:36:54.355297 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed55faac-4fdb-436d-81cd-4e9d8ff4db52 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:36:54.356356 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application [None req-421d59c9-319a-4744-8d42-cc368e4cccff None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 16:36:54.356356 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 16:36:54.356356 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 16:36:54.356356 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 16:36:54.356356 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 16:36:54.356356 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 16:36:54.356356 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 16:36:54.356356 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 16:36:54.356356 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 16:36:54.356356 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 16:36:54.356356 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 16:36:54.356356 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 16:36:54.356356 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 16:36:54.356356 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 16:36:54.356356 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 16:36:54.357434 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 16:36:54.357434 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 16:36:54.357434 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 16:36:54.357434 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 16:36:54.357434 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 16:36:54.357434 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 16:36:54.357434 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 16:36:54.357434 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 16:36:54.357434 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application with Lock( Aug 01 16:36:54.357434 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 16:36:54.357434 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self._enter() Aug 01 16:36:54.357434 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 16:36:54.357434 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 16:36:54.357434 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 16:36:54.357434 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self.creator() Aug 01 16:36:54.357434 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 16:36:54.357434 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application created_value = creator( Aug 01 16:36:54.358580 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 16:36:54.358580 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 16:36:54.358580 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 16:36:54.358580 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application Aug 01 16:36:54.359539 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed55faac-4fdb-436d-81cd-4e9d8ff4db52 None admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:36:54.362103 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 310/621] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Aug 1 16:36:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 16:36:54.363706 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63ac310-c802-44ab-9813-46708a52d1ad None None] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:36:54.363990 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63ac310-c802-44ab-9813-46708a52d1ad None None] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:36:54.364263 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63ac310-c802-44ab-9813-46708a52d1ad None None] PATH_INFO: `/` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:36:54.365753 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 311/622] 10.222.0.20 () {54 vars in 926 bytes} [Mon Aug 1 16:36:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 16:36:54.367088 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f63ac310-c802-44ab-9813-46708a52d1ad None None] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:36:54.382662 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application [None req-ed55faac-4fdb-436d-81cd-4e9d8ff4db52 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 16:36:54.382662 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 16:36:54.382662 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 16:36:54.382662 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 16:36:54.382662 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 16:36:54.382662 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 16:36:54.382662 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 16:36:54.382662 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 16:36:54.382662 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 16:36:54.382662 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 16:36:54.382662 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 16:36:54.382662 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 16:36:54.382662 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 16:36:54.382662 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 16:36:54.382662 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 16:36:54.383588 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 16:36:54.383588 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 16:36:54.383588 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 16:36:54.383588 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 16:36:54.383588 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 16:36:54.383588 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 16:36:54.383588 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 16:36:54.383588 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 16:36:54.383588 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application with Lock( Aug 01 16:36:54.383588 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 16:36:54.383588 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self._enter() Aug 01 16:36:54.383588 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 16:36:54.383588 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 16:36:54.383588 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 16:36:54.383588 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self.creator() Aug 01 16:36:54.383588 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 16:36:54.383588 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application created_value = creator( Aug 01 16:36:54.385114 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 16:36:54.385114 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 16:36:54.385114 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 16:36:54.385114 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application Aug 01 16:36:54.385114 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 312/623] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Aug 1 16:36:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 01 16:36:54.385401 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939d50ab-3d29-4ba4-9c56-49057df447fb None None] REQUEST_METHOD: `POST` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:36:54.385914 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939d50ab-3d29-4ba4-9c56-49057df447fb None None] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:36:54.386245 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939d50ab-3d29-4ba4-9c56-49057df447fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:36:54.406765 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.auth.core [None req-939d50ab-3d29-4ba4-9c56-49057df447fb None None] MFA Rules not processed for user `1007f8e379b74e169b98fb7c3c0ba5b4`. Rule list: `[]` (Enabled: `True`). {{(pid=87057) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:36:54.407339 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.fernet_utils [None req-939d50ab-3d29-4ba4-9c56-49057df447fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87057) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:36:54.475626 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG dbcounter [-] [87057] Writing DB stats keystone:SELECT=100 {{(pid=87057) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:36:54.486094 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 313/624] 10.222.0.20 () {58 vars in 1023 bytes} [Mon Aug 1 16:36:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 16:36:54.486094 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-939d50ab-3d29-4ba4-9c56-49057df447fb None None] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:36:54.504800 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f63ac310-c802-44ab-9813-46708a52d1ad None None] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:36:54.552457 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG dbcounter [-] [87058] Writing DB stats keystone:SELECT=100 {{(pid=87058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:36:54.559269 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-314af992-f0b8-40f7-be50-ff4be1858b35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:36:54.560298 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314af992-f0b8-40f7-be50-ff4be1858b35 None admin] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:36:54.560794 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314af992-f0b8-40f7-be50-ff4be1858b35 None admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:36:54.561230 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314af992-f0b8-40f7-be50-ff4be1858b35 None admin] PATH_INFO: `/v3/domains` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:36:54.562119 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-314af992-f0b8-40f7-be50-ff4be1858b35 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87058) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:36:54.562587 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-314af992-f0b8-40f7-be50-ff4be1858b35 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:36:54.567241 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-314af992-f0b8-40f7-be50-ff4be1858b35 None admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:36:54.583840 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 312/625] 10.222.0.20 () {56 vars in 1109 bytes} [Mon Aug 1 16:36:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:36:54.585934 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-314af992-f0b8-40f7-be50-ff4be1858b35 None admin] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:36:54.592610 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-939d50ab-3d29-4ba4-9c56-49057df447fb None None] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:36:54.702875 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-314af992-f0b8-40f7-be50-ff4be1858b35 None admin] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:36:54.742928 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c12e4c1c-5afc-4a52-a09a-47643b99e339 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:36:54.744052 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12e4c1c-5afc-4a52-a09a-47643b99e339 None admin] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:36:54.744552 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12e4c1c-5afc-4a52-a09a-47643b99e339 None admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:36:54.745022 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12e4c1c-5afc-4a52-a09a-47643b99e339 None admin] PATH_INFO: `/v3/roles/service` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:36:54.751785 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab517a7-4a4e-4f69-812f-04b8b97bb779 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:36:54.752413 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab517a7-4a4e-4f69-812f-04b8b97bb779 None admin] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:36:54.752695 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab517a7-4a4e-4f69-812f-04b8b97bb779 None admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:36:54.752966 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab517a7-4a4e-4f69-812f-04b8b97bb779 None admin] PATH_INFO: `/v3/domains` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:36:54.753670 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ab517a7-4a4e-4f69-812f-04b8b97bb779 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87057) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:36:54.753970 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ab517a7-4a4e-4f69-812f-04b8b97bb779 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:36:54.755743 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ab517a7-4a4e-4f69-812f-04b8b97bb779 None admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:36:54.759699 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c12e4c1c-5afc-4a52-a09a-47643b99e339 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 16:36:54.765073 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application [None req-c12e4c1c-5afc-4a52-a09a-47643b99e339 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 01 16:36:54.765073 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 16:36:54.765073 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 16:36:54.765073 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 16:36:54.765073 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 16:36:54.765073 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 16:36:54.765073 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 16:36:54.765073 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 16:36:54.765073 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 16:36:54.765073 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 16:36:54.765073 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 16:36:54.765073 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 16:36:54.765073 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 16:36:54.765073 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 16:36:54.765073 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 16:36:54.765073 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 16:36:54.766074 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 16:36:54.766074 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 16:36:54.766074 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 16:36:54.766074 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 16:36:54.766074 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 16:36:54.766074 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 16:36:54.766074 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 16:36:54.766074 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 16:36:54.766074 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 16:36:54.766074 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 16:36:54.766074 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 16:36:54.766074 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application with Lock( Aug 01 16:36:54.766074 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 16:36:54.766074 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self._enter() Aug 01 16:36:54.766074 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 16:36:54.766074 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 16:36:54.767212 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 16:36:54.767212 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self.creator() Aug 01 16:36:54.767212 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 16:36:54.767212 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application created_value = creator( Aug 01 16:36:54.767212 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 16:36:54.767212 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 16:36:54.767212 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 16:36:54.767212 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 16:36:54.767212 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 16:36:54.767212 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 16:36:54.767212 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 01 16:36:54.767212 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application Aug 01 16:36:54.768946 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 314/626] 10.222.0.20 () {56 vars in 1109 bytes} [Mon Aug 1 16:36:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 287 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:36:54.769807 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab517a7-4a4e-4f69-812f-04b8b97bb779 None admin] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:36:54.770242 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 313/627] 10.222.0.20 () {56 vars in 1102 bytes} [Mon Aug 1 16:36:54 2022] GET /identity/v3/roles/service => generated 85 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 16:36:54.776763 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c12e4c1c-5afc-4a52-a09a-47643b99e339 None admin] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:36:54.833222 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG dbcounter [-] [87057] Writing DB stats keystone:SELECT=100 {{(pid=87057) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:36:54.836133 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG dbcounter [-] [87058] Writing DB stats keystone:SELECT=100 {{(pid=87058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:36:54.878699 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab517a7-4a4e-4f69-812f-04b8b97bb779 None admin] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:36:54.885730 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c12e4c1c-5afc-4a52-a09a-47643b99e339 None admin] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:36:54.918679 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b8feeb7-6b07-48c7-8182-b0bf08573463 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:36:54.919331 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8feeb7-6b07-48c7-8182-b0bf08573463 None admin] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:36:54.919610 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8feeb7-6b07-48c7-8182-b0bf08573463 None admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:36:54.919879 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8feeb7-6b07-48c7-8182-b0bf08573463 None admin] PATH_INFO: `/v3/projects/service` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:36:54.925304 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a770edc-3857-403a-b844-5759ed7d4a4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:36:54.925945 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a770edc-3857-403a-b844-5759ed7d4a4d None admin] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:36:54.926245 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a770edc-3857-403a-b844-5759ed7d4a4d None admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:36:54.926519 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a770edc-3857-403a-b844-5759ed7d4a4d None admin] PATH_INFO: `/v3/users/swift` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:36:54.926803 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b8feeb7-6b07-48c7-8182-b0bf08573463 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:36:54.928312 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b8feeb7-6b07-48c7-8182-b0bf08573463 None admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:36:54.934317 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a770edc-3857-403a-b844-5759ed7d4a4d None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:36:54.935746 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a770edc-3857-403a-b844-5759ed7d4a4d None admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:36:54.939548 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application [None req-8b8feeb7-6b07-48c7-8182-b0bf08573463 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 16:36:54.939548 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 16:36:54.939548 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 16:36:54.939548 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 16:36:54.939548 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 16:36:54.939548 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 16:36:54.939548 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 16:36:54.939548 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 16:36:54.939548 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 16:36:54.939548 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 16:36:54.939548 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 16:36:54.939548 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 16:36:54.939548 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 01 16:36:54.939548 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 01 16:36:54.939548 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 01 16:36:54.940498 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 01 16:36:54.940498 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 16:36:54.940498 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 16:36:54.940498 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 16:36:54.940498 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 16:36:54.940498 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 16:36:54.940498 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 16:36:54.940498 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 16:36:54.940498 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application with Lock( Aug 01 16:36:54.940498 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 16:36:54.940498 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self._enter() Aug 01 16:36:54.940498 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 16:36:54.940498 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 16:36:54.940498 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 16:36:54.940498 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self.creator() Aug 01 16:36:54.940498 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 16:36:54.940498 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application created_value = creator( Aug 01 16:36:54.941395 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 01 16:36:54.941395 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 01 16:36:54.941395 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 01 16:36:54.941395 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 01 16:36:54.941395 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 01 16:36:54.941395 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 01 16:36:54.941395 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 16:36:54.941395 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application Aug 01 16:36:54.941395 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 315/628] 10.222.0.20 () {56 vars in 1111 bytes} [Mon Aug 1 16:36:54 2022] GET /identity/v3/projects/service => generated 88 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 16:36:54.941896 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b8feeb7-6b07-48c7-8182-b0bf08573463 None admin] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:36:54.951062 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application [None req-8a770edc-3857-403a-b844-5759ed7d4a4d None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 01 16:36:54.951062 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 16:36:54.951062 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 16:36:54.951062 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 16:36:54.951062 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 16:36:54.951062 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 16:36:54.951062 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 16:36:54.951062 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 16:36:54.951062 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 16:36:54.951062 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 16:36:54.951062 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 16:36:54.951062 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 16:36:54.951062 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 01 16:36:54.951062 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 01 16:36:54.951062 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 01 16:36:54.951062 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 01 16:36:54.952034 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 16:36:54.952034 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 16:36:54.952034 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 01 16:36:54.952034 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 16:36:54.952034 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 01 16:36:54.952034 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 16:36:54.952034 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 16:36:54.952034 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 16:36:54.952034 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 16:36:54.952034 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 16:36:54.952034 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 16:36:54.952034 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application with Lock( Aug 01 16:36:54.952034 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 16:36:54.952034 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self._enter() Aug 01 16:36:54.952034 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 16:36:54.952034 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 16:36:54.952034 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 16:36:54.952034 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self.creator() Aug 01 16:36:54.953423 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 16:36:54.953423 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application created_value = creator( Aug 01 16:36:54.953423 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 01 16:36:54.953423 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 01 16:36:54.953423 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 01 16:36:54.953423 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 01 16:36:54.953423 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 01 16:36:54.953423 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 01 16:36:54.953423 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 01 16:36:54.953423 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application Aug 01 16:36:54.953423 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 314/629] 10.222.0.20 () {56 vars in 1096 bytes} [Mon Aug 1 16:36:54 2022] GET /identity/v3/users/swift => generated 83 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 16:36:54.954159 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89966b74-b37a-49e9-8b09-366a44c42ef1 None None] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:36:54.954159 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89966b74-b37a-49e9-8b09-366a44c42ef1 None None] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:36:54.954318 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89966b74-b37a-49e9-8b09-366a44c42ef1 None None] PATH_INFO: `/` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:36:54.955585 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 315/630] 10.222.0.20 () {54 vars in 926 bytes} [Mon Aug 1 16:36:54 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 16:36:54.956124 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89966b74-b37a-49e9-8b09-366a44c42ef1 None None] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:36:55.059334 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b8feeb7-6b07-48c7-8182-b0bf08573463 None admin] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:36:55.060966 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89966b74-b37a-49e9-8b09-366a44c42ef1 None None] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:36:55.075145 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG dbcounter [-] [87058] Writing DB stats keystone:SELECT=100 {{(pid=87058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:36:55.079418 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG dbcounter [-] [87057] Writing DB stats keystone:SELECT=100 {{(pid=87057) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:36:55.096733 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-452037ff-abdf-44bc-9e6b-238e4d3b76b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:36:55.096907 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc70f59-4092-42f7-bc64-87e3b36301da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:36:55.097351 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc70f59-4092-42f7-bc64-87e3b36301da None admin] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:36:55.097475 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452037ff-abdf-44bc-9e6b-238e4d3b76b7 None admin] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:36:55.097657 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452037ff-abdf-44bc-9e6b-238e4d3b76b7 None admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:36:55.097759 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc70f59-4092-42f7-bc64-87e3b36301da None admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:36:55.097906 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc70f59-4092-42f7-bc64-87e3b36301da None admin] PATH_INFO: `/v3/roles` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:36:55.097996 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452037ff-abdf-44bc-9e6b-238e4d3b76b7 None admin] PATH_INFO: `/v3/projects` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:36:55.098730 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-452037ff-abdf-44bc-9e6b-238e4d3b76b7 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87058) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:36:55.098830 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cc70f59-4092-42f7-bc64-87e3b36301da None admin] RBAC: Adding query filter params (name=service) {{(pid=87057) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:36:55.099029 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cc70f59-4092-42f7-bc64-87e3b36301da None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:36:55.099123 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-452037ff-abdf-44bc-9e6b-238e4d3b76b7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:36:55.100422 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cc70f59-4092-42f7-bc64-87e3b36301da None admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:36:55.100531 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-452037ff-abdf-44bc-9e6b-238e4d3b76b7 None admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:36:55.112249 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 316/631] 10.222.0.20 () {56 vars in 1148 bytes} [Mon Aug 1 16:36:54 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:36:55.117613 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-452037ff-abdf-44bc-9e6b-238e4d3b76b7 None admin] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:36:55.117763 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 316/632] 10.222.0.20 () {56 vars in 1103 bytes} [Mon Aug 1 16:36:54 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:36:55.117763 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57479d3-ce77-4514-87bd-88a8989ead6c None None] REQUEST_METHOD: `POST` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:36:55.117763 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57479d3-ce77-4514-87bd-88a8989ead6c None None] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:36:55.117763 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57479d3-ce77-4514-87bd-88a8989ead6c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:36:55.135927 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.auth.core [None req-d57479d3-ce77-4514-87bd-88a8989ead6c None None] MFA Rules not processed for user `1007f8e379b74e169b98fb7c3c0ba5b4`. Rule list: `[]` (Enabled: `True`). {{(pid=87057) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:36:55.137357 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.fernet_utils [None req-d57479d3-ce77-4514-87bd-88a8989ead6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87057) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:36:55.222131 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 317/633] 10.222.0.20 () {58 vars in 1023 bytes} [Mon Aug 1 16:36:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 16:36:55.222131 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d57479d3-ce77-4514-87bd-88a8989ead6c None None] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:36:55.222402 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-452037ff-abdf-44bc-9e6b-238e4d3b76b7 None admin] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:36:55.260120 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef5403d-0451-4021-be26-6529feac843f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:36:55.261011 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef5403d-0451-4021-be26-6529feac843f None admin] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:36:55.261293 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef5403d-0451-4021-be26-6529feac843f None admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:36:55.261565 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef5403d-0451-4021-be26-6529feac843f None admin] PATH_INFO: `/v3/users` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:36:55.262401 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ef5403d-0451-4021-be26-6529feac843f None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=87058) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:36:55.262710 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ef5403d-0451-4021-be26-6529feac843f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:36:55.264133 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ef5403d-0451-4021-be26-6529feac843f None admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:36:55.288248 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 317/634] 10.222.0.20 () {56 vars in 1135 bytes} [Mon Aug 1 16:36:55 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:36:55.289006 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef5403d-0451-4021-be26-6529feac843f None admin] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:36:55.320533 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG dbcounter [-] [87058] Writing DB stats keystone:SELECT=100 {{(pid=87058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:36:55.328692 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d57479d3-ce77-4514-87bd-88a8989ead6c None None] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:36:55.339498 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG dbcounter [-] [87057] Writing DB stats keystone:SELECT=100 {{(pid=87057) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:36:55.363718 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b64a24-dc7d-47b0-a929-b969693a9a6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:36:55.364364 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b64a24-dc7d-47b0-a929-b969693a9a6a None admin] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:36:55.364641 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b64a24-dc7d-47b0-a929-b969693a9a6a None admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:36:55.364915 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b64a24-dc7d-47b0-a929-b969693a9a6a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:36:55.376191 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40b64a24-dc7d-47b0-a929-b969693a9a6a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:36:55.377744 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40b64a24-dc7d-47b0-a929-b969693a9a6a None admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:36:55.392451 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application [None req-40b64a24-dc7d-47b0-a929-b969693a9a6a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 16:36:55.392451 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 16:36:55.392451 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 16:36:55.392451 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 16:36:55.392451 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 16:36:55.392451 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 16:36:55.392451 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 16:36:55.392451 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 16:36:55.392451 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 16:36:55.392451 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 16:36:55.392451 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 16:36:55.392451 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 16:36:55.392451 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 16:36:55.392451 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 16:36:55.392451 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 16:36:55.393357 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 16:36:55.393357 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 16:36:55.393357 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 16:36:55.393357 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 16:36:55.393357 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 16:36:55.393357 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 16:36:55.393357 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 16:36:55.393357 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 16:36:55.393357 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application with Lock( Aug 01 16:36:55.393357 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 16:36:55.393357 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self._enter() Aug 01 16:36:55.393357 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 16:36:55.393357 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 16:36:55.393357 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 16:36:55.393357 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self.creator() Aug 01 16:36:55.393357 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 16:36:55.393357 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application created_value = creator( Aug 01 16:36:55.394310 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 16:36:55.394310 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 16:36:55.394310 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 16:36:55.394310 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application Aug 01 16:36:55.394310 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 318/635] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Aug 1 16:36:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 16:36:55.394614 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b64a24-dc7d-47b0-a929-b969693a9a6a None admin] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:36:55.400857 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef5403d-0451-4021-be26-6529feac843f None admin] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:36:55.437292 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f51c87-574f-45e6-800e-1d5603963829 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:36:55.437912 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f51c87-574f-45e6-800e-1d5603963829 None admin] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:36:55.438224 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f51c87-574f-45e6-800e-1d5603963829 None admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:36:55.438499 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f51c87-574f-45e6-800e-1d5603963829 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:36:55.439301 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74f51c87-574f-45e6-800e-1d5603963829 None admin] RBAC: Adding query filter params (role.id=51bb93e277c446b49bb80a32afcd3af7, scope.project.id=7b992d6a65df4bf9b27c13192d57c5f0, user.id=6533106d03eb4ca185a8839336d2aff9) {{(pid=87058) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:36:55.439613 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74f51c87-574f-45e6-800e-1d5603963829 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:36:55.441060 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74f51c87-574f-45e6-800e-1d5603963829 None admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:36:55.447620 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 318/636] 10.222.0.20 () {56 vars in 1374 bytes} [Mon Aug 1 16:36:55 2022] GET /identity/v3/role_assignments?user.id=6533106d03eb4ca185a8839336d2aff9&scope.project.id=7b992d6a65df4bf9b27c13192d57c5f0&role.id=51bb93e277c446b49bb80a32afcd3af7 => generated 608 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:36:55.448402 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f51c87-574f-45e6-800e-1d5603963829 None admin] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:36:55.535359 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b64a24-dc7d-47b0-a929-b969693a9a6a None admin] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:36:55.563885 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG dbcounter [-] [87058] Writing DB stats keystone:SELECT=100 {{(pid=87058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:36:55.571304 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f51c87-574f-45e6-800e-1d5603963829 None admin] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:36:55.590243 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f4a2399-e15f-4d2b-9b17-1f7ab517e70e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:36:55.591759 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4a2399-e15f-4d2b-9b17-1f7ab517e70e None admin] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:36:55.592229 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4a2399-e15f-4d2b-9b17-1f7ab517e70e None admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:36:55.592717 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4a2399-e15f-4d2b-9b17-1f7ab517e70e None admin] PATH_INFO: `/v3/roles/service` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:36:55.610529 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f4a2399-e15f-4d2b-9b17-1f7ab517e70e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 16:36:55.610799 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4767ee15-7449-4f18-8168-6e93e2b55f40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:36:55.611458 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4767ee15-7449-4f18-8168-6e93e2b55f40 None admin] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:36:55.611741 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4767ee15-7449-4f18-8168-6e93e2b55f40 None admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:36:55.612016 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4767ee15-7449-4f18-8168-6e93e2b55f40 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:36:55.618782 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application [None req-6f4a2399-e15f-4d2b-9b17-1f7ab517e70e None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 01 16:36:55.618782 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 16:36:55.618782 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 16:36:55.618782 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 16:36:55.618782 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 16:36:55.618782 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 16:36:55.618782 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 16:36:55.618782 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 16:36:55.618782 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 16:36:55.618782 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 16:36:55.618782 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 16:36:55.618782 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 16:36:55.618782 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 16:36:55.618782 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 16:36:55.618782 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 16:36:55.618782 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 16:36:55.619789 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 16:36:55.619789 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 16:36:55.619789 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 16:36:55.619789 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 16:36:55.619789 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 16:36:55.619789 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 16:36:55.619789 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 16:36:55.619789 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 16:36:55.619789 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 16:36:55.619789 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 16:36:55.619789 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 16:36:55.619789 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application with Lock( Aug 01 16:36:55.619789 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 16:36:55.619789 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self._enter() Aug 01 16:36:55.619789 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 16:36:55.619789 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 16:36:55.620633 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 16:36:55.620633 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self.creator() Aug 01 16:36:55.620633 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 16:36:55.620633 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application created_value = creator( Aug 01 16:36:55.620633 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 16:36:55.620633 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 16:36:55.620633 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 16:36:55.620633 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 16:36:55.620633 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 16:36:55.620633 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 16:36:55.620633 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 01 16:36:55.620633 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application Aug 01 16:36:55.624805 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 319/637] 10.222.0.20 () {56 vars in 1102 bytes} [Mon Aug 1 16:36:55 2022] GET /identity/v3/roles/service => generated 85 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 16:36:55.626223 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f4a2399-e15f-4d2b-9b17-1f7ab517e70e None admin] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:36:55.626428 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4767ee15-7449-4f18-8168-6e93e2b55f40 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:36:55.628083 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4767ee15-7449-4f18-8168-6e93e2b55f40 None admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:36:55.645731 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application [None req-4767ee15-7449-4f18-8168-6e93e2b55f40 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 16:36:55.645731 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 16:36:55.645731 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 16:36:55.645731 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 16:36:55.645731 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 16:36:55.645731 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 16:36:55.645731 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 16:36:55.645731 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 16:36:55.645731 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 16:36:55.645731 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 16:36:55.645731 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 16:36:55.645731 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 16:36:55.645731 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 16:36:55.645731 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 16:36:55.645731 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 16:36:55.647416 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 16:36:55.647416 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 16:36:55.647416 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 16:36:55.647416 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 16:36:55.647416 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 16:36:55.647416 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 16:36:55.647416 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 16:36:55.647416 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 16:36:55.647416 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application with Lock( Aug 01 16:36:55.647416 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 16:36:55.647416 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self._enter() Aug 01 16:36:55.647416 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 16:36:55.647416 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 16:36:55.647416 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 16:36:55.647416 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self.creator() Aug 01 16:36:55.647416 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 16:36:55.647416 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application created_value = creator( Aug 01 16:36:55.648774 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 16:36:55.648774 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 16:36:55.648774 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 16:36:55.648774 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application Aug 01 16:36:55.648774 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 319/638] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Aug 1 16:36:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 16:36:55.648774 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4767ee15-7449-4f18-8168-6e93e2b55f40 None admin] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:36:55.661777 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG dbcounter [-] [87057] Writing DB stats keystone:SELECT=100 {{(pid=87057) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:36:55.754330 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f4a2399-e15f-4d2b-9b17-1f7ab517e70e None admin] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:36:55.767603 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4767ee15-7449-4f18-8168-6e93e2b55f40 None admin] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:36:55.797410 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec4361e-e87a-4ca5-8e11-f564d10772c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:36:55.797960 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec4361e-e87a-4ca5-8e11-f564d10772c9 None admin] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:36:55.798436 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec4361e-e87a-4ca5-8e11-f564d10772c9 None admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:36:55.798708 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec4361e-e87a-4ca5-8e11-f564d10772c9 None admin] PATH_INFO: `/v3/domains` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:36:55.799529 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aec4361e-e87a-4ca5-8e11-f564d10772c9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87057) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:36:55.799933 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aec4361e-e87a-4ca5-8e11-f564d10772c9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:36:55.801363 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aec4361e-e87a-4ca5-8e11-f564d10772c9 None admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:36:55.817491 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33210d9c-9e1e-4146-9452-b87988c556de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:36:55.818126 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33210d9c-9e1e-4146-9452-b87988c556de None admin] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:36:55.818486 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33210d9c-9e1e-4146-9452-b87988c556de None admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:36:55.819416 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33210d9c-9e1e-4146-9452-b87988c556de None admin] PATH_INFO: `/v3/roles` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:36:55.820163 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33210d9c-9e1e-4146-9452-b87988c556de None admin] RBAC: Adding query filter params (name=service) {{(pid=87058) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:36:55.820701 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33210d9c-9e1e-4146-9452-b87988c556de None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:36:55.822077 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33210d9c-9e1e-4146-9452-b87988c556de None admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:36:55.825316 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 320/639] 10.222.0.20 () {56 vars in 1109 bytes} [Mon Aug 1 16:36:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:36:55.825966 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec4361e-e87a-4ca5-8e11-f564d10772c9 None admin] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:36:55.840087 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 320/640] 10.222.0.20 () {56 vars in 1103 bytes} [Mon Aug 1 16:36:55 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:36:55.841830 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b7ef8b-ba43-44a8-bccb-ba6501cb635a None None] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:36:55.842136 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b7ef8b-ba43-44a8-bccb-ba6501cb635a None None] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:36:55.842402 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b7ef8b-ba43-44a8-bccb-ba6501cb635a None None] PATH_INFO: `/` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:36:55.843662 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 321/641] 10.222.0.20 () {54 vars in 926 bytes} [Mon Aug 1 16:36:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 16:36:55.844349 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b7ef8b-ba43-44a8-bccb-ba6501cb635a None None] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:36:55.870419 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG dbcounter [-] [87058] Writing DB stats keystone:SELECT=100 {{(pid=87058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:36:55.932704 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG dbcounter [-] [87057] Writing DB stats keystone:SELECT=100 {{(pid=87057) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:36:55.951697 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec4361e-e87a-4ca5-8e11-f564d10772c9 None admin] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:36:55.992077 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e1582fc-ed93-407e-9e69-a5868331bc1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:36:55.992978 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1582fc-ed93-407e-9e69-a5868331bc1b None admin] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:36:55.993260 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1582fc-ed93-407e-9e69-a5868331bc1b None admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:36:55.993534 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1582fc-ed93-407e-9e69-a5868331bc1b None admin] PATH_INFO: `/v3/domains` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:36:55.994257 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e1582fc-ed93-407e-9e69-a5868331bc1b None admin] RBAC: Adding query filter params (name=Default) {{(pid=87057) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:36:55.994560 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e1582fc-ed93-407e-9e69-a5868331bc1b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:36:55.995270 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b7ef8b-ba43-44a8-bccb-ba6501cb635a None None] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:36:55.995943 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e1582fc-ed93-407e-9e69-a5868331bc1b None admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:36:56.014995 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 321/642] 10.222.0.20 () {56 vars in 1109 bytes} [Mon Aug 1 16:36:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:36:56.015913 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e1582fc-ed93-407e-9e69-a5868331bc1b None admin] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:36:56.042099 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c08324-bd07-4eb7-af92-da394ab78e2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:36:56.042467 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c08324-bd07-4eb7-af92-da394ab78e2a None admin] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:36:56.042747 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c08324-bd07-4eb7-af92-da394ab78e2a None admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:36:56.043018 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c08324-bd07-4eb7-af92-da394ab78e2a None admin] PATH_INFO: `/v3/users/glance` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:36:56.052313 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17c08324-bd07-4eb7-af92-da394ab78e2a None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:36:56.054276 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17c08324-bd07-4eb7-af92-da394ab78e2a None admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:36:56.065632 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application [None req-17c08324-bd07-4eb7-af92-da394ab78e2a None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Aug 01 16:36:56.065632 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 16:36:56.065632 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 16:36:56.065632 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 16:36:56.065632 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 16:36:56.065632 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 16:36:56.065632 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 16:36:56.065632 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 16:36:56.065632 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 16:36:56.065632 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 16:36:56.065632 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 16:36:56.065632 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 16:36:56.065632 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 01 16:36:56.065632 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 01 16:36:56.065632 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 01 16:36:56.065632 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 01 16:36:56.066756 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 16:36:56.066756 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 16:36:56.066756 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 01 16:36:56.066756 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 16:36:56.066756 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 01 16:36:56.066756 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 16:36:56.066756 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 16:36:56.066756 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 16:36:56.066756 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 16:36:56.066756 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 16:36:56.066756 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 16:36:56.066756 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application with Lock( Aug 01 16:36:56.066756 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 16:36:56.066756 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self._enter() Aug 01 16:36:56.066756 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 16:36:56.066756 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 16:36:56.066756 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 16:36:56.066756 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self.creator() Aug 01 16:36:56.067776 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 16:36:56.067776 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application created_value = creator( Aug 01 16:36:56.067776 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 01 16:36:56.067776 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 01 16:36:56.067776 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 01 16:36:56.067776 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 01 16:36:56.067776 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 01 16:36:56.067776 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 01 16:36:56.067776 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Aug 01 16:36:56.067776 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application Aug 01 16:36:56.067776 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 322/643] 10.222.0.20 () {56 vars in 1099 bytes} [Mon Aug 1 16:36:55 2022] GET /identity/v3/users/glance => generated 84 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 16:36:56.068398 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe099a8-5f9f-48f8-a5de-7a6386ab3461 None None] REQUEST_METHOD: `POST` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:36:56.068398 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe099a8-5f9f-48f8-a5de-7a6386ab3461 None None] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:36:56.069597 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe099a8-5f9f-48f8-a5de-7a6386ab3461 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:36:56.087593 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.auth.core [None req-2fe099a8-5f9f-48f8-a5de-7a6386ab3461 None None] MFA Rules not processed for user `1007f8e379b74e169b98fb7c3c0ba5b4`. Rule list: `[]` (Enabled: `True`). {{(pid=87058) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:36:56.088810 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.fernet_utils [None req-2fe099a8-5f9f-48f8-a5de-7a6386ab3461 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:36:56.139566 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e1582fc-ed93-407e-9e69-a5868331bc1b None admin] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:36:56.169993 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG dbcounter [-] [87058] Writing DB stats keystone:SELECT=100 {{(pid=87058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:36:56.180544 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-106b1e4d-728d-4999-b2f8-c90e75f5a30b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:36:56.181299 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 323/644] 10.222.0.20 () {58 vars in 1023 bytes} [Mon Aug 1 16:36:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 16:36:56.181498 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106b1e4d-728d-4999-b2f8-c90e75f5a30b None admin] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:36:56.181720 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106b1e4d-728d-4999-b2f8-c90e75f5a30b None admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:36:56.182079 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106b1e4d-728d-4999-b2f8-c90e75f5a30b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:36:56.182994 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fe099a8-5f9f-48f8-a5de-7a6386ab3461 None None] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:36:56.200627 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-106b1e4d-728d-4999-b2f8-c90e75f5a30b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:36:56.203449 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-106b1e4d-728d-4999-b2f8-c90e75f5a30b None admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:36:56.207179 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG dbcounter [-] [87057] Writing DB stats keystone:SELECT=100 {{(pid=87057) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:36:56.222568 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application [None req-106b1e4d-728d-4999-b2f8-c90e75f5a30b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 16:36:56.222568 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 16:36:56.222568 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 16:36:56.222568 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 16:36:56.222568 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 16:36:56.222568 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 16:36:56.222568 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 16:36:56.222568 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 16:36:56.222568 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 16:36:56.222568 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 16:36:56.222568 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 16:36:56.222568 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 16:36:56.222568 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 16:36:56.222568 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 16:36:56.222568 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 16:36:56.223664 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 16:36:56.223664 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 16:36:56.223664 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 16:36:56.223664 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 16:36:56.223664 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 16:36:56.223664 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 16:36:56.223664 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 16:36:56.223664 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 16:36:56.223664 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application with Lock( Aug 01 16:36:56.223664 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 16:36:56.223664 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self._enter() Aug 01 16:36:56.223664 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 16:36:56.223664 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 16:36:56.223664 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 16:36:56.223664 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self.creator() Aug 01 16:36:56.223664 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 16:36:56.223664 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application created_value = creator( Aug 01 16:36:56.225000 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 16:36:56.225000 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 16:36:56.225000 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 16:36:56.225000 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application Aug 01 16:36:56.231936 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 322/645] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Aug 1 16:36:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 16:36:56.231936 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-106b1e4d-728d-4999-b2f8-c90e75f5a30b None admin] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:36:56.318526 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fe099a8-5f9f-48f8-a5de-7a6386ab3461 None None] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:36:56.332208 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-106b1e4d-728d-4999-b2f8-c90e75f5a30b None admin] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:36:56.356663 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c3aca8e-ef59-4ebe-984d-40561e611be2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:36:56.357297 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3aca8e-ef59-4ebe-984d-40561e611be2 None admin] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:36:56.357573 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3aca8e-ef59-4ebe-984d-40561e611be2 None admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:36:56.357859 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3aca8e-ef59-4ebe-984d-40561e611be2 None admin] PATH_INFO: `/v3/projects/service` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:36:56.364589 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c3aca8e-ef59-4ebe-984d-40561e611be2 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:36:56.366619 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c3aca8e-ef59-4ebe-984d-40561e611be2 None admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:36:56.367398 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d51c3a-ff43-4aa0-95a8-2d3b0f031b8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:36:56.368048 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d51c3a-ff43-4aa0-95a8-2d3b0f031b8f None admin] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:36:56.368333 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d51c3a-ff43-4aa0-95a8-2d3b0f031b8f None admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:36:56.368605 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d51c3a-ff43-4aa0-95a8-2d3b0f031b8f None admin] PATH_INFO: `/v3/users` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:36:56.369403 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05d51c3a-ff43-4aa0-95a8-2d3b0f031b8f None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=87057) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:36:56.369707 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05d51c3a-ff43-4aa0-95a8-2d3b0f031b8f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:36:56.371140 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05d51c3a-ff43-4aa0-95a8-2d3b0f031b8f None admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:36:56.378058 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application [None req-6c3aca8e-ef59-4ebe-984d-40561e611be2 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 16:36:56.378058 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 16:36:56.378058 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 16:36:56.378058 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 16:36:56.378058 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 16:36:56.378058 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 16:36:56.378058 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 16:36:56.378058 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 16:36:56.378058 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 16:36:56.378058 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 16:36:56.378058 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 16:36:56.378058 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 16:36:56.378058 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 01 16:36:56.378058 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 01 16:36:56.378058 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 01 16:36:56.379159 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 01 16:36:56.379159 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 16:36:56.379159 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 16:36:56.379159 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 16:36:56.379159 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 16:36:56.379159 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 16:36:56.379159 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 16:36:56.379159 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 16:36:56.379159 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application with Lock( Aug 01 16:36:56.379159 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 16:36:56.379159 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self._enter() Aug 01 16:36:56.379159 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 16:36:56.379159 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 16:36:56.379159 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 16:36:56.379159 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self.creator() Aug 01 16:36:56.379159 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 16:36:56.379159 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application created_value = creator( Aug 01 16:36:56.380063 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 01 16:36:56.380063 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 01 16:36:56.380063 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 01 16:36:56.380063 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 01 16:36:56.380063 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 01 16:36:56.380063 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 01 16:36:56.380063 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 16:36:56.380063 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application Aug 01 16:36:56.380063 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 324/646] 10.222.0.20 () {56 vars in 1111 bytes} [Mon Aug 1 16:36:56 2022] GET /identity/v3/projects/service => generated 88 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 16:36:56.380063 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c3aca8e-ef59-4ebe-984d-40561e611be2 None admin] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:36:56.395425 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 323/647] 10.222.0.20 () {56 vars in 1137 bytes} [Mon Aug 1 16:36:56 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:36:56.396514 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d51c3a-ff43-4aa0-95a8-2d3b0f031b8f None admin] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:36:56.445602 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG dbcounter [-] [87058] Writing DB stats keystone:SELECT=100 {{(pid=87058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:36:56.455732 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG dbcounter [-] [87057] Writing DB stats keystone:SELECT=100 {{(pid=87057) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:36:56.491231 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c3aca8e-ef59-4ebe-984d-40561e611be2 None admin] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:36:56.508109 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d51c3a-ff43-4aa0-95a8-2d3b0f031b8f None admin] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:36:56.529554 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbbd6a4b-8f3e-4c58-8ec8-d397b45b27ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:36:56.530242 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbbd6a4b-8f3e-4c58-8ec8-d397b45b27ab None admin] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:36:56.530528 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbbd6a4b-8f3e-4c58-8ec8-d397b45b27ab None admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:36:56.530806 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbbd6a4b-8f3e-4c58-8ec8-d397b45b27ab None admin] PATH_INFO: `/v3/roles/service` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:36:56.542287 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbbd6a4b-8f3e-4c58-8ec8-d397b45b27ab None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 16:36:56.547165 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application [None req-dbbd6a4b-8f3e-4c58-8ec8-d397b45b27ab None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 01 16:36:56.547165 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 16:36:56.547165 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 16:36:56.547165 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 16:36:56.547165 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 16:36:56.547165 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 16:36:56.547165 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 16:36:56.547165 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 16:36:56.547165 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 16:36:56.547165 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 16:36:56.547165 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 16:36:56.547165 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 16:36:56.547165 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 16:36:56.547165 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 16:36:56.547165 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 16:36:56.547165 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 16:36:56.548090 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 16:36:56.548090 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 16:36:56.548090 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 16:36:56.548090 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 16:36:56.548090 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 16:36:56.548090 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 16:36:56.548090 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 16:36:56.548090 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 16:36:56.548090 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 16:36:56.548090 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 16:36:56.548090 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 16:36:56.548090 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application with Lock( Aug 01 16:36:56.548090 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 16:36:56.548090 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self._enter() Aug 01 16:36:56.548090 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 16:36:56.548090 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 16:36:56.548945 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 16:36:56.548945 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self.creator() Aug 01 16:36:56.548945 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 16:36:56.548945 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application created_value = creator( Aug 01 16:36:56.548945 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 16:36:56.548945 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 16:36:56.548945 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 16:36:56.548945 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 16:36:56.548945 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 16:36:56.548945 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 16:36:56.548945 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 01 16:36:56.548945 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application Aug 01 16:36:56.548945 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 325/648] 10.222.0.20 () {56 vars in 1102 bytes} [Mon Aug 1 16:36:56 2022] GET /identity/v3/roles/service => generated 85 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 16:36:56.549641 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbbd6a4b-8f3e-4c58-8ec8-d397b45b27ab None admin] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:36:56.554200 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59336094-8645-478d-ac69-f18ea237bbf6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:36:56.555059 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59336094-8645-478d-ac69-f18ea237bbf6 None admin] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:36:56.555552 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59336094-8645-478d-ac69-f18ea237bbf6 None admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:36:56.556066 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59336094-8645-478d-ac69-f18ea237bbf6 None admin] PATH_INFO: `/v3/domains` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:36:56.557120 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59336094-8645-478d-ac69-f18ea237bbf6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87057) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:36:56.557544 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59336094-8645-478d-ac69-f18ea237bbf6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:36:56.559001 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59336094-8645-478d-ac69-f18ea237bbf6 None admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:36:56.578338 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 324/649] 10.222.0.20 () {56 vars in 1109 bytes} [Mon Aug 1 16:36:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:36:56.579170 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59336094-8645-478d-ac69-f18ea237bbf6 None admin] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:36:56.663098 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbbd6a4b-8f3e-4c58-8ec8-d397b45b27ab None admin] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:36:56.684169 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG dbcounter [-] [87058] Writing DB stats keystone:SELECT=100 {{(pid=87058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:36:56.689338 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59336094-8645-478d-ac69-f18ea237bbf6 None admin] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:36:56.704372 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG dbcounter [-] [87057] Writing DB stats keystone:SELECT=100 {{(pid=87057) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:36:56.704580 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b77202a6-6e2e-48bc-b6f4-5284c6a84503 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:36:56.705380 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77202a6-6e2e-48bc-b6f4-5284c6a84503 None admin] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:36:56.705671 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77202a6-6e2e-48bc-b6f4-5284c6a84503 None admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:36:56.706225 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77202a6-6e2e-48bc-b6f4-5284c6a84503 None admin] PATH_INFO: `/v3/projects` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:36:56.706721 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b77202a6-6e2e-48bc-b6f4-5284c6a84503 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87058) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:36:56.707037 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b77202a6-6e2e-48bc-b6f4-5284c6a84503 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:36:56.708543 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b77202a6-6e2e-48bc-b6f4-5284c6a84503 None admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:36:56.726662 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 326/650] 10.222.0.20 () {56 vars in 1148 bytes} [Mon Aug 1 16:36:56 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:36:56.727558 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b77202a6-6e2e-48bc-b6f4-5284c6a84503 None admin] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:36:56.728138 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a83038b-fed7-4fee-9442-efe724c15ee3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:36:56.728942 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a83038b-fed7-4fee-9442-efe724c15ee3 None admin] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:36:56.729733 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a83038b-fed7-4fee-9442-efe724c15ee3 None admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:36:56.730279 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a83038b-fed7-4fee-9442-efe724c15ee3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:36:56.805066 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a83038b-fed7-4fee-9442-efe724c15ee3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:36:56.806629 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a83038b-fed7-4fee-9442-efe724c15ee3 None admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:36:56.832195 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b77202a6-6e2e-48bc-b6f4-5284c6a84503 None admin] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:36:56.872171 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3651c874-5ce8-412f-b5f0-eab9c01ceef7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:36:56.872801 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3651c874-5ce8-412f-b5f0-eab9c01ceef7 None admin] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:36:56.873310 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3651c874-5ce8-412f-b5f0-eab9c01ceef7 None admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:36:56.873310 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3651c874-5ce8-412f-b5f0-eab9c01ceef7 None admin] PATH_INFO: `/v3/roles` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:36:56.874014 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3651c874-5ce8-412f-b5f0-eab9c01ceef7 None admin] RBAC: Adding query filter params (name=service) {{(pid=87058) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:36:56.874343 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3651c874-5ce8-412f-b5f0-eab9c01ceef7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:36:56.875695 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3651c874-5ce8-412f-b5f0-eab9c01ceef7 None admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:36:56.888579 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 327/651] 10.222.0.20 () {56 vars in 1103 bytes} [Mon Aug 1 16:36:56 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:36:56.888579 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3651c874-5ce8-412f-b5f0-eab9c01ceef7 None admin] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:36:56.922996 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 325/652] 10.222.0.20 () {58 vars in 1282 bytes} [Mon Aug 1 16:36:56 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 344 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 01 16:36:56.923830 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a83038b-fed7-4fee-9442-efe724c15ee3 None admin] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:36:56.938826 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG dbcounter [-] [87057] Writing DB stats keystone:SELECT=100 {{(pid=87057) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:36:56.942214 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG dbcounter [-] [87058] Writing DB stats keystone:SELECT=100 {{(pid=87058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:36:57.004325 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3651c874-5ce8-412f-b5f0-eab9c01ceef7 None admin] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:36:57.032628 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a83038b-fed7-4fee-9442-efe724c15ee3 None admin] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:36:57.041940 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9986e120-c905-403b-9cb4-43952917aa62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:36:57.042616 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9986e120-c905-403b-9cb4-43952917aa62 None admin] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:36:57.042898 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9986e120-c905-403b-9cb4-43952917aa62 None admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:36:57.043174 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9986e120-c905-403b-9cb4-43952917aa62 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:36:57.051018 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9986e120-c905-403b-9cb4-43952917aa62 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:36:57.052455 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9986e120-c905-403b-9cb4-43952917aa62 None admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:36:57.064548 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application [None req-9986e120-c905-403b-9cb4-43952917aa62 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 01 16:36:57.064548 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 16:36:57.064548 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 16:36:57.064548 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 16:36:57.064548 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 16:36:57.064548 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 16:36:57.064548 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 16:36:57.064548 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 16:36:57.064548 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 16:36:57.064548 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 16:36:57.064548 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 16:36:57.064548 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 16:36:57.064548 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 01 16:36:57.064548 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 01 16:36:57.064548 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 01 16:36:57.064548 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 01 16:36:57.065552 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 16:36:57.065552 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 16:36:57.065552 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 01 16:36:57.065552 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 16:36:57.065552 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 01 16:36:57.065552 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 16:36:57.065552 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 16:36:57.065552 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 16:36:57.065552 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 16:36:57.065552 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 16:36:57.065552 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 16:36:57.065552 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application with Lock( Aug 01 16:36:57.065552 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 16:36:57.065552 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self._enter() Aug 01 16:36:57.065552 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 16:36:57.065552 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 16:36:57.065552 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 16:36:57.065552 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self.creator() Aug 01 16:36:57.072488 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 16:36:57.072488 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application created_value = creator( Aug 01 16:36:57.072488 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 01 16:36:57.072488 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 01 16:36:57.072488 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 01 16:36:57.072488 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 01 16:36:57.072488 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 01 16:36:57.072488 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 01 16:36:57.072488 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 01 16:36:57.072488 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application Aug 01 16:36:57.072488 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 328/653] 10.222.0.20 () {56 vars in 1099 bytes} [Mon Aug 1 16:36:56 2022] GET /identity/v3/users/cinder => generated 84 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 16:36:57.072488 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9986e120-c905-403b-9cb4-43952917aa62 None admin] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:36:57.087870 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f426dfd-6c6d-4f6e-9294-badd9dc28ee6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:36:57.088866 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f426dfd-6c6d-4f6e-9294-badd9dc28ee6 None admin] REQUEST_METHOD: `PUT` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:36:57.089426 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f426dfd-6c6d-4f6e-9294-badd9dc28ee6 None admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:36:57.089702 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f426dfd-6c6d-4f6e-9294-badd9dc28ee6 None admin] PATH_INFO: `/v3/projects/7b992d6a65df4bf9b27c13192d57c5f0/users/9ab4add1faa04faa811ef7a9341cc35c/roles/51bb93e277c446b49bb80a32afcd3af7` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:36:57.106203 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f426dfd-6c6d-4f6e-9294-badd9dc28ee6 None admin] RBAC: Authorizing `identity:create_grant(project_id=7b992d6a65df4bf9b27c13192d57c5f0, user_id=9ab4add1faa04faa811ef7a9341cc35c, role_id=51bb93e277c446b49bb80a32afcd3af7)` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:36:57.107932 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f426dfd-6c6d-4f6e-9294-badd9dc28ee6 None admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:36:57.186337 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 326/654] 10.222.0.20 () {58 vars in 1439 bytes} [Mon Aug 1 16:36:56 2022] PUT /identity/v3/projects/7b992d6a65df4bf9b27c13192d57c5f0/users/9ab4add1faa04faa811ef7a9341cc35c/roles/51bb93e277c446b49bb80a32afcd3af7 => generated 0 bytes in 253 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 16:36:57.187440 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f426dfd-6c6d-4f6e-9294-badd9dc28ee6 None admin] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:36:57.294428 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG dbcounter [-] [87057] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87057) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:36:57.303813 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG dbcounter [-] [87058] Writing DB stats keystone:SELECT=100 {{(pid=87058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:36:57.306892 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9986e120-c905-403b-9cb4-43952917aa62 None admin] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:36:57.347151 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44852d23-60f0-4bfd-a03d-afa0e7718010 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:36:57.348197 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44852d23-60f0-4bfd-a03d-afa0e7718010 None admin] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:36:57.348327 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f426dfd-6c6d-4f6e-9294-badd9dc28ee6 None admin] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:36:57.348809 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44852d23-60f0-4bfd-a03d-afa0e7718010 None admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:36:57.349304 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44852d23-60f0-4bfd-a03d-afa0e7718010 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:36:57.364952 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44852d23-60f0-4bfd-a03d-afa0e7718010 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:36:57.368225 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44852d23-60f0-4bfd-a03d-afa0e7718010 None admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:36:57.388755 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application [None req-44852d23-60f0-4bfd-a03d-afa0e7718010 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 16:36:57.388755 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 16:36:57.388755 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 16:36:57.388755 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 16:36:57.388755 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 16:36:57.388755 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 16:36:57.388755 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 16:36:57.388755 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 16:36:57.388755 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 16:36:57.388755 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 16:36:57.388755 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 16:36:57.388755 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 16:36:57.388755 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 16:36:57.388755 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 16:36:57.388755 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 16:36:57.389874 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 16:36:57.389874 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 16:36:57.389874 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 16:36:57.389874 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 16:36:57.389874 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 16:36:57.389874 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 16:36:57.389874 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 16:36:57.389874 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 16:36:57.389874 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application with Lock( Aug 01 16:36:57.389874 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 16:36:57.389874 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self._enter() Aug 01 16:36:57.389874 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 16:36:57.389874 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 16:36:57.389874 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 16:36:57.389874 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self.creator() Aug 01 16:36:57.389874 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 16:36:57.389874 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application created_value = creator( Aug 01 16:36:57.390956 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 16:36:57.390956 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 16:36:57.390956 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 16:36:57.390956 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application Aug 01 16:36:57.392765 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 329/655] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Aug 1 16:36:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 326 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 16:36:57.393693 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44852d23-60f0-4bfd-a03d-afa0e7718010 None admin] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:36:57.399069 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca9be32c-0bab-4b4c-88f1-7bf3fcfc1b6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:36:57.400005 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9be32c-0bab-4b4c-88f1-7bf3fcfc1b6e None admin] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:36:57.400514 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9be32c-0bab-4b4c-88f1-7bf3fcfc1b6e None admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:36:57.400792 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9be32c-0bab-4b4c-88f1-7bf3fcfc1b6e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:36:57.413573 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca9be32c-0bab-4b4c-88f1-7bf3fcfc1b6e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:36:57.415116 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca9be32c-0bab-4b4c-88f1-7bf3fcfc1b6e None admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:36:57.432219 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application [None req-ca9be32c-0bab-4b4c-88f1-7bf3fcfc1b6e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 16:36:57.432219 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 16:36:57.432219 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 16:36:57.432219 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 16:36:57.432219 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 16:36:57.432219 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 16:36:57.432219 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 16:36:57.432219 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 16:36:57.432219 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 16:36:57.432219 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 16:36:57.432219 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 16:36:57.432219 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 16:36:57.432219 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 16:36:57.432219 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 16:36:57.432219 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 16:36:57.433333 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 16:36:57.433333 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 16:36:57.433333 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 16:36:57.433333 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 16:36:57.433333 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 16:36:57.433333 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 16:36:57.433333 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 16:36:57.433333 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 16:36:57.433333 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application with Lock( Aug 01 16:36:57.433333 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 16:36:57.433333 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self._enter() Aug 01 16:36:57.433333 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 16:36:57.433333 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 16:36:57.433333 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 16:36:57.433333 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self.creator() Aug 01 16:36:57.433333 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 16:36:57.433333 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application created_value = creator( Aug 01 16:36:57.434340 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 16:36:57.434340 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 16:36:57.434340 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 16:36:57.434340 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application Aug 01 16:36:57.434340 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 327/656] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Aug 1 16:36:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 247 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 16:36:57.434644 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca9be32c-0bab-4b4c-88f1-7bf3fcfc1b6e None admin] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:36:57.513231 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44852d23-60f0-4bfd-a03d-afa0e7718010 None admin] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:36:57.550870 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca9be32c-0bab-4b4c-88f1-7bf3fcfc1b6e None admin] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:36:57.551645 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-055b0de6-5edf-42f3-9198-8b900e86ae4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:36:57.552370 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055b0de6-5edf-42f3-9198-8b900e86ae4e None admin] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:36:57.552750 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055b0de6-5edf-42f3-9198-8b900e86ae4e None admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:36:57.553848 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055b0de6-5edf-42f3-9198-8b900e86ae4e None admin] PATH_INFO: `/v3/users` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:36:57.555416 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-055b0de6-5edf-42f3-9198-8b900e86ae4e None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=87058) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:36:57.555416 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-055b0de6-5edf-42f3-9198-8b900e86ae4e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:36:57.557088 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-055b0de6-5edf-42f3-9198-8b900e86ae4e None admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:36:57.567954 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG dbcounter [-] [87057] Writing DB stats keystone:SELECT=100 {{(pid=87057) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:36:57.578072 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 330/657] 10.222.0.20 () {56 vars in 1137 bytes} [Mon Aug 1 16:36:57 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:36:57.579053 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-055b0de6-5edf-42f3-9198-8b900e86ae4e None admin] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:36:57.591915 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2606cae-660c-475b-b464-ff481c343a5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:36:57.592828 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG dbcounter [-] [87058] Writing DB stats keystone:SELECT=100 {{(pid=87058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:36:57.593688 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2606cae-660c-475b-b464-ff481c343a5d None admin] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:36:57.594194 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2606cae-660c-475b-b464-ff481c343a5d None admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:36:57.594608 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2606cae-660c-475b-b464-ff481c343a5d None admin] PATH_INFO: `/v3/domains` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:36:57.595493 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2606cae-660c-475b-b464-ff481c343a5d None admin] RBAC: Adding query filter params (name=Default) {{(pid=87057) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:36:57.595975 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2606cae-660c-475b-b464-ff481c343a5d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:36:57.597575 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2606cae-660c-475b-b464-ff481c343a5d None admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:36:57.610617 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 328/658] 10.222.0.20 () {56 vars in 1109 bytes} [Mon Aug 1 16:36:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:36:57.611943 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2606cae-660c-475b-b464-ff481c343a5d None admin] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:36:57.694917 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-055b0de6-5edf-42f3-9198-8b900e86ae4e None admin] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:36:57.726936 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2606cae-660c-475b-b464-ff481c343a5d None admin] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:36:57.731898 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-873d41f2-17a2-4322-ba72-0b190c3cf18d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:36:57.732518 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873d41f2-17a2-4322-ba72-0b190c3cf18d None admin] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:36:57.732791 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873d41f2-17a2-4322-ba72-0b190c3cf18d None admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:36:57.733058 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873d41f2-17a2-4322-ba72-0b190c3cf18d None admin] PATH_INFO: `/v3/domains` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:36:57.733743 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-873d41f2-17a2-4322-ba72-0b190c3cf18d None admin] RBAC: Adding query filter params (name=Default) {{(pid=87058) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:36:57.734063 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-873d41f2-17a2-4322-ba72-0b190c3cf18d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:36:57.735420 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-873d41f2-17a2-4322-ba72-0b190c3cf18d None admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:36:57.751458 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 331/659] 10.222.0.20 () {56 vars in 1109 bytes} [Mon Aug 1 16:36:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:36:57.751458 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-873d41f2-17a2-4322-ba72-0b190c3cf18d None admin] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:36:57.776492 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-294731fc-26e4-4a33-8850-d586f3ba3c8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:36:57.777529 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294731fc-26e4-4a33-8850-d586f3ba3c8e None admin] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:36:57.778114 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294731fc-26e4-4a33-8850-d586f3ba3c8e None admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:36:57.778397 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294731fc-26e4-4a33-8850-d586f3ba3c8e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:36:57.823585 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG dbcounter [-] [87058] Writing DB stats keystone:SELECT=100 {{(pid=87058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:36:57.828194 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG dbcounter [-] [87057] Writing DB stats keystone:SELECT=100 {{(pid=87057) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:36:57.853821 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-294731fc-26e4-4a33-8850-d586f3ba3c8e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:36:57.855412 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-294731fc-26e4-4a33-8850-d586f3ba3c8e None admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:36:57.859301 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-873d41f2-17a2-4322-ba72-0b190c3cf18d None admin] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:36:57.893990 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8018096c-82fe-48ff-bd4d-486b7edf7f3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:36:57.894685 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8018096c-82fe-48ff-bd4d-486b7edf7f3f None admin] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:36:57.894934 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8018096c-82fe-48ff-bd4d-486b7edf7f3f None admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:36:57.895206 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8018096c-82fe-48ff-bd4d-486b7edf7f3f None admin] PATH_INFO: `/v3/users/neutron` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:36:57.907794 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8018096c-82fe-48ff-bd4d-486b7edf7f3f None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:36:57.909245 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8018096c-82fe-48ff-bd4d-486b7edf7f3f None admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:36:57.921455 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application [None req-8018096c-82fe-48ff-bd4d-486b7edf7f3f None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Aug 01 16:36:57.921455 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 16:36:57.921455 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 16:36:57.921455 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 16:36:57.921455 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 16:36:57.921455 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 16:36:57.921455 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 16:36:57.921455 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 16:36:57.921455 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 16:36:57.921455 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 16:36:57.921455 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 16:36:57.921455 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 16:36:57.921455 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 01 16:36:57.921455 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 01 16:36:57.921455 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 01 16:36:57.921455 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 01 16:36:57.922492 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 16:36:57.922492 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 16:36:57.922492 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 01 16:36:57.922492 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 16:36:57.922492 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 01 16:36:57.922492 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 16:36:57.922492 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 16:36:57.922492 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 16:36:57.922492 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 16:36:57.922492 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 16:36:57.922492 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 16:36:57.922492 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application with Lock( Aug 01 16:36:57.922492 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 16:36:57.922492 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self._enter() Aug 01 16:36:57.922492 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 16:36:57.922492 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 16:36:57.922492 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 16:36:57.922492 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self.creator() Aug 01 16:36:57.928386 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 16:36:57.928386 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application created_value = creator( Aug 01 16:36:57.928386 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 01 16:36:57.928386 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 01 16:36:57.928386 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 01 16:36:57.928386 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 01 16:36:57.928386 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 01 16:36:57.928386 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 01 16:36:57.928386 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Aug 01 16:36:57.928386 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application Aug 01 16:36:57.928386 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 332/660] 10.222.0.20 () {56 vars in 1102 bytes} [Mon Aug 1 16:36:57 2022] GET /identity/v3/users/neutron => generated 85 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 16:36:57.928386 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501a9a6f-205f-428b-81b9-414401f9e099 None None] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:36:57.928386 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501a9a6f-205f-428b-81b9-414401f9e099 None None] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:36:57.929238 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501a9a6f-205f-428b-81b9-414401f9e099 None None] PATH_INFO: `/` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:36:57.929238 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 333/661] 10.222.0.20 () {54 vars in 926 bytes} [Mon Aug 1 16:36:57 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 16:36:57.929238 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-501a9a6f-205f-428b-81b9-414401f9e099 None None] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:36:57.968996 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 329/662] 10.222.0.20 () {58 vars in 1282 bytes} [Mon Aug 1 16:36:57 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 357 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 01 16:36:57.969866 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-294731fc-26e4-4a33-8850-d586f3ba3c8e None admin] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:36:58.040605 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-501a9a6f-205f-428b-81b9-414401f9e099 None None] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:36:58.051781 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG dbcounter [-] [87057] Writing DB stats keystone:SELECT=100 {{(pid=87057) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:36:58.070297 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG dbcounter [-] [87058] Writing DB stats keystone:SELECT=100 {{(pid=87058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:36:58.075881 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0eab4d5-0299-4cd5-81ec-0693838c7f04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:36:58.076595 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0eab4d5-0299-4cd5-81ec-0693838c7f04 None admin] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:36:58.076946 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0eab4d5-0299-4cd5-81ec-0693838c7f04 None admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:36:58.077282 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0eab4d5-0299-4cd5-81ec-0693838c7f04 None admin] PATH_INFO: `/v3/projects/service` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:36:58.077707 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-294731fc-26e4-4a33-8850-d586f3ba3c8e None admin] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:36:58.084077 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0eab4d5-0299-4cd5-81ec-0693838c7f04 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:36:58.085924 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0eab4d5-0299-4cd5-81ec-0693838c7f04 None admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:36:58.097572 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application [None req-a0eab4d5-0299-4cd5-81ec-0693838c7f04 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 16:36:58.097572 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 16:36:58.097572 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 16:36:58.097572 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 16:36:58.097572 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 16:36:58.097572 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 16:36:58.097572 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 16:36:58.097572 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 16:36:58.097572 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 16:36:58.097572 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 16:36:58.097572 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 16:36:58.097572 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 16:36:58.097572 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 01 16:36:58.097572 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 01 16:36:58.097572 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 01 16:36:58.098779 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 01 16:36:58.098779 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 16:36:58.098779 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 16:36:58.098779 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 16:36:58.098779 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 16:36:58.098779 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 16:36:58.098779 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 16:36:58.098779 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 16:36:58.098779 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application with Lock( Aug 01 16:36:58.098779 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 16:36:58.098779 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self._enter() Aug 01 16:36:58.098779 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 16:36:58.098779 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 16:36:58.098779 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 16:36:58.098779 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self.creator() Aug 01 16:36:58.098779 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 16:36:58.098779 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application created_value = creator( Aug 01 16:36:58.105802 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 01 16:36:58.105802 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 01 16:36:58.105802 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 01 16:36:58.105802 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 01 16:36:58.105802 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 01 16:36:58.105802 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 01 16:36:58.105802 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 16:36:58.105802 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application Aug 01 16:36:58.105802 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 334/663] 10.222.0.20 () {56 vars in 1111 bytes} [Mon Aug 1 16:36:57 2022] GET /identity/v3/projects/service => generated 88 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 16:36:58.105802 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84247c5-f92e-4ffe-bd92-a3f94343eadb None None] REQUEST_METHOD: `POST` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:36:58.105802 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84247c5-f92e-4ffe-bd92-a3f94343eadb None None] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:36:58.106601 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84247c5-f92e-4ffe-bd92-a3f94343eadb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:36:58.119489 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.auth.core [None req-f84247c5-f92e-4ffe-bd92-a3f94343eadb None None] MFA Rules not processed for user `1007f8e379b74e169b98fb7c3c0ba5b4`. Rule list: `[]` (Enabled: `True`). {{(pid=87058) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:36:58.120685 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.fernet_utils [None req-f84247c5-f92e-4ffe-bd92-a3f94343eadb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:36:58.122020 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-381ef220-7e68-4f44-aeeb-5af2de9b5038 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:36:58.122849 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381ef220-7e68-4f44-aeeb-5af2de9b5038 None admin] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:36:58.123223 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381ef220-7e68-4f44-aeeb-5af2de9b5038 None admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:36:58.124059 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381ef220-7e68-4f44-aeeb-5af2de9b5038 None admin] PATH_INFO: `/v3/users` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:36:58.124360 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-381ef220-7e68-4f44-aeeb-5af2de9b5038 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=87057) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:36:58.124721 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-381ef220-7e68-4f44-aeeb-5af2de9b5038 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:36:58.126168 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-381ef220-7e68-4f44-aeeb-5af2de9b5038 None admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:36:58.144972 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 330/664] 10.222.0.20 () {56 vars in 1139 bytes} [Mon Aug 1 16:36:57 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:36:58.150595 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-381ef220-7e68-4f44-aeeb-5af2de9b5038 None admin] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:36:58.194291 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 335/665] 10.222.0.20 () {58 vars in 1023 bytes} [Mon Aug 1 16:36:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 16:36:58.198217 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f84247c5-f92e-4ffe-bd92-a3f94343eadb None None] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:36:58.258003 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-381ef220-7e68-4f44-aeeb-5af2de9b5038 None admin] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:36:58.290016 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG dbcounter [-] [87057] Writing DB stats keystone:SELECT=100 {{(pid=87057) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:36:58.295256 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56f3fa32-08c5-41a5-a6a6-65ba6bbd7f11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:36:58.295941 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f3fa32-08c5-41a5-a6a6-65ba6bbd7f11 None admin] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:36:58.296206 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f3fa32-08c5-41a5-a6a6-65ba6bbd7f11 None admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:36:58.296478 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f3fa32-08c5-41a5-a6a6-65ba6bbd7f11 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:36:58.303973 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f84247c5-f92e-4ffe-bd92-a3f94343eadb None None] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:36:58.307172 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56f3fa32-08c5-41a5-a6a6-65ba6bbd7f11 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:36:58.308619 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56f3fa32-08c5-41a5-a6a6-65ba6bbd7f11 None admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:36:58.324237 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application [None req-56f3fa32-08c5-41a5-a6a6-65ba6bbd7f11 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 16:36:58.324237 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 16:36:58.324237 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 16:36:58.324237 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 16:36:58.324237 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 16:36:58.324237 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 16:36:58.324237 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 16:36:58.324237 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 16:36:58.324237 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 16:36:58.324237 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 16:36:58.324237 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 16:36:58.324237 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 16:36:58.324237 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 16:36:58.324237 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 16:36:58.324237 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 16:36:58.325797 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 16:36:58.325797 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 16:36:58.325797 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 16:36:58.325797 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 16:36:58.325797 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 16:36:58.325797 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 16:36:58.325797 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 16:36:58.325797 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 16:36:58.325797 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application with Lock( Aug 01 16:36:58.325797 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 16:36:58.325797 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self._enter() Aug 01 16:36:58.325797 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 16:36:58.325797 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 16:36:58.325797 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 16:36:58.325797 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self.creator() Aug 01 16:36:58.325797 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 16:36:58.325797 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application created_value = creator( Aug 01 16:36:58.326851 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 16:36:58.326851 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 16:36:58.326851 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 16:36:58.326851 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application Aug 01 16:36:58.326851 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 331/666] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Aug 1 16:36:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 16:36:58.326851 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56f3fa32-08c5-41a5-a6a6-65ba6bbd7f11 None admin] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:36:58.334793 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG dbcounter [-] [87058] Writing DB stats keystone:SELECT=100 {{(pid=87058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:36:58.351006 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e1db0d8-eead-4f82-a5f4-06015ec3fc3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:36:58.351645 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1db0d8-eead-4f82-a5f4-06015ec3fc3b None admin] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:36:58.351914 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1db0d8-eead-4f82-a5f4-06015ec3fc3b None admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:36:58.352183 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1db0d8-eead-4f82-a5f4-06015ec3fc3b None admin] PATH_INFO: `/v3/projects` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:36:58.353024 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e1db0d8-eead-4f82-a5f4-06015ec3fc3b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87058) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:36:58.353328 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e1db0d8-eead-4f82-a5f4-06015ec3fc3b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:36:58.354780 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e1db0d8-eead-4f82-a5f4-06015ec3fc3b None admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:36:58.372004 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 336/667] 10.222.0.20 () {56 vars in 1148 bytes} [Mon Aug 1 16:36:58 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:36:58.372004 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e1db0d8-eead-4f82-a5f4-06015ec3fc3b None admin] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:36:58.437667 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56f3fa32-08c5-41a5-a6a6-65ba6bbd7f11 None admin] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:36:58.472446 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e1db0d8-eead-4f82-a5f4-06015ec3fc3b None admin] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:36:58.472854 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd678d1a-3aa3-4ba8-8d2f-cd06066fc586 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:36:58.473539 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd678d1a-3aa3-4ba8-8d2f-cd06066fc586 None admin] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:36:58.473885 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd678d1a-3aa3-4ba8-8d2f-cd06066fc586 None admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:36:58.474325 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd678d1a-3aa3-4ba8-8d2f-cd06066fc586 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:36:58.507368 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74cb6f17-baa5-4278-9f89-2fba4239d9b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:36:58.508009 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74cb6f17-baa5-4278-9f89-2fba4239d9b1 None admin] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:36:58.508296 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74cb6f17-baa5-4278-9f89-2fba4239d9b1 None admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:36:58.508567 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74cb6f17-baa5-4278-9f89-2fba4239d9b1 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:36:58.518818 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74cb6f17-baa5-4278-9f89-2fba4239d9b1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 16:36:58.524420 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application [None req-74cb6f17-baa5-4278-9f89-2fba4239d9b1 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 16:36:58.524420 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 16:36:58.524420 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 16:36:58.524420 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 16:36:58.524420 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 16:36:58.524420 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 16:36:58.524420 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 16:36:58.524420 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 16:36:58.524420 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 16:36:58.524420 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 16:36:58.524420 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 16:36:58.524420 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 16:36:58.524420 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 16:36:58.524420 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 16:36:58.524420 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 16:36:58.524420 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 16:36:58.525579 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 16:36:58.525579 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 16:36:58.525579 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 16:36:58.525579 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 16:36:58.525579 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 16:36:58.525579 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 16:36:58.525579 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 16:36:58.525579 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 16:36:58.525579 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 16:36:58.525579 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 16:36:58.525579 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 16:36:58.525579 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application with Lock( Aug 01 16:36:58.525579 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 16:36:58.525579 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self._enter() Aug 01 16:36:58.525579 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 16:36:58.525579 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 16:36:58.526642 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 16:36:58.526642 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self.creator() Aug 01 16:36:58.526642 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 16:36:58.526642 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application created_value = creator( Aug 01 16:36:58.526642 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 16:36:58.526642 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 16:36:58.526642 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 16:36:58.526642 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 16:36:58.526642 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 16:36:58.526642 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 16:36:58.526642 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 16:36:58.526642 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application Aug 01 16:36:58.530295 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 337/668] 10.222.0.20 () {56 vars in 1096 bytes} [Mon Aug 1 16:36:58 2022] GET /identity/v3/roles/admin => generated 83 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 16:36:58.531121 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74cb6f17-baa5-4278-9f89-2fba4239d9b1 None admin] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:36:58.533829 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG dbcounter [-] [87057] Writing DB stats keystone:SELECT=100 {{(pid=87057) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:36:58.549707 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd678d1a-3aa3-4ba8-8d2f-cd06066fc586 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:36:58.551270 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd678d1a-3aa3-4ba8-8d2f-cd06066fc586 None admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:36:58.632308 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74cb6f17-baa5-4278-9f89-2fba4239d9b1 None admin] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:36:58.657586 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 332/669] 10.222.0.20 () {58 vars in 1282 bytes} [Mon Aug 1 16:36:58 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 331 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 01 16:36:58.658478 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd678d1a-3aa3-4ba8-8d2f-cd06066fc586 None admin] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:36:58.671264 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d609e09a-71a7-4fdf-ac14-3970e13bd383 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:36:58.671875 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d609e09a-71a7-4fdf-ac14-3970e13bd383 None admin] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:36:58.672154 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d609e09a-71a7-4fdf-ac14-3970e13bd383 None admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:36:58.672419 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d609e09a-71a7-4fdf-ac14-3970e13bd383 None admin] PATH_INFO: `/v3/domains` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:36:58.673105 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d609e09a-71a7-4fdf-ac14-3970e13bd383 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87058) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:36:58.673399 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d609e09a-71a7-4fdf-ac14-3970e13bd383 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:36:58.674774 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d609e09a-71a7-4fdf-ac14-3970e13bd383 None admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:36:58.686763 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 338/670] 10.222.0.20 () {56 vars in 1109 bytes} [Mon Aug 1 16:36:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:36:58.687802 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d609e09a-71a7-4fdf-ac14-3970e13bd383 None admin] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:36:58.691739 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG dbcounter [-] [87058] Writing DB stats keystone:SELECT=100 {{(pid=87058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:36:58.750387 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG dbcounter [-] [87057] Writing DB stats keystone:SELECT=100 {{(pid=87057) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:36:58.767007 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd678d1a-3aa3-4ba8-8d2f-cd06066fc586 None admin] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:36:58.794707 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG dbcounter [-] [87058] Writing DB stats keystone:SELECT=100 {{(pid=87058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:36:58.797819 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d609e09a-71a7-4fdf-ac14-3970e13bd383 None admin] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:36:58.804826 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76674d8b-d9f0-4a1a-93e7-08a621de8499 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:36:58.809348 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76674d8b-d9f0-4a1a-93e7-08a621de8499 None admin] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:36:58.809348 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76674d8b-d9f0-4a1a-93e7-08a621de8499 None admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:36:58.809348 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76674d8b-d9f0-4a1a-93e7-08a621de8499 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:36:58.809348 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76674d8b-d9f0-4a1a-93e7-08a621de8499 None admin] RBAC: Adding query filter params (role.id=51bb93e277c446b49bb80a32afcd3af7, scope.project.id=7b992d6a65df4bf9b27c13192d57c5f0, user.id=d80694e439e2401d9491712666409016) {{(pid=87057) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:36:58.809348 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76674d8b-d9f0-4a1a-93e7-08a621de8499 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:36:58.809348 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76674d8b-d9f0-4a1a-93e7-08a621de8499 None admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:36:58.822070 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 333/671] 10.222.0.20 () {56 vars in 1374 bytes} [Mon Aug 1 16:36:58 2022] GET /identity/v3/role_assignments?user.id=d80694e439e2401d9491712666409016&scope.project.id=7b992d6a65df4bf9b27c13192d57c5f0&role.id=51bb93e277c446b49bb80a32afcd3af7 => generated 608 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:36:58.822070 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76674d8b-d9f0-4a1a-93e7-08a621de8499 None admin] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:36:58.845354 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-150ff18d-ed5b-4b42-a896-adb9c996fd7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:36:58.846008 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-150ff18d-ed5b-4b42-a896-adb9c996fd7e None admin] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:36:58.846314 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-150ff18d-ed5b-4b42-a896-adb9c996fd7e None admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:36:58.846590 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-150ff18d-ed5b-4b42-a896-adb9c996fd7e None admin] PATH_INFO: `/v3/roles` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:36:58.847340 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-150ff18d-ed5b-4b42-a896-adb9c996fd7e None admin] RBAC: Adding query filter params (name=admin) {{(pid=87058) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:36:58.847650 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-150ff18d-ed5b-4b42-a896-adb9c996fd7e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:36:58.850434 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-150ff18d-ed5b-4b42-a896-adb9c996fd7e None admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:36:58.863724 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 339/672] 10.222.0.20 () {56 vars in 1099 bytes} [Mon Aug 1 16:36:58 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:36:58.864777 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-150ff18d-ed5b-4b42-a896-adb9c996fd7e None admin] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:36:58.994052 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76674d8b-d9f0-4a1a-93e7-08a621de8499 None admin] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:36:59.012877 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-150ff18d-ed5b-4b42-a896-adb9c996fd7e None admin] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:36:59.038709 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd3e65f-7cf2-4778-88da-ab6f9c608c4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:36:59.041785 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd3e65f-7cf2-4778-88da-ab6f9c608c4e None admin] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:36:59.041785 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd3e65f-7cf2-4778-88da-ab6f9c608c4e None admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:36:59.041785 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd3e65f-7cf2-4778-88da-ab6f9c608c4e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:36:59.057328 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bd3e65f-7cf2-4778-88da-ab6f9c608c4e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:36:59.058770 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bd3e65f-7cf2-4778-88da-ab6f9c608c4e None admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:36:59.061433 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2fd51b9-e73e-47e5-9d3d-6133e74cca2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:36:59.062198 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2fd51b9-e73e-47e5-9d3d-6133e74cca2e None admin] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:36:59.062482 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2fd51b9-e73e-47e5-9d3d-6133e74cca2e None admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:36:59.062755 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2fd51b9-e73e-47e5-9d3d-6133e74cca2e None admin] PATH_INFO: `/v3/projects/service` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:36:59.069215 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG dbcounter [-] [87057] Writing DB stats keystone:SELECT=100 {{(pid=87057) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:36:59.082440 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2fd51b9-e73e-47e5-9d3d-6133e74cca2e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:36:59.082440 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2fd51b9-e73e-47e5-9d3d-6133e74cca2e None admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:36:59.090926 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application [None req-7bd3e65f-7cf2-4778-88da-ab6f9c608c4e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 16:36:59.090926 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 16:36:59.090926 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 16:36:59.090926 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 16:36:59.090926 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 16:36:59.090926 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 16:36:59.090926 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 16:36:59.090926 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 16:36:59.090926 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 16:36:59.090926 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 16:36:59.090926 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 16:36:59.090926 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 16:36:59.090926 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 16:36:59.090926 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 16:36:59.090926 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 16:36:59.091750 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 16:36:59.091750 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 16:36:59.091750 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 16:36:59.091750 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 16:36:59.091750 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 16:36:59.091750 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 16:36:59.091750 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 16:36:59.091750 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 16:36:59.091750 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application with Lock( Aug 01 16:36:59.091750 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 16:36:59.091750 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self._enter() Aug 01 16:36:59.091750 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 16:36:59.091750 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 16:36:59.091750 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 16:36:59.091750 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self.creator() Aug 01 16:36:59.091750 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 16:36:59.091750 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application created_value = creator( Aug 01 16:36:59.092625 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 16:36:59.092625 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 16:36:59.092625 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 16:36:59.092625 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application Aug 01 16:36:59.092625 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 334/673] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Aug 1 16:36:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 263 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 16:36:59.092625 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd3e65f-7cf2-4778-88da-ab6f9c608c4e None admin] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:36:59.098280 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application [None req-a2fd51b9-e73e-47e5-9d3d-6133e74cca2e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 16:36:59.098280 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 16:36:59.098280 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 16:36:59.098280 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 16:36:59.098280 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 16:36:59.098280 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 16:36:59.098280 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 16:36:59.098280 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 16:36:59.098280 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 16:36:59.098280 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 16:36:59.098280 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 16:36:59.098280 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 16:36:59.098280 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 01 16:36:59.098280 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 01 16:36:59.098280 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 01 16:36:59.099195 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 01 16:36:59.099195 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 16:36:59.099195 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 16:36:59.099195 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 16:36:59.099195 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 16:36:59.099195 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 16:36:59.099195 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 16:36:59.099195 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 16:36:59.099195 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application with Lock( Aug 01 16:36:59.099195 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 16:36:59.099195 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self._enter() Aug 01 16:36:59.099195 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 16:36:59.099195 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 16:36:59.099195 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 16:36:59.099195 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self.creator() Aug 01 16:36:59.099195 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 16:36:59.099195 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application created_value = creator( Aug 01 16:36:59.100167 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 01 16:36:59.100167 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 01 16:36:59.100167 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 01 16:36:59.100167 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 01 16:36:59.100167 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 01 16:36:59.100167 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 01 16:36:59.100167 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 16:36:59.100167 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application Aug 01 16:36:59.100167 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 340/674] 10.222.0.20 () {56 vars in 1111 bytes} [Mon Aug 1 16:36:58 2022] GET /identity/v3/projects/service => generated 88 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 16:36:59.100694 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2fd51b9-e73e-47e5-9d3d-6133e74cca2e None admin] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:36:59.150526 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG dbcounter [-] [87058] Writing DB stats keystone:SELECT=100 {{(pid=87058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:36:59.221454 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd3e65f-7cf2-4778-88da-ab6f9c608c4e None admin] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:36:59.228327 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2fd51b9-e73e-47e5-9d3d-6133e74cca2e None admin] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:36:59.260217 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d75ea041-c7d3-4ec9-8119-912e502da2e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:36:59.260968 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75ea041-c7d3-4ec9-8119-912e502da2e3 None admin] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:36:59.261251 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75ea041-c7d3-4ec9-8119-912e502da2e3 None admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:36:59.261525 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75ea041-c7d3-4ec9-8119-912e502da2e3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:36:59.275094 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d75ea041-c7d3-4ec9-8119-912e502da2e3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:36:59.276430 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-199f83ed-0ccd-4087-971d-9ef2cb89486f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:36:59.276726 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d75ea041-c7d3-4ec9-8119-912e502da2e3 None admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:36:59.277523 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199f83ed-0ccd-4087-971d-9ef2cb89486f None admin] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:36:59.278007 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199f83ed-0ccd-4087-971d-9ef2cb89486f None admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:36:59.278786 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199f83ed-0ccd-4087-971d-9ef2cb89486f None admin] PATH_INFO: `/v3/domains` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:36:59.280148 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-199f83ed-0ccd-4087-971d-9ef2cb89486f None admin] RBAC: Adding query filter params (name=Default) {{(pid=87058) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:36:59.280672 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-199f83ed-0ccd-4087-971d-9ef2cb89486f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:36:59.282288 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-199f83ed-0ccd-4087-971d-9ef2cb89486f None admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:36:59.292929 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application [None req-d75ea041-c7d3-4ec9-8119-912e502da2e3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 16:36:59.292929 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 16:36:59.292929 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 16:36:59.292929 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 16:36:59.292929 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 16:36:59.292929 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 16:36:59.292929 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 16:36:59.292929 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 16:36:59.292929 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 16:36:59.292929 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 16:36:59.292929 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 16:36:59.292929 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 16:36:59.292929 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 16:36:59.292929 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 16:36:59.292929 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 16:36:59.293861 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 16:36:59.293861 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 16:36:59.293861 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 16:36:59.293861 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 16:36:59.293861 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 16:36:59.293861 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 16:36:59.293861 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 16:36:59.293861 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 16:36:59.293861 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application with Lock( Aug 01 16:36:59.293861 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 16:36:59.293861 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self._enter() Aug 01 16:36:59.293861 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 16:36:59.293861 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 16:36:59.293861 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 16:36:59.293861 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self.creator() Aug 01 16:36:59.293861 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 16:36:59.293861 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application created_value = creator( Aug 01 16:36:59.294799 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 16:36:59.294799 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 16:36:59.294799 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 16:36:59.294799 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application Aug 01 16:36:59.298183 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 335/675] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Aug 1 16:36:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 16:36:59.299069 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d75ea041-c7d3-4ec9-8119-912e502da2e3 None admin] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:36:59.304661 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 341/676] 10.222.0.20 () {56 vars in 1109 bytes} [Mon Aug 1 16:36:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:36:59.310405 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-199f83ed-0ccd-4087-971d-9ef2cb89486f None admin] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:36:59.359354 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG dbcounter [-] [87057] Writing DB stats keystone:SELECT=100 {{(pid=87057) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:36:59.400830 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG dbcounter [-] [87058] Writing DB stats keystone:SELECT=100 {{(pid=87058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:36:59.408131 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d75ea041-c7d3-4ec9-8119-912e502da2e3 None admin] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:36:59.421582 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-199f83ed-0ccd-4087-971d-9ef2cb89486f None admin] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:36:59.442999 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b22169-d330-4bb6-8ec0-b4ca374cef9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:36:59.444261 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b22169-d330-4bb6-8ec0-b4ca374cef9b None admin] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:36:59.444653 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b22169-d330-4bb6-8ec0-b4ca374cef9b None admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:36:59.444930 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b22169-d330-4bb6-8ec0-b4ca374cef9b None admin] PATH_INFO: `/v3/projects` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:36:59.445686 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86b22169-d330-4bb6-8ec0-b4ca374cef9b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87057) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:36:59.445997 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86b22169-d330-4bb6-8ec0-b4ca374cef9b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:36:59.447463 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86b22169-d330-4bb6-8ec0-b4ca374cef9b None admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:36:59.459680 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 336/677] 10.222.0.20 () {56 vars in 1148 bytes} [Mon Aug 1 16:36:59 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:36:59.460498 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b22169-d330-4bb6-8ec0-b4ca374cef9b None admin] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:36:59.465315 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-653555ae-9e5c-4582-9be2-82a0f795d03e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:36:59.465939 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653555ae-9e5c-4582-9be2-82a0f795d03e None admin] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:36:59.466241 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653555ae-9e5c-4582-9be2-82a0f795d03e None admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:36:59.466509 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653555ae-9e5c-4582-9be2-82a0f795d03e None admin] PATH_INFO: `/v3/domains` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:36:59.467321 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-653555ae-9e5c-4582-9be2-82a0f795d03e None admin] RBAC: Adding query filter params (name=Default) {{(pid=87058) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:36:59.467733 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-653555ae-9e5c-4582-9be2-82a0f795d03e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:36:59.469128 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-653555ae-9e5c-4582-9be2-82a0f795d03e None admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:36:59.486060 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 342/678] 10.222.0.20 () {56 vars in 1109 bytes} [Mon Aug 1 16:36:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:36:59.486060 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04432e3c-556b-415d-8039-2cd8e8613624 None None] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:36:59.486060 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04432e3c-556b-415d-8039-2cd8e8613624 None None] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:36:59.486060 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04432e3c-556b-415d-8039-2cd8e8613624 None None] PATH_INFO: `/` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:36:59.486060 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 343/679] 10.222.0.20 () {54 vars in 926 bytes} [Mon Aug 1 16:36:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 16:36:59.486060 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04432e3c-556b-415d-8039-2cd8e8613624 None None] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:36:59.574352 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b22169-d330-4bb6-8ec0-b4ca374cef9b None admin] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:36:59.588009 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG dbcounter [-] [87057] Writing DB stats keystone:SELECT=100 {{(pid=87057) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:36:59.603635 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04432e3c-556b-415d-8039-2cd8e8613624 None None] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:36:59.609402 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e9396c4-5f74-4337-abcf-ae7cbb78f4e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:36:59.610073 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9396c4-5f74-4337-abcf-ae7cbb78f4e0 None admin] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:36:59.610359 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9396c4-5f74-4337-abcf-ae7cbb78f4e0 None admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:36:59.610704 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9396c4-5f74-4337-abcf-ae7cbb78f4e0 None admin] PATH_INFO: `/v3/projects/service` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:36:59.618502 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e9396c4-5f74-4337-abcf-ae7cbb78f4e0 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:36:59.620243 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e9396c4-5f74-4337-abcf-ae7cbb78f4e0 None admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:36:59.632310 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application [None req-8e9396c4-5f74-4337-abcf-ae7cbb78f4e0 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 16:36:59.632310 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 16:36:59.632310 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 16:36:59.632310 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 16:36:59.632310 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 16:36:59.632310 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 16:36:59.632310 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 16:36:59.632310 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 16:36:59.632310 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 16:36:59.632310 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 16:36:59.632310 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 16:36:59.632310 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 16:36:59.632310 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 01 16:36:59.632310 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 01 16:36:59.632310 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 01 16:36:59.633265 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 01 16:36:59.633265 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 16:36:59.633265 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 16:36:59.633265 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 16:36:59.633265 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 16:36:59.633265 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 16:36:59.633265 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 16:36:59.633265 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 16:36:59.633265 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application with Lock( Aug 01 16:36:59.633265 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 16:36:59.633265 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self._enter() Aug 01 16:36:59.633265 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 16:36:59.633265 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 16:36:59.633265 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 16:36:59.633265 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self.creator() Aug 01 16:36:59.633265 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 16:36:59.633265 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application created_value = creator( Aug 01 16:36:59.634736 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 01 16:36:59.634736 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 01 16:36:59.634736 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 01 16:36:59.634736 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 01 16:36:59.634736 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 01 16:36:59.634736 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 01 16:36:59.634736 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 16:36:59.634736 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application Aug 01 16:36:59.634736 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 337/680] 10.222.0.20 () {56 vars in 1111 bytes} [Mon Aug 1 16:36:59 2022] GET /identity/v3/projects/service => generated 88 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 16:36:59.634736 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e9396c4-5f74-4337-abcf-ae7cbb78f4e0 None admin] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:36:59.644121 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ff7710-3bee-497d-875e-1af003f103c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:36:59.644740 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ff7710-3bee-497d-875e-1af003f103c8 None admin] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:36:59.645019 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ff7710-3bee-497d-875e-1af003f103c8 None admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:36:59.645288 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ff7710-3bee-497d-875e-1af003f103c8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:36:59.646113 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7ff7710-3bee-497d-875e-1af003f103c8 None admin] RBAC: Adding query filter params (role.id=51bb93e277c446b49bb80a32afcd3af7, scope.project.id=7b992d6a65df4bf9b27c13192d57c5f0, user.id=25970b52ca15448fa1abb1f974d64038) {{(pid=87058) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:36:59.646421 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7ff7710-3bee-497d-875e-1af003f103c8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:36:59.648393 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7ff7710-3bee-497d-875e-1af003f103c8 None admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:36:59.654680 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 344/681] 10.222.0.20 () {56 vars in 1374 bytes} [Mon Aug 1 16:36:59 2022] GET /identity/v3/role_assignments?user.id=25970b52ca15448fa1abb1f974d64038&scope.project.id=7b992d6a65df4bf9b27c13192d57c5f0&role.id=51bb93e277c446b49bb80a32afcd3af7 => generated 608 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:36:59.668535 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ddd69a-5af2-474a-afbf-c183f56ba4f5 None None] REQUEST_METHOD: `POST` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:36:59.669217 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ddd69a-5af2-474a-afbf-c183f56ba4f5 None None] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:36:59.669660 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ddd69a-5af2-474a-afbf-c183f56ba4f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:36:59.685728 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG dbcounter [-] [87058] Writing DB stats keystone:SELECT=100 {{(pid=87058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:36:59.692282 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.auth.core [None req-26ddd69a-5af2-474a-afbf-c183f56ba4f5 None None] MFA Rules not processed for user `1007f8e379b74e169b98fb7c3c0ba5b4`. Rule list: `[]` (Enabled: `True`). {{(pid=87058) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:36:59.694487 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.fernet_utils [None req-26ddd69a-5af2-474a-afbf-c183f56ba4f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:36:59.750790 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e9396c4-5f74-4337-abcf-ae7cbb78f4e0 None admin] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:36:59.778456 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 345/682] 10.222.0.20 () {58 vars in 1023 bytes} [Mon Aug 1 16:36:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 16:36:59.779701 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26ddd69a-5af2-474a-afbf-c183f56ba4f5 None None] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:36:59.804286 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-666b5898-be0e-4ea7-81c6-942787f31bfb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:36:59.805672 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-666b5898-be0e-4ea7-81c6-942787f31bfb None admin] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:36:59.805953 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-666b5898-be0e-4ea7-81c6-942787f31bfb None admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:36:59.806268 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-666b5898-be0e-4ea7-81c6-942787f31bfb None admin] PATH_INFO: `/v3/users/nova` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:36:59.818331 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-666b5898-be0e-4ea7-81c6-942787f31bfb None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:36:59.820471 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-666b5898-be0e-4ea7-81c6-942787f31bfb None admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:36:59.837474 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application [None req-666b5898-be0e-4ea7-81c6-942787f31bfb None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 01 16:36:59.837474 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 16:36:59.837474 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 16:36:59.837474 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 16:36:59.837474 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 16:36:59.837474 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 16:36:59.837474 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 16:36:59.837474 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 16:36:59.837474 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 16:36:59.837474 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 16:36:59.837474 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 16:36:59.837474 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 16:36:59.837474 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 01 16:36:59.837474 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 01 16:36:59.837474 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 01 16:36:59.837474 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 01 16:36:59.838612 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 16:36:59.838612 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 16:36:59.838612 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 01 16:36:59.838612 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 16:36:59.838612 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 01 16:36:59.838612 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 16:36:59.838612 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 16:36:59.838612 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 16:36:59.838612 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 16:36:59.838612 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 16:36:59.838612 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 16:36:59.838612 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application with Lock( Aug 01 16:36:59.838612 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 16:36:59.838612 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self._enter() Aug 01 16:36:59.838612 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 16:36:59.838612 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 16:36:59.838612 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 16:36:59.838612 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self.creator() Aug 01 16:36:59.839603 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 16:36:59.839603 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application created_value = creator( Aug 01 16:36:59.839603 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 01 16:36:59.839603 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 01 16:36:59.839603 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 01 16:36:59.839603 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 01 16:36:59.839603 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 01 16:36:59.839603 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 01 16:36:59.839603 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 01 16:36:59.839603 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application Aug 01 16:36:59.839603 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 338/683] 10.222.0.20 () {56 vars in 1093 bytes} [Mon Aug 1 16:36:59 2022] GET /identity/v3/users/nova => generated 82 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 16:36:59.840239 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-666b5898-be0e-4ea7-81c6-942787f31bfb None admin] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:36:59.877120 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG dbcounter [-] [87057] Writing DB stats keystone:SELECT=100 {{(pid=87057) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:36:59.933438 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26ddd69a-5af2-474a-afbf-c183f56ba4f5 None None] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:36:59.957715 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-666b5898-be0e-4ea7-81c6-942787f31bfb None admin] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:36:59.973307 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65eb9c77-0026-4c10-8204-f41e38b898d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:36:59.974434 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65eb9c77-0026-4c10-8204-f41e38b898d4 None admin] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:36:59.974896 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65eb9c77-0026-4c10-8204-f41e38b898d4 None admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:36:59.975377 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65eb9c77-0026-4c10-8204-f41e38b898d4 None admin] PATH_INFO: `/v3/projects` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:36:59.976275 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65eb9c77-0026-4c10-8204-f41e38b898d4 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87058) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:36:59.976598 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65eb9c77-0026-4c10-8204-f41e38b898d4 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:36:59.978756 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65eb9c77-0026-4c10-8204-f41e38b898d4 None admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:36:59.981888 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG dbcounter [-] [87058] Writing DB stats keystone:SELECT=100 {{(pid=87058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:36:59.995045 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 346/684] 10.222.0.20 () {56 vars in 1148 bytes} [Mon Aug 1 16:36:59 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:36:59.996524 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65eb9c77-0026-4c10-8204-f41e38b898d4 None admin] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:37:00.000788 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd82990-0e9a-4149-89ce-818f78a888e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:37:00.001614 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd82990-0e9a-4149-89ce-818f78a888e1 None admin] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:37:00.001959 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd82990-0e9a-4149-89ce-818f78a888e1 None admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:37:00.002335 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd82990-0e9a-4149-89ce-818f78a888e1 None admin] PATH_INFO: `/v3/roles/service` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:37:00.014439 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bd82990-0e9a-4149-89ce-818f78a888e1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 16:37:00.019781 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application [None req-7bd82990-0e9a-4149-89ce-818f78a888e1 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 01 16:37:00.019781 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 16:37:00.019781 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 16:37:00.019781 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 16:37:00.019781 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 16:37:00.019781 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 16:37:00.019781 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 16:37:00.019781 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 16:37:00.019781 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 16:37:00.019781 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 16:37:00.019781 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 16:37:00.019781 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 16:37:00.019781 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 16:37:00.019781 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 16:37:00.019781 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 16:37:00.019781 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 16:37:00.020957 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 16:37:00.020957 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 16:37:00.020957 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 16:37:00.020957 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 16:37:00.020957 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 16:37:00.020957 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 16:37:00.020957 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 16:37:00.020957 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 16:37:00.020957 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 16:37:00.020957 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 16:37:00.020957 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 16:37:00.020957 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application with Lock( Aug 01 16:37:00.020957 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 16:37:00.020957 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self._enter() Aug 01 16:37:00.020957 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 16:37:00.020957 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 16:37:00.022117 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 16:37:00.022117 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self.creator() Aug 01 16:37:00.022117 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 16:37:00.022117 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application created_value = creator( Aug 01 16:37:00.022117 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 16:37:00.022117 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 16:37:00.022117 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 16:37:00.022117 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 16:37:00.022117 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 16:37:00.022117 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 16:37:00.022117 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 01 16:37:00.022117 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application Aug 01 16:37:00.028051 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 339/685] 10.222.0.20 () {56 vars in 1102 bytes} [Mon Aug 1 16:36:59 2022] GET /identity/v3/roles/service => generated 85 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 16:37:00.028051 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd82990-0e9a-4149-89ce-818f78a888e1 None admin] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:37:00.130232 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG dbcounter [-] [87057] Writing DB stats keystone:SELECT=100 {{(pid=87057) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:37:00.134095 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65eb9c77-0026-4c10-8204-f41e38b898d4 None admin] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:37:00.145815 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd82990-0e9a-4149-89ce-818f78a888e1 None admin] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:37:00.180572 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0122722b-05d5-489c-b138-413bf92bc8a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:37:00.181290 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0122722b-05d5-489c-b138-413bf92bc8a1 None admin] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:37:00.181580 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0122722b-05d5-489c-b138-413bf92bc8a1 None admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:37:00.181854 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0122722b-05d5-489c-b138-413bf92bc8a1 None admin] PATH_INFO: `/v3/users` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:37:00.182792 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0122722b-05d5-489c-b138-413bf92bc8a1 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=87058) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:37:00.183113 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0122722b-05d5-489c-b138-413bf92bc8a1 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:37:00.184749 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0122722b-05d5-489c-b138-413bf92bc8a1 None admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:37:00.185502 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0494e3b3-af02-4eb8-a775-15d77a0e8a0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:37:00.186395 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0494e3b3-af02-4eb8-a775-15d77a0e8a0f None admin] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:37:00.186881 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0494e3b3-af02-4eb8-a775-15d77a0e8a0f None admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:37:00.187355 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0494e3b3-af02-4eb8-a775-15d77a0e8a0f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:37:00.188383 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0494e3b3-af02-4eb8-a775-15d77a0e8a0f None admin] RBAC: Adding query filter params (role.id=51bb93e277c446b49bb80a32afcd3af7, scope.project.id=7b992d6a65df4bf9b27c13192d57c5f0, user.id=3d9bd8d649154c979be3744bfd016f95) {{(pid=87057) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:37:00.188909 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0494e3b3-af02-4eb8-a775-15d77a0e8a0f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:37:00.190647 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0494e3b3-af02-4eb8-a775-15d77a0e8a0f None admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:37:00.199090 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 340/686] 10.222.0.20 () {56 vars in 1374 bytes} [Mon Aug 1 16:37:00 2022] GET /identity/v3/role_assignments?user.id=3d9bd8d649154c979be3744bfd016f95&scope.project.id=7b992d6a65df4bf9b27c13192d57c5f0&role.id=51bb93e277c446b49bb80a32afcd3af7 => generated 608 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:37:00.200352 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0494e3b3-af02-4eb8-a775-15d77a0e8a0f None admin] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:37:00.214205 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 347/687] 10.222.0.20 () {56 vars in 1133 bytes} [Mon Aug 1 16:36:59 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:37:00.226047 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0122722b-05d5-489c-b138-413bf92bc8a1 None admin] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:37:00.330376 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG dbcounter [-] [87058] Writing DB stats keystone:SELECT=100 {{(pid=87058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:37:00.352947 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0494e3b3-af02-4eb8-a775-15d77a0e8a0f None admin] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:37:00.428690 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0122722b-05d5-489c-b138-413bf92bc8a1 None admin] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:37:00.437581 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-476456d7-003a-4110-86d1-58a563ad396e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:37:00.438590 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476456d7-003a-4110-86d1-58a563ad396e None admin] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:37:00.439105 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476456d7-003a-4110-86d1-58a563ad396e None admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:37:00.439598 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476456d7-003a-4110-86d1-58a563ad396e None admin] PATH_INFO: `/v3/roles` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:37:00.440612 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-476456d7-003a-4110-86d1-58a563ad396e None admin] RBAC: Adding query filter params (name=service) {{(pid=87057) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:37:00.441132 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-476456d7-003a-4110-86d1-58a563ad396e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:37:00.442855 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-476456d7-003a-4110-86d1-58a563ad396e None admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:37:00.456286 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 341/688] 10.222.0.20 () {56 vars in 1103 bytes} [Mon Aug 1 16:37:00 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:37:00.463295 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-476456d7-003a-4110-86d1-58a563ad396e None admin] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:37:00.476628 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d221bd8c-6d8a-40f1-90d4-b002492f1299 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:37:00.477641 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d221bd8c-6d8a-40f1-90d4-b002492f1299 None admin] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:37:00.477929 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d221bd8c-6d8a-40f1-90d4-b002492f1299 None admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:37:00.478236 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d221bd8c-6d8a-40f1-90d4-b002492f1299 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:37:00.487651 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG dbcounter [-] [87057] Writing DB stats keystone:SELECT=100 {{(pid=87057) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:37:00.566821 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d221bd8c-6d8a-40f1-90d4-b002492f1299 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:37:00.569322 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d221bd8c-6d8a-40f1-90d4-b002492f1299 None admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:37:00.587901 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-476456d7-003a-4110-86d1-58a563ad396e None admin] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:37:00.629851 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa5fa2e1-1be9-4497-9139-bc97495c1c4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:37:00.631077 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5fa2e1-1be9-4497-9139-bc97495c1c4d None admin] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:37:00.631616 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5fa2e1-1be9-4497-9139-bc97495c1c4d None admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:37:00.632095 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5fa2e1-1be9-4497-9139-bc97495c1c4d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:37:00.637057 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG dbcounter [-] [87058] Writing DB stats keystone:SELECT=100 {{(pid=87058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:37:00.644555 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa5fa2e1-1be9-4497-9139-bc97495c1c4d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:37:00.646373 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa5fa2e1-1be9-4497-9139-bc97495c1c4d None admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:37:00.663603 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application [None req-fa5fa2e1-1be9-4497-9139-bc97495c1c4d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 16:37:00.663603 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 16:37:00.663603 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 16:37:00.663603 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 16:37:00.663603 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 16:37:00.663603 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 16:37:00.663603 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 16:37:00.663603 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 16:37:00.663603 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 16:37:00.663603 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 16:37:00.663603 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 16:37:00.663603 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 16:37:00.663603 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 16:37:00.663603 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 16:37:00.663603 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 16:37:00.664559 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 16:37:00.664559 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 16:37:00.664559 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 16:37:00.664559 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 16:37:00.664559 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 16:37:00.664559 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 16:37:00.664559 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 16:37:00.664559 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 16:37:00.664559 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application with Lock( Aug 01 16:37:00.664559 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 16:37:00.664559 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self._enter() Aug 01 16:37:00.664559 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 16:37:00.664559 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 16:37:00.664559 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 16:37:00.664559 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self.creator() Aug 01 16:37:00.664559 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 16:37:00.664559 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application created_value = creator( Aug 01 16:37:00.665487 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 16:37:00.665487 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 16:37:00.665487 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 16:37:00.665487 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application Aug 01 16:37:00.667824 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 342/689] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Aug 1 16:37:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 16:37:00.673494 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa5fa2e1-1be9-4497-9139-bc97495c1c4d None admin] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:37:00.691447 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 348/690] 10.222.0.20 () {58 vars in 1282 bytes} [Mon Aug 1 16:37:00 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 466 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 01 16:37:00.697382 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d221bd8c-6d8a-40f1-90d4-b002492f1299 None admin] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:37:00.765052 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG dbcounter [-] [87057] Writing DB stats keystone:SELECT=100 {{(pid=87057) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:37:00.802246 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d221bd8c-6d8a-40f1-90d4-b002492f1299 None admin] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:37:00.824237 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa5fa2e1-1be9-4497-9139-bc97495c1c4d None admin] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:37:00.843325 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81891c85-a5f9-4edb-83ff-1f05dbed9856 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:37:00.843910 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81891c85-a5f9-4edb-83ff-1f05dbed9856 None admin] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:37:00.844195 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81891c85-a5f9-4edb-83ff-1f05dbed9856 None admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:37:00.844473 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81891c85-a5f9-4edb-83ff-1f05dbed9856 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:37:00.865181 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81891c85-a5f9-4edb-83ff-1f05dbed9856 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:37:00.866734 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81891c85-a5f9-4edb-83ff-1f05dbed9856 None admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:37:00.893117 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7137705c-757a-4389-9ba5-8d8fbe928084 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:37:00.893759 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7137705c-757a-4389-9ba5-8d8fbe928084 None admin] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:37:00.894125 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7137705c-757a-4389-9ba5-8d8fbe928084 None admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:37:00.894404 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7137705c-757a-4389-9ba5-8d8fbe928084 None admin] PATH_INFO: `/v3/domains` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:37:00.895136 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7137705c-757a-4389-9ba5-8d8fbe928084 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87057) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:37:00.895443 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7137705c-757a-4389-9ba5-8d8fbe928084 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:37:00.900087 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application [None req-81891c85-a5f9-4edb-83ff-1f05dbed9856 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 16:37:00.900087 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 16:37:00.900087 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 16:37:00.900087 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 16:37:00.900087 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 16:37:00.900087 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 16:37:00.900087 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 16:37:00.900087 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 16:37:00.900087 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 16:37:00.900087 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 16:37:00.900087 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 16:37:00.900087 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 16:37:00.900087 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 16:37:00.900087 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 16:37:00.900087 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 16:37:00.901124 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 16:37:00.901124 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 16:37:00.901124 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 16:37:00.901124 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 16:37:00.901124 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 16:37:00.901124 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 16:37:00.901124 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 16:37:00.901124 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 16:37:00.901124 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application with Lock( Aug 01 16:37:00.901124 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 16:37:00.901124 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self._enter() Aug 01 16:37:00.901124 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 16:37:00.901124 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 16:37:00.901124 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 16:37:00.901124 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self.creator() Aug 01 16:37:00.901124 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 16:37:00.901124 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application created_value = creator( Aug 01 16:37:00.902021 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 16:37:00.902021 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 16:37:00.902021 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 16:37:00.902021 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application Aug 01 16:37:00.902291 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7137705c-757a-4389-9ba5-8d8fbe928084 None admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:37:00.907777 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 349/691] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Aug 1 16:37:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 01 16:37:00.909979 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81891c85-a5f9-4edb-83ff-1f05dbed9856 None admin] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:37:00.929844 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 343/692] 10.222.0.20 () {56 vars in 1109 bytes} [Mon Aug 1 16:37:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:37:00.934291 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG dbcounter [-] [87058] Writing DB stats keystone:SELECT=100 {{(pid=87058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:37:00.941246 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7137705c-757a-4389-9ba5-8d8fbe928084 None admin] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:37:01.099303 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81891c85-a5f9-4edb-83ff-1f05dbed9856 None admin] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:37:01.108856 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7137705c-757a-4389-9ba5-8d8fbe928084 None admin] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:37:01.136747 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d513977f-b319-4e34-8644-e895227e6e59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:37:01.137411 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d513977f-b319-4e34-8644-e895227e6e59 None admin] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:37:01.137647 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d513977f-b319-4e34-8644-e895227e6e59 None admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:37:01.137919 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d513977f-b319-4e34-8644-e895227e6e59 None admin] PATH_INFO: `/v3/domains` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:37:01.138641 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d513977f-b319-4e34-8644-e895227e6e59 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87058) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:37:01.138941 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d513977f-b319-4e34-8644-e895227e6e59 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:37:01.140305 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d513977f-b319-4e34-8644-e895227e6e59 None admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:37:01.148335 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG dbcounter [-] [87057] Writing DB stats keystone:SELECT=100 {{(pid=87057) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:37:01.154862 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 350/693] 10.222.0.20 () {56 vars in 1109 bytes} [Mon Aug 1 16:37:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 16:37:01.155893 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dce3e27-057e-48f0-8b74-ed73166edcbf None None] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:37:01.156174 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dce3e27-057e-48f0-8b74-ed73166edcbf None None] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:37:01.158317 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dce3e27-057e-48f0-8b74-ed73166edcbf None None] PATH_INFO: `/` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:37:01.158420 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad12c833-854b-4e67-b649-4210ae3c61a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:37:01.158420 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad12c833-854b-4e67-b649-4210ae3c61a2 None admin] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:37:01.158420 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad12c833-854b-4e67-b649-4210ae3c61a2 None admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:37:01.158420 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad12c833-854b-4e67-b649-4210ae3c61a2 None admin] PATH_INFO: `/v3/users/swift` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:37:01.161719 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 351/694] 10.222.0.20 () {54 vars in 926 bytes} [Mon Aug 1 16:37:01 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 16:37:01.163101 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dce3e27-057e-48f0-8b74-ed73166edcbf None None] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:37:01.169048 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad12c833-854b-4e67-b649-4210ae3c61a2 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:37:01.170790 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad12c833-854b-4e67-b649-4210ae3c61a2 None admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:37:01.189357 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application [None req-ad12c833-854b-4e67-b649-4210ae3c61a2 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 01 16:37:01.189357 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 16:37:01.189357 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 16:37:01.189357 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 16:37:01.189357 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 16:37:01.189357 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 16:37:01.189357 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 16:37:01.189357 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 16:37:01.189357 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 16:37:01.189357 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 16:37:01.189357 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 16:37:01.189357 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 16:37:01.189357 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 01 16:37:01.189357 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 01 16:37:01.189357 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 01 16:37:01.189357 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 01 16:37:01.190408 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 16:37:01.190408 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 16:37:01.190408 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 01 16:37:01.190408 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 16:37:01.190408 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 01 16:37:01.190408 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 16:37:01.190408 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 16:37:01.190408 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 16:37:01.190408 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 16:37:01.190408 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 16:37:01.190408 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 16:37:01.190408 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application with Lock( Aug 01 16:37:01.190408 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 16:37:01.190408 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self._enter() Aug 01 16:37:01.190408 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 16:37:01.190408 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 16:37:01.190408 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 16:37:01.190408 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self.creator() Aug 01 16:37:01.191362 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 16:37:01.191362 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application created_value = creator( Aug 01 16:37:01.191362 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 01 16:37:01.191362 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 01 16:37:01.191362 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 01 16:37:01.191362 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 01 16:37:01.191362 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 01 16:37:01.191362 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 01 16:37:01.191362 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 01 16:37:01.191362 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application Aug 01 16:37:01.191912 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 344/695] 10.222.0.20 () {56 vars in 1096 bytes} [Mon Aug 1 16:37:00 2022] GET /identity/v3/users/swift => generated 83 bytes in 252 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 16:37:01.192965 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de56cfe-443d-451f-a518-70f851f75f92 None None] REQUEST_METHOD: `POST` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:37:01.193251 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de56cfe-443d-451f-a518-70f851f75f92 None None] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:37:01.193808 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de56cfe-443d-451f-a518-70f851f75f92 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:37:01.211924 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.auth.core [None req-9de56cfe-443d-451f-a518-70f851f75f92 None None] MFA Rules not processed for user `1007f8e379b74e169b98fb7c3c0ba5b4`. Rule list: `[]` (Enabled: `True`). {{(pid=87057) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:37:01.213103 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.fernet_utils [None req-9de56cfe-443d-451f-a518-70f851f75f92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87057) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:37:01.239524 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG dbcounter [-] [87058] Writing DB stats keystone:SELECT=100 {{(pid=87058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:37:01.279007 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dce3e27-057e-48f0-8b74-ed73166edcbf None None] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:37:01.285137 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 345/696] 10.222.0.20 () {58 vars in 1023 bytes} [Mon Aug 1 16:37:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 16:37:01.285980 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de56cfe-443d-451f-a518-70f851f75f92 None None] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:37:01.321112 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e400449-4018-4fb1-bfe5-0bc5a08ad606 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:37:01.322159 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e400449-4018-4fb1-bfe5-0bc5a08ad606 None admin] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:37:01.322956 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e400449-4018-4fb1-bfe5-0bc5a08ad606 None admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:37:01.323438 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e400449-4018-4fb1-bfe5-0bc5a08ad606 None admin] PATH_INFO: `/v3/projects/service` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:37:01.332114 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e400449-4018-4fb1-bfe5-0bc5a08ad606 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:37:01.334845 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e400449-4018-4fb1-bfe5-0bc5a08ad606 None admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:37:01.353074 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application [None req-2e400449-4018-4fb1-bfe5-0bc5a08ad606 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 16:37:01.353074 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 16:37:01.353074 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 16:37:01.353074 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 16:37:01.353074 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 16:37:01.353074 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 16:37:01.353074 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 16:37:01.353074 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 16:37:01.353074 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 16:37:01.353074 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 16:37:01.353074 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 16:37:01.353074 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 16:37:01.353074 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 01 16:37:01.353074 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 01 16:37:01.353074 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 01 16:37:01.354387 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 01 16:37:01.354387 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 16:37:01.354387 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 16:37:01.354387 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 16:37:01.354387 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 16:37:01.354387 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 16:37:01.354387 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 16:37:01.354387 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 16:37:01.354387 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application with Lock( Aug 01 16:37:01.354387 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 16:37:01.354387 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self._enter() Aug 01 16:37:01.354387 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 16:37:01.354387 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 16:37:01.354387 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 16:37:01.354387 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self.creator() Aug 01 16:37:01.354387 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 16:37:01.354387 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application created_value = creator( Aug 01 16:37:01.355279 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 01 16:37:01.355279 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 01 16:37:01.355279 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 01 16:37:01.355279 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 01 16:37:01.355279 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 01 16:37:01.355279 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 01 16:37:01.355279 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 16:37:01.355279 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application Aug 01 16:37:01.355279 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 352/697] 10.222.0.20 () {56 vars in 1111 bytes} [Mon Aug 1 16:37:01 2022] GET /identity/v3/projects/service => generated 88 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 16:37:01.355279 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e400449-4018-4fb1-bfe5-0bc5a08ad606 None admin] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:37:01.422809 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de56cfe-443d-451f-a518-70f851f75f92 None None] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:37:01.446418 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG dbcounter [-] [87057] Writing DB stats keystone:SELECT=100 {{(pid=87057) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:37:01.462441 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a7eb14d-4ef9-4d72-bdf2-a04072806ef3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:37:01.463195 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7eb14d-4ef9-4d72-bdf2-a04072806ef3 None admin] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:37:01.463546 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7eb14d-4ef9-4d72-bdf2-a04072806ef3 None admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:37:01.463921 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7eb14d-4ef9-4d72-bdf2-a04072806ef3 None admin] PATH_INFO: `/v3/users` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:37:01.464891 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a7eb14d-4ef9-4d72-bdf2-a04072806ef3 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=87057) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:37:01.465304 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a7eb14d-4ef9-4d72-bdf2-a04072806ef3 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:37:01.467006 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a7eb14d-4ef9-4d72-bdf2-a04072806ef3 None admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:37:01.490676 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 346/698] 10.222.0.20 () {56 vars in 1135 bytes} [Mon Aug 1 16:37:01 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:37:01.491853 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a7eb14d-4ef9-4d72-bdf2-a04072806ef3 None admin] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:37:01.495146 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e400449-4018-4fb1-bfe5-0bc5a08ad606 None admin] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:37:01.525250 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG dbcounter [-] [87058] Writing DB stats keystone:SELECT=100 {{(pid=87058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:37:01.535007 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-769e7aba-84b0-48ed-8907-741a05dc498b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:37:01.535786 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769e7aba-84b0-48ed-8907-741a05dc498b None admin] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:37:01.537011 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769e7aba-84b0-48ed-8907-741a05dc498b None admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:37:01.537011 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769e7aba-84b0-48ed-8907-741a05dc498b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:37:01.549106 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-769e7aba-84b0-48ed-8907-741a05dc498b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:37:01.550822 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-769e7aba-84b0-48ed-8907-741a05dc498b None admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:37:01.567475 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application [None req-769e7aba-84b0-48ed-8907-741a05dc498b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 16:37:01.567475 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 16:37:01.567475 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 16:37:01.567475 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 16:37:01.567475 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 16:37:01.567475 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 16:37:01.567475 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 16:37:01.567475 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 16:37:01.567475 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 16:37:01.567475 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 16:37:01.567475 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 16:37:01.567475 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 16:37:01.567475 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 16:37:01.567475 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 16:37:01.567475 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 16:37:01.568376 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 16:37:01.568376 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 16:37:01.568376 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 16:37:01.568376 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 16:37:01.568376 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 16:37:01.568376 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 16:37:01.568376 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 16:37:01.568376 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 16:37:01.568376 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application with Lock( Aug 01 16:37:01.568376 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 16:37:01.568376 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self._enter() Aug 01 16:37:01.568376 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 16:37:01.568376 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 16:37:01.568376 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 16:37:01.568376 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self.creator() Aug 01 16:37:01.568376 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 16:37:01.568376 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application created_value = creator( Aug 01 16:37:01.569599 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 16:37:01.569599 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 16:37:01.569599 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 16:37:01.569599 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application Aug 01 16:37:01.573758 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 353/699] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Aug 1 16:37:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 16:37:01.574681 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-769e7aba-84b0-48ed-8907-741a05dc498b None admin] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:37:01.619157 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a7eb14d-4ef9-4d72-bdf2-a04072806ef3 None admin] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:37:01.664948 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a8e03a0-203c-4f7f-989e-4cf31203ae5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:37:01.666060 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8e03a0-203c-4f7f-989e-4cf31203ae5e None admin] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:37:01.666891 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8e03a0-203c-4f7f-989e-4cf31203ae5e None admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:37:01.667651 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8e03a0-203c-4f7f-989e-4cf31203ae5e None admin] PATH_INFO: `/v3/projects` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:37:01.668620 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a8e03a0-203c-4f7f-989e-4cf31203ae5e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87057) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:37:01.669238 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a8e03a0-203c-4f7f-989e-4cf31203ae5e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:37:01.671166 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a8e03a0-203c-4f7f-989e-4cf31203ae5e None admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:37:01.685618 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-769e7aba-84b0-48ed-8907-741a05dc498b None admin] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:37:01.686735 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 347/700] 10.222.0.20 () {56 vars in 1148 bytes} [Mon Aug 1 16:37:01 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:37:01.687969 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a8e03a0-203c-4f7f-989e-4cf31203ae5e None admin] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:37:01.732687 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e3fda9-3134-4f0c-a797-325b3e02aae7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:37:01.733633 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e3fda9-3134-4f0c-a797-325b3e02aae7 None admin] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:37:01.734147 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e3fda9-3134-4f0c-a797-325b3e02aae7 None admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:37:01.734612 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e3fda9-3134-4f0c-a797-325b3e02aae7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:37:01.736794 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG dbcounter [-] [87057] Writing DB stats keystone:SELECT=100 {{(pid=87057) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:37:01.790957 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG dbcounter [-] [87058] Writing DB stats keystone:SELECT=100 {{(pid=87058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:37:01.813722 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8e3fda9-3134-4f0c-a797-325b3e02aae7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:37:01.815512 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8e3fda9-3134-4f0c-a797-325b3e02aae7 None admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:37:01.818267 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a8e03a0-203c-4f7f-989e-4cf31203ae5e None admin] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:37:01.866001 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8526929d-a9a0-4158-a356-01b729e56e57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:37:01.866734 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8526929d-a9a0-4158-a356-01b729e56e57 None admin] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:37:01.867016 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8526929d-a9a0-4158-a356-01b729e56e57 None admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:37:01.867292 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8526929d-a9a0-4158-a356-01b729e56e57 None admin] PATH_INFO: `/v3/domains` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:37:01.867990 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8526929d-a9a0-4158-a356-01b729e56e57 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87057) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:37:01.868298 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8526929d-a9a0-4158-a356-01b729e56e57 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:37:01.869754 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8526929d-a9a0-4158-a356-01b729e56e57 None admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:37:01.881605 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 348/701] 10.222.0.20 () {56 vars in 1109 bytes} [Mon Aug 1 16:37:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:37:01.882586 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8526929d-a9a0-4158-a356-01b729e56e57 None admin] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:37:01.941081 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 354/702] 10.222.0.20 () {58 vars in 1282 bytes} [Mon Aug 1 16:37:01 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 367 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 01 16:37:01.942854 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e01e47-0032-4370-b61d-432c6b613eb9 None None] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:37:01.943297 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e01e47-0032-4370-b61d-432c6b613eb9 None None] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:37:01.944182 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e01e47-0032-4370-b61d-432c6b613eb9 None None] PATH_INFO: `/` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:37:01.949481 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 355/703] 10.222.0.20 () {54 vars in 926 bytes} [Mon Aug 1 16:37:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 16:37:01.949481 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e01e47-0032-4370-b61d-432c6b613eb9 None None] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:37:01.993772 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG dbcounter [-] [87057] Writing DB stats keystone:SELECT=100 {{(pid=87057) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:37:01.998086 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8526929d-a9a0-4158-a356-01b729e56e57 None admin] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:37:02.030506 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG dbcounter [-] [87058] Writing DB stats keystone:SELECT=100 {{(pid=87058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:37:02.032816 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f7bd360-1be2-488d-87a2-cf475fd77810 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:37:02.033813 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7bd360-1be2-488d-87a2-cf475fd77810 None admin] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:37:02.034132 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7bd360-1be2-488d-87a2-cf475fd77810 None admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:37:02.034458 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7bd360-1be2-488d-87a2-cf475fd77810 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:37:02.035597 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f7bd360-1be2-488d-87a2-cf475fd77810 None admin] RBAC: Adding query filter params (role.id=440bb943dd5b46fea038b903b6eaaf81, scope.project.id=7b992d6a65df4bf9b27c13192d57c5f0, user.id=6533106d03eb4ca185a8839336d2aff9) {{(pid=87057) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:37:02.035984 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f7bd360-1be2-488d-87a2-cf475fd77810 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:37:02.037475 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f7bd360-1be2-488d-87a2-cf475fd77810 None admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:37:02.043301 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 349/704] 10.222.0.20 () {56 vars in 1374 bytes} [Mon Aug 1 16:37:01 2022] GET /identity/v3/role_assignments?user.id=6533106d03eb4ca185a8839336d2aff9&scope.project.id=7b992d6a65df4bf9b27c13192d57c5f0&role.id=440bb943dd5b46fea038b903b6eaaf81 => generated 259 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:37:02.043896 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f7bd360-1be2-488d-87a2-cf475fd77810 None admin] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:37:02.066732 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e01e47-0032-4370-b61d-432c6b613eb9 None None] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:37:02.121752 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b7def76-f1ec-4e5c-b36b-ab4a0c6a393f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:37:02.122823 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7def76-f1ec-4e5c-b36b-ab4a0c6a393f None admin] REQUEST_METHOD: `POST` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:37:02.123633 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7def76-f1ec-4e5c-b36b-ab4a0c6a393f None admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:37:02.124207 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7def76-f1ec-4e5c-b36b-ab4a0c6a393f None admin] PATH_INFO: `/v3/users` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:37:02.125180 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b7def76-f1ec-4e5c-b36b-ab4a0c6a393f None admin] RBAC: Authorizing `identity:create_user()` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:37:02.126839 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b7def76-f1ec-4e5c-b36b-ab4a0c6a393f None admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:37:02.168471 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f7bd360-1be2-488d-87a2-cf475fd77810 None admin] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:37:02.208851 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d65e5ca-31f2-45ef-ac13-682e5ddb77eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:37:02.210361 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d65e5ca-31f2-45ef-ac13-682e5ddb77eb None admin] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:37:02.210361 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d65e5ca-31f2-45ef-ac13-682e5ddb77eb None admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:37:02.210618 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d65e5ca-31f2-45ef-ac13-682e5ddb77eb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:37:02.223315 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 356/705] 10.222.0.20 () {60 vars in 1132 bytes} [Mon Aug 1 16:37:01 2022] POST /identity/v3/users => generated 254 bytes in 273 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 16:37:02.223832 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d86422c-111e-42d3-aaa8-516d926d6ff8 None None] REQUEST_METHOD: `POST` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:37:02.224844 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d86422c-111e-42d3-aaa8-516d926d6ff8 None None] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:37:02.224844 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d86422c-111e-42d3-aaa8-516d926d6ff8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:37:02.230166 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d65e5ca-31f2-45ef-ac13-682e5ddb77eb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:37:02.231666 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d65e5ca-31f2-45ef-ac13-682e5ddb77eb None admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:37:02.259657 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application [None req-2d65e5ca-31f2-45ef-ac13-682e5ddb77eb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 16:37:02.259657 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 16:37:02.259657 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 16:37:02.259657 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 16:37:02.259657 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 16:37:02.259657 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 16:37:02.259657 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 16:37:02.259657 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 16:37:02.259657 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 16:37:02.259657 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 16:37:02.259657 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 16:37:02.259657 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 16:37:02.259657 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 16:37:02.259657 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 16:37:02.259657 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 16:37:02.260709 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 16:37:02.260709 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 16:37:02.260709 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 16:37:02.260709 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 16:37:02.260709 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 16:37:02.260709 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 16:37:02.260709 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 16:37:02.260709 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 16:37:02.260709 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application with Lock( Aug 01 16:37:02.260709 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 16:37:02.260709 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self._enter() Aug 01 16:37:02.260709 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 16:37:02.260709 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 16:37:02.260709 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 16:37:02.260709 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self.creator() Aug 01 16:37:02.260709 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 16:37:02.260709 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application created_value = creator( Aug 01 16:37:02.262217 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 16:37:02.262217 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 16:37:02.262217 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 16:37:02.262217 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application Aug 01 16:37:02.262217 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 350/706] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Aug 1 16:37:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 16:37:02.267091 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.auth.core [None req-8d86422c-111e-42d3-aaa8-516d926d6ff8 None None] MFA Rules not processed for user `1007f8e379b74e169b98fb7c3c0ba5b4`. Rule list: `[]` (Enabled: `True`). {{(pid=87058) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:37:02.268311 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.fernet_utils [None req-8d86422c-111e-42d3-aaa8-516d926d6ff8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:37:02.270209 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d65e5ca-31f2-45ef-ac13-682e5ddb77eb None admin] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:37:02.306479 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG dbcounter [-] [87057] Writing DB stats keystone:SELECT=100 {{(pid=87057) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:37:02.405460 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 357/707] 10.222.0.20 () {58 vars in 1023 bytes} [Mon Aug 1 16:37:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 16:37:02.434678 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d86422c-111e-42d3-aaa8-516d926d6ff8 None None] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:37:02.480192 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d65e5ca-31f2-45ef-ac13-682e5ddb77eb None admin] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:37:02.517179 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb099d0e-6655-4223-957b-ce31abde8c47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:37:02.517899 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb099d0e-6655-4223-957b-ce31abde8c47 None admin] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:37:02.518318 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb099d0e-6655-4223-957b-ce31abde8c47 None admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:37:02.518661 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb099d0e-6655-4223-957b-ce31abde8c47 None admin] PATH_INFO: `/v3/domains` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:37:02.519288 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG dbcounter [-] [87058] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:37:02.519455 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb099d0e-6655-4223-957b-ce31abde8c47 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87057) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:37:02.519874 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb099d0e-6655-4223-957b-ce31abde8c47 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:37:02.522803 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb099d0e-6655-4223-957b-ce31abde8c47 None admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:37:02.539260 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 351/708] 10.222.0.20 () {56 vars in 1109 bytes} [Mon Aug 1 16:37:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:37:02.543830 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb099d0e-6655-4223-957b-ce31abde8c47 None admin] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:37:02.569947 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d86422c-111e-42d3-aaa8-516d926d6ff8 None None] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:37:02.608366 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac57c9e-486a-4cff-8541-9847c56f387d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:37:02.609406 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac57c9e-486a-4cff-8541-9847c56f387d None admin] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:37:02.609891 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac57c9e-486a-4cff-8541-9847c56f387d None admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:37:02.610403 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac57c9e-486a-4cff-8541-9847c56f387d None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:37:02.611232 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ac57c9e-486a-4cff-8541-9847c56f387d None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:37:02.613157 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ac57c9e-486a-4cff-8541-9847c56f387d None admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:37:02.625936 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application [None req-2ac57c9e-486a-4cff-8541-9847c56f387d None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 01 16:37:02.625936 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 16:37:02.625936 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 01 16:37:02.625936 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 01 16:37:02.625936 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 01 16:37:02.625936 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 01 16:37:02.625936 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 01 16:37:02.625936 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 01 16:37:02.625936 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 01 16:37:02.625936 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application Aug 01 16:37:02.625936 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 01 16:37:02.625936 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application Aug 01 16:37:02.625936 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 16:37:02.625936 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 16:37:02.625936 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 16:37:02.625936 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 16:37:02.627007 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 16:37:02.627007 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 16:37:02.627007 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 16:37:02.627007 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 16:37:02.627007 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 16:37:02.627007 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 16:37:02.627007 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 16:37:02.627007 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 01 16:37:02.627007 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 01 16:37:02.627007 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 01 16:37:02.627007 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 01 16:37:02.627007 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 16:37:02.627007 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 16:37:02.627007 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 16:37:02.627007 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 16:37:02.627007 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 16:37:02.627867 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 16:37:02.627867 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 16:37:02.627867 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application with Lock( Aug 01 16:37:02.627867 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 16:37:02.627867 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self._enter() Aug 01 16:37:02.627867 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 16:37:02.627867 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 16:37:02.627867 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 16:37:02.627867 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self.creator() Aug 01 16:37:02.627867 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 16:37:02.627867 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application created_value = creator( Aug 01 16:37:02.627867 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 01 16:37:02.627867 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 01 16:37:02.627867 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 01 16:37:02.627867 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application Aug 01 16:37:02.630376 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG dbcounter [-] [87057] Writing DB stats keystone:SELECT=100 {{(pid=87057) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:37:02.631659 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 358/709] 10.222.0.20 () {56 vars in 1129 bytes} [Mon Aug 1 16:37:02 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 16:37:02.639799 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac57c9e-486a-4cff-8541-9847c56f387d None admin] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:37:02.668169 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb099d0e-6655-4223-957b-ce31abde8c47 None admin] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:37:02.716365 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-421fdc2a-5376-409c-904b-46eb6437e0d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:37:02.717521 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421fdc2a-5376-409c-904b-46eb6437e0d0 None admin] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:37:02.718371 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421fdc2a-5376-409c-904b-46eb6437e0d0 None admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:37:02.718978 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421fdc2a-5376-409c-904b-46eb6437e0d0 None admin] PATH_INFO: `/v3/projects/service` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:37:02.728747 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-421fdc2a-5376-409c-904b-46eb6437e0d0 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:37:02.730559 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-421fdc2a-5376-409c-904b-46eb6437e0d0 None admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:37:02.744068 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application [None req-421fdc2a-5376-409c-904b-46eb6437e0d0 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 16:37:02.744068 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 16:37:02.744068 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 16:37:02.744068 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 16:37:02.744068 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 16:37:02.744068 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 16:37:02.744068 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 16:37:02.744068 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 16:37:02.744068 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 16:37:02.744068 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 16:37:02.744068 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 16:37:02.744068 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 16:37:02.744068 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 01 16:37:02.744068 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 01 16:37:02.744068 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 01 16:37:02.745027 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 01 16:37:02.745027 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 16:37:02.745027 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 16:37:02.745027 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 16:37:02.745027 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 16:37:02.745027 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 16:37:02.745027 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 16:37:02.745027 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 16:37:02.745027 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application with Lock( Aug 01 16:37:02.745027 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 16:37:02.745027 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self._enter() Aug 01 16:37:02.745027 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 16:37:02.745027 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 16:37:02.745027 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 16:37:02.745027 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self.creator() Aug 01 16:37:02.745027 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 16:37:02.745027 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application created_value = creator( Aug 01 16:37:02.745915 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 01 16:37:02.745915 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 01 16:37:02.745915 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 01 16:37:02.745915 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 01 16:37:02.745915 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 01 16:37:02.745915 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 01 16:37:02.745915 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 16:37:02.745915 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application Aug 01 16:37:02.752755 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 352/710] 10.222.0.20 () {56 vars in 1111 bytes} [Mon Aug 1 16:37:02 2022] GET /identity/v3/projects/service => generated 88 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 16:37:02.754300 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-421fdc2a-5376-409c-904b-46eb6437e0d0 None admin] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:37:02.773479 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac57c9e-486a-4cff-8541-9847c56f387d None admin] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:37:02.806150 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG dbcounter [-] [87058] Writing DB stats keystone:SELECT=100 {{(pid=87058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:37:02.824164 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c13281-ae40-4d27-9119-0d32d4575a81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:37:02.825216 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c13281-ae40-4d27-9119-0d32d4575a81 None admin] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:37:02.829837 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c13281-ae40-4d27-9119-0d32d4575a81 None admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:37:02.829837 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c13281-ae40-4d27-9119-0d32d4575a81 None admin] PATH_INFO: `/v3/services` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:37:02.829837 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89c13281-ae40-4d27-9119-0d32d4575a81 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=87058) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:37:02.829837 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89c13281-ae40-4d27-9119-0d32d4575a81 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:37:02.829837 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89c13281-ae40-4d27-9119-0d32d4575a81 None admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:37:02.836330 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 359/711] 10.222.0.20 () {56 vars in 1124 bytes} [Mon Aug 1 16:37:02 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:37:02.837877 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b919d93a-2135-422b-9f6d-1141d5e9da23 None None] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:37:02.842252 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b919d93a-2135-422b-9f6d-1141d5e9da23 None None] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:37:02.842720 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b919d93a-2135-422b-9f6d-1141d5e9da23 None None] PATH_INFO: `/` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:37:02.845001 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 360/712] 10.222.0.20 () {54 vars in 926 bytes} [Mon Aug 1 16:37:02 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 16:37:02.851157 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b919d93a-2135-422b-9f6d-1141d5e9da23 None None] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:37:02.877600 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-421fdc2a-5376-409c-904b-46eb6437e0d0 None admin] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:37:02.919024 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-873735b6-964a-4ed8-b1a4-b0f99dbe67ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:37:02.919968 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873735b6-964a-4ed8-b1a4-b0f99dbe67ab None admin] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:37:02.920452 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873735b6-964a-4ed8-b1a4-b0f99dbe67ab None admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:37:02.920989 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873735b6-964a-4ed8-b1a4-b0f99dbe67ab None admin] PATH_INFO: `/v3/projects` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:37:02.921941 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-873735b6-964a-4ed8-b1a4-b0f99dbe67ab None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87057) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:37:02.922490 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-873735b6-964a-4ed8-b1a4-b0f99dbe67ab None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:37:02.924345 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-873735b6-964a-4ed8-b1a4-b0f99dbe67ab None admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:37:02.926478 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG dbcounter [-] [87057] Writing DB stats keystone:SELECT=100 {{(pid=87057) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:37:02.939439 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 353/713] 10.222.0.20 () {56 vars in 1148 bytes} [Mon Aug 1 16:37:02 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:37:02.941077 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a017f495-b29a-4858-95e8-61306f62662d None None] REQUEST_METHOD: `POST` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:37:02.941570 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a017f495-b29a-4858-95e8-61306f62662d None None] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:37:02.942096 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a017f495-b29a-4858-95e8-61306f62662d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:37:02.963418 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.auth.core [None req-a017f495-b29a-4858-95e8-61306f62662d None None] MFA Rules not processed for user `1007f8e379b74e169b98fb7c3c0ba5b4`. Rule list: `[]` (Enabled: `True`). {{(pid=87057) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:37:02.965001 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.fernet_utils [None req-a017f495-b29a-4858-95e8-61306f62662d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87057) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:37:02.992676 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b919d93a-2135-422b-9f6d-1141d5e9da23 None None] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:37:03.038723 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c72026a1-0d5f-4e61-9f51-3d5efb0b5313 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:37:03.039349 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72026a1-0d5f-4e61-9f51-3d5efb0b5313 None admin] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:37:03.039631 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72026a1-0d5f-4e61-9f51-3d5efb0b5313 None admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:37:03.039907 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72026a1-0d5f-4e61-9f51-3d5efb0b5313 None admin] PATH_INFO: `/v3/services` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:37:03.040595 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c72026a1-0d5f-4e61-9f51-3d5efb0b5313 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=87058) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:37:03.040958 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c72026a1-0d5f-4e61-9f51-3d5efb0b5313 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:37:03.042108 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 354/714] 10.222.0.20 () {58 vars in 1023 bytes} [Mon Aug 1 16:37:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 16:37:03.042425 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c72026a1-0d5f-4e61-9f51-3d5efb0b5313 None admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:37:03.043382 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a017f495-b29a-4858-95e8-61306f62662d None None] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:37:03.060704 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 361/715] 10.222.0.20 () {56 vars in 1124 bytes} [Mon Aug 1 16:37:02 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:37:03.092906 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c72026a1-0d5f-4e61-9f51-3d5efb0b5313 None admin] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:37:03.185878 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG dbcounter [-] [87058] Writing DB stats keystone:SELECT=100 {{(pid=87058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:37:03.245388 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c72026a1-0d5f-4e61-9f51-3d5efb0b5313 None admin] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:37:03.279302 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a017f495-b29a-4858-95e8-61306f62662d None None] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:37:03.290598 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-908870ea-5dfa-4e42-a85b-8c0145b3af66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:37:03.291539 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908870ea-5dfa-4e42-a85b-8c0145b3af66 None admin] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:37:03.291961 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908870ea-5dfa-4e42-a85b-8c0145b3af66 None admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:37:03.292776 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908870ea-5dfa-4e42-a85b-8c0145b3af66 None admin] PATH_INFO: `/v3/services/network` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:37:03.293852 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-908870ea-5dfa-4e42-a85b-8c0145b3af66 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:37:03.295688 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-908870ea-5dfa-4e42-a85b-8c0145b3af66 None admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:37:03.308024 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG dbcounter [-] [87057] Writing DB stats keystone:SELECT=100 {{(pid=87057) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:37:03.310184 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application [None req-908870ea-5dfa-4e42-a85b-8c0145b3af66 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Aug 01 16:37:03.310184 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 16:37:03.310184 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 01 16:37:03.310184 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 01 16:37:03.310184 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 01 16:37:03.310184 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 01 16:37:03.310184 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 01 16:37:03.310184 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 01 16:37:03.310184 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 01 16:37:03.310184 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application Aug 01 16:37:03.310184 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 01 16:37:03.310184 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application Aug 01 16:37:03.310184 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 16:37:03.310184 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 16:37:03.310184 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 16:37:03.310184 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 16:37:03.310184 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 16:37:03.311359 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 16:37:03.311359 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 16:37:03.311359 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 16:37:03.311359 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 16:37:03.311359 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 16:37:03.311359 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 16:37:03.311359 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 01 16:37:03.311359 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 01 16:37:03.311359 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 01 16:37:03.311359 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 01 16:37:03.311359 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 16:37:03.311359 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 16:37:03.311359 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 16:37:03.311359 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 16:37:03.311359 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 16:37:03.311359 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 16:37:03.312362 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 16:37:03.312362 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application with Lock( Aug 01 16:37:03.312362 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 16:37:03.312362 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self._enter() Aug 01 16:37:03.312362 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 16:37:03.312362 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 16:37:03.312362 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 16:37:03.312362 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self.creator() Aug 01 16:37:03.312362 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 16:37:03.312362 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application created_value = creator( Aug 01 16:37:03.312362 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 01 16:37:03.312362 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 01 16:37:03.312362 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 01 16:37:03.312362 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application Aug 01 16:37:03.315471 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 362/716] 10.222.0.20 () {56 vars in 1111 bytes} [Mon Aug 1 16:37:03 2022] GET /identity/v3/services/network => generated 88 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 16:37:03.322495 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-908870ea-5dfa-4e42-a85b-8c0145b3af66 None admin] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:37:03.327918 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbbd3e29-78e0-4004-9870-324c7f17255f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:37:03.327918 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbbd3e29-78e0-4004-9870-324c7f17255f None admin] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:37:03.328421 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbbd3e29-78e0-4004-9870-324c7f17255f None admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:37:03.328793 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbbd3e29-78e0-4004-9870-324c7f17255f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:37:03.329769 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbbd3e29-78e0-4004-9870-324c7f17255f None admin] RBAC: Adding query filter params (role.id=51bb93e277c446b49bb80a32afcd3af7, scope.project.id=7b992d6a65df4bf9b27c13192d57c5f0, user.id=9ab4add1faa04faa811ef7a9341cc35c) {{(pid=87057) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:37:03.330217 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbbd3e29-78e0-4004-9870-324c7f17255f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:37:03.331752 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbbd3e29-78e0-4004-9870-324c7f17255f None admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:37:03.338788 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 355/717] 10.222.0.20 () {56 vars in 1374 bytes} [Mon Aug 1 16:37:03 2022] GET /identity/v3/role_assignments?user.id=9ab4add1faa04faa811ef7a9341cc35c&scope.project.id=7b992d6a65df4bf9b27c13192d57c5f0&role.id=51bb93e277c446b49bb80a32afcd3af7 => generated 608 bytes in 296 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:37:03.450714 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-908870ea-5dfa-4e42-a85b-8c0145b3af66 None admin] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:37:03.461669 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG dbcounter [-] [87058] Writing DB stats keystone:SELECT=100 {{(pid=87058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:37:03.476759 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd9f25a-2544-406b-88c7-496fcab88ded None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:37:03.477379 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd9f25a-2544-406b-88c7-496fcab88ded None admin] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:37:03.477654 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd9f25a-2544-406b-88c7-496fcab88ded None admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:37:03.477923 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd9f25a-2544-406b-88c7-496fcab88ded None admin] PATH_INFO: `/v3/services` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:37:03.478625 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fd9f25a-2544-406b-88c7-496fcab88ded None admin] RBAC: Adding query filter params (name=network) {{(pid=87058) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:37:03.478924 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fd9f25a-2544-406b-88c7-496fcab88ded None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:37:03.480290 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fd9f25a-2544-406b-88c7-496fcab88ded None admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:37:03.485148 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 363/718] 10.222.0.20 () {56 vars in 1112 bytes} [Mon Aug 1 16:37:03 2022] GET /identity/v3/services?name=network => generated 124 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:37:03.491425 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbbd3e29-78e0-4004-9870-324c7f17255f None admin] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:37:03.628404 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbbd3e29-78e0-4004-9870-324c7f17255f None admin] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:37:03.666554 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9228a5c-4f2f-463f-b7bf-dd0246aea932 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:37:03.667182 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9228a5c-4f2f-463f-b7bf-dd0246aea932 None admin] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:37:03.667529 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9228a5c-4f2f-463f-b7bf-dd0246aea932 None admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:37:03.667802 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9228a5c-4f2f-463f-b7bf-dd0246aea932 None admin] PATH_INFO: `/v3/services` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:37:03.668482 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9228a5c-4f2f-463f-b7bf-dd0246aea932 None admin] RBAC: Adding query filter params (type=network) {{(pid=87057) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:37:03.668781 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9228a5c-4f2f-463f-b7bf-dd0246aea932 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:37:03.670166 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9228a5c-4f2f-463f-b7bf-dd0246aea932 None admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:37:03.676636 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 356/719] 10.222.0.20 () {56 vars in 1112 bytes} [Mon Aug 1 16:37:03 2022] GET /identity/v3/services?type=network => generated 124 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:37:04.416235 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885a2a1d-978a-4c27-b213-58b5c1933fd2 None None] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:37:04.417262 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885a2a1d-978a-4c27-b213-58b5c1933fd2 None None] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:37:04.417774 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885a2a1d-978a-4c27-b213-58b5c1933fd2 None None] PATH_INFO: `/` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:37:04.419673 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 364/720] 10.222.0.20 () {54 vars in 926 bytes} [Mon Aug 1 16:37:04 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 16:37:04.427796 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7c35b1-b0f6-495b-b8c6-53f4a5e155e4 None None] REQUEST_METHOD: `POST` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:37:04.427796 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7c35b1-b0f6-495b-b8c6-53f4a5e155e4 None None] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:37:04.427796 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7c35b1-b0f6-495b-b8c6-53f4a5e155e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:37:04.527065 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.auth.core [None req-6a7c35b1-b0f6-495b-b8c6-53f4a5e155e4 None None] MFA Rules not processed for user `1007f8e379b74e169b98fb7c3c0ba5b4`. Rule list: `[]` (Enabled: `True`). {{(pid=87057) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:37:04.528288 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.fernet_utils [None req-6a7c35b1-b0f6-495b-b8c6-53f4a5e155e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87057) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:37:04.588604 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG dbcounter [-] [87057] Writing DB stats keystone:SELECT=100 {{(pid=87057) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:37:04.619036 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 357/721] 10.222.0.20 () {58 vars in 1023 bytes} [Mon Aug 1 16:37:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 16:37:04.653361 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-885a2a1d-978a-4c27-b213-58b5c1933fd2 None None] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:37:04.787976 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-885a2a1d-978a-4c27-b213-58b5c1933fd2 None None] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:37:04.830833 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc40a17c-5186-4946-b3ca-2b0cdeb96af5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:37:04.831475 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc40a17c-5186-4946-b3ca-2b0cdeb96af5 None admin] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:37:04.831750 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc40a17c-5186-4946-b3ca-2b0cdeb96af5 None admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:37:04.832019 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc40a17c-5186-4946-b3ca-2b0cdeb96af5 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:37:04.846884 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc40a17c-5186-4946-b3ca-2b0cdeb96af5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 16:37:04.853528 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application [None req-cc40a17c-5186-4946-b3ca-2b0cdeb96af5 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 16:37:04.853528 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 16:37:04.853528 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 16:37:04.853528 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 16:37:04.853528 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 16:37:04.853528 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 16:37:04.853528 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 16:37:04.853528 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 16:37:04.853528 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 16:37:04.853528 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 16:37:04.853528 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 16:37:04.853528 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 16:37:04.853528 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 16:37:04.853528 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 16:37:04.853528 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 16:37:04.853528 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 16:37:04.854487 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 16:37:04.854487 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 16:37:04.854487 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 16:37:04.854487 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 16:37:04.854487 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 16:37:04.854487 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 16:37:04.854487 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 16:37:04.854487 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 16:37:04.854487 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 16:37:04.854487 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 16:37:04.854487 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 16:37:04.854487 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application with Lock( Aug 01 16:37:04.854487 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 16:37:04.854487 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self._enter() Aug 01 16:37:04.854487 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 16:37:04.854487 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 16:37:04.855999 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 16:37:04.855999 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self.creator() Aug 01 16:37:04.855999 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 16:37:04.855999 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application created_value = creator( Aug 01 16:37:04.855999 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 16:37:04.855999 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 16:37:04.855999 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 16:37:04.855999 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 16:37:04.855999 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 16:37:04.855999 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 16:37:04.855999 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 16:37:04.855999 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application Aug 01 16:37:04.855999 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 365/722] 10.222.0.20 () {56 vars in 1096 bytes} [Mon Aug 1 16:37:04 2022] GET /identity/v3/roles/admin => generated 83 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 16:37:04.861825 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582aad9b-ff68-4866-b4aa-e26c71412911 None None] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:37:04.862141 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582aad9b-ff68-4866-b4aa-e26c71412911 None None] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:37:04.862415 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582aad9b-ff68-4866-b4aa-e26c71412911 None None] PATH_INFO: `/` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:37:04.863755 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 358/723] 10.222.0.20 () {54 vars in 926 bytes} [Mon Aug 1 16:37:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 16:37:04.864600 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc40a17c-5186-4946-b3ca-2b0cdeb96af5 None admin] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:37:04.869342 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482dd23a-e4f3-4498-a9fa-9375d532a05c None None] REQUEST_METHOD: `POST` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:37:04.869677 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482dd23a-e4f3-4498-a9fa-9375d532a05c None None] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:37:04.869954 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482dd23a-e4f3-4498-a9fa-9375d532a05c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:37:04.907362 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.auth.core [None req-482dd23a-e4f3-4498-a9fa-9375d532a05c None None] MFA Rules not processed for user `1007f8e379b74e169b98fb7c3c0ba5b4`. Rule list: `[]` (Enabled: `True`). {{(pid=87057) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:37:04.910095 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.fernet_utils [None req-482dd23a-e4f3-4498-a9fa-9375d532a05c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87057) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:37:04.960146 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG dbcounter [-] [87058] Writing DB stats keystone:SELECT=100 {{(pid=87058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:37:05.016795 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc40a17c-5186-4946-b3ca-2b0cdeb96af5 None admin] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:37:05.052621 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 359/724] 10.222.0.20 () {58 vars in 1023 bytes} [Mon Aug 1 16:37:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 01 16:37:05.056520 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f5b8d3f-e7ad-408c-bea4-a890b7fd3a11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:37:05.057136 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5b8d3f-e7ad-408c-bea4-a890b7fd3a11 None admin] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:37:05.057408 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5b8d3f-e7ad-408c-bea4-a890b7fd3a11 None admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:37:05.057676 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5b8d3f-e7ad-408c-bea4-a890b7fd3a11 None admin] PATH_INFO: `/v3/roles` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:37:05.058423 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f5b8d3f-e7ad-408c-bea4-a890b7fd3a11 None admin] RBAC: Adding query filter params (name=admin) {{(pid=87058) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:37:05.058723 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f5b8d3f-e7ad-408c-bea4-a890b7fd3a11 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:37:05.060071 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f5b8d3f-e7ad-408c-bea4-a890b7fd3a11 None admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:37:05.069365 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 366/725] 10.222.0.20 () {56 vars in 1099 bytes} [Mon Aug 1 16:37:04 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:37:05.074869 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-482dd23a-e4f3-4498-a9fa-9375d532a05c None None] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:37:05.107917 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f5b8d3f-e7ad-408c-bea4-a890b7fd3a11 None admin] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:37:05.183459 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-482dd23a-e4f3-4498-a9fa-9375d532a05c None None] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:37:05.204021 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG dbcounter [-] [87057] Writing DB stats keystone:SELECT=100 {{(pid=87057) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:37:05.249990 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc95b1c7-f875-4ffd-ad96-377c5ef4695f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:37:05.250675 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc95b1c7-f875-4ffd-ad96-377c5ef4695f None admin] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:37:05.250952 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc95b1c7-f875-4ffd-ad96-377c5ef4695f None admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:37:05.251237 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc95b1c7-f875-4ffd-ad96-377c5ef4695f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:37:05.273923 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc95b1c7-f875-4ffd-ad96-377c5ef4695f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:37:05.275460 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc95b1c7-f875-4ffd-ad96-377c5ef4695f None admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:37:05.293893 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f5b8d3f-e7ad-408c-bea4-a890b7fd3a11 None admin] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:37:05.307258 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application [None req-cc95b1c7-f875-4ffd-ad96-377c5ef4695f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 16:37:05.307258 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 16:37:05.307258 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 16:37:05.307258 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 16:37:05.307258 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 16:37:05.307258 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 16:37:05.307258 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 16:37:05.307258 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 16:37:05.307258 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 16:37:05.307258 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 16:37:05.307258 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 16:37:05.307258 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 16:37:05.307258 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 16:37:05.307258 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 16:37:05.307258 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 16:37:05.308156 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 16:37:05.308156 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 16:37:05.308156 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 16:37:05.308156 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 16:37:05.308156 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 16:37:05.308156 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 16:37:05.308156 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 16:37:05.308156 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 16:37:05.308156 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application with Lock( Aug 01 16:37:05.308156 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 16:37:05.308156 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self._enter() Aug 01 16:37:05.308156 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 16:37:05.308156 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 16:37:05.308156 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 16:37:05.308156 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self.creator() Aug 01 16:37:05.308156 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 16:37:05.308156 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application created_value = creator( Aug 01 16:37:05.309054 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 16:37:05.309054 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 16:37:05.309054 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 16:37:05.309054 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application Aug 01 16:37:05.309054 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 360/726] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Aug 1 16:37:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 16:37:05.319349 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG dbcounter [-] [87058] Writing DB stats keystone:SELECT=100 {{(pid=87058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:37:05.320239 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc95b1c7-f875-4ffd-ad96-377c5ef4695f None admin] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:37:05.368455 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f18afd73-a8ba-4275-a5be-f6d597149043 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:37:05.369101 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18afd73-a8ba-4275-a5be-f6d597149043 None admin] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:37:05.369375 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18afd73-a8ba-4275-a5be-f6d597149043 None admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:37:05.369699 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18afd73-a8ba-4275-a5be-f6d597149043 None admin] PATH_INFO: `/v3/roles/service` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:37:05.390763 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f18afd73-a8ba-4275-a5be-f6d597149043 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 16:37:05.396364 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application [None req-f18afd73-a8ba-4275-a5be-f6d597149043 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 01 16:37:05.396364 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 16:37:05.396364 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 16:37:05.396364 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 16:37:05.396364 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 16:37:05.396364 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 16:37:05.396364 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 16:37:05.396364 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 16:37:05.396364 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 16:37:05.396364 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 16:37:05.396364 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 16:37:05.396364 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 16:37:05.396364 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 16:37:05.396364 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 16:37:05.396364 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 16:37:05.396364 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 16:37:05.398738 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 16:37:05.398738 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 16:37:05.398738 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 16:37:05.398738 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 16:37:05.398738 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 16:37:05.398738 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 16:37:05.398738 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 16:37:05.398738 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 16:37:05.398738 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 16:37:05.398738 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 16:37:05.398738 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 16:37:05.398738 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application with Lock( Aug 01 16:37:05.398738 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 16:37:05.398738 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self._enter() Aug 01 16:37:05.398738 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 16:37:05.398738 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 16:37:05.400756 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 16:37:05.400756 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self.creator() Aug 01 16:37:05.400756 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 16:37:05.400756 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application created_value = creator( Aug 01 16:37:05.400756 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 16:37:05.400756 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 16:37:05.400756 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 16:37:05.400756 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 16:37:05.400756 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 16:37:05.400756 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 16:37:05.400756 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 01 16:37:05.400756 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application Aug 01 16:37:05.402291 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 367/727] 10.222.0.20 () {56 vars in 1102 bytes} [Mon Aug 1 16:37:05 2022] GET /identity/v3/roles/service => generated 85 bytes in 294 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 01 16:37:05.408290 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f18afd73-a8ba-4275-a5be-f6d597149043 None admin] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:37:05.506869 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc95b1c7-f875-4ffd-ad96-377c5ef4695f None admin] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:37:05.538526 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f18afd73-a8ba-4275-a5be-f6d597149043 None admin] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:37:05.560177 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0330a15-35b3-4440-87da-b857a07f798f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:37:05.561199 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0330a15-35b3-4440-87da-b857a07f798f None admin] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:37:05.561757 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0330a15-35b3-4440-87da-b857a07f798f None admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:37:05.562263 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0330a15-35b3-4440-87da-b857a07f798f None admin] PATH_INFO: `/v3/domains` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:37:05.563187 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0330a15-35b3-4440-87da-b857a07f798f None admin] RBAC: Adding query filter params (name=Default) {{(pid=87057) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:37:05.563799 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0330a15-35b3-4440-87da-b857a07f798f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:37:05.565485 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0330a15-35b3-4440-87da-b857a07f798f None admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:37:05.570001 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe52e555-59de-4e59-bf18-6a842dd0a401 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:37:05.570647 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe52e555-59de-4e59-bf18-6a842dd0a401 None admin] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:37:05.570923 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe52e555-59de-4e59-bf18-6a842dd0a401 None admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:37:05.571189 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe52e555-59de-4e59-bf18-6a842dd0a401 None admin] PATH_INFO: `/v3/roles` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:37:05.571915 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe52e555-59de-4e59-bf18-6a842dd0a401 None admin] RBAC: Adding query filter params (name=service) {{(pid=87058) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:37:05.572217 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe52e555-59de-4e59-bf18-6a842dd0a401 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:37:05.573581 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe52e555-59de-4e59-bf18-6a842dd0a401 None admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:37:05.581608 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 361/728] 10.222.0.20 () {56 vars in 1109 bytes} [Mon Aug 1 16:37:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 16:37:05.583217 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f82805-1e6f-45c6-b1c7-6af4ec9bee3b None None] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:37:05.583707 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f82805-1e6f-45c6-b1c7-6af4ec9bee3b None None] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:37:05.584918 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f82805-1e6f-45c6-b1c7-6af4ec9bee3b None None] PATH_INFO: `/` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:37:05.586597 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 368/730] 10.222.0.20 () {56 vars in 1103 bytes} [Mon Aug 1 16:37:05 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:37:05.590076 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe52e555-59de-4e59-bf18-6a842dd0a401 None admin] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:37:05.590304 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 362/730] 10.222.0.20 () {54 vars in 926 bytes} [Mon Aug 1 16:37:05 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 16:37:05.592205 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213cdeb3-a3ba-4f91-9105-1dc9e49ecc82 None None] REQUEST_METHOD: `POST` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:37:05.592489 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213cdeb3-a3ba-4f91-9105-1dc9e49ecc82 None None] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:37:05.592760 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213cdeb3-a3ba-4f91-9105-1dc9e49ecc82 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:37:05.613887 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.auth.core [None req-213cdeb3-a3ba-4f91-9105-1dc9e49ecc82 None None] MFA Rules not processed for user `1007f8e379b74e169b98fb7c3c0ba5b4`. Rule list: `[]` (Enabled: `True`). {{(pid=87057) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:37:05.615512 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.fernet_utils [None req-213cdeb3-a3ba-4f91-9105-1dc9e49ecc82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87057) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:37:05.625750 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG dbcounter [-] [87058] Writing DB stats keystone:SELECT=100 {{(pid=87058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:37:05.658485 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG dbcounter [-] [87057] Writing DB stats keystone:SELECT=100 {{(pid=87057) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:37:05.702455 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 363/731] 10.222.0.20 () {58 vars in 1023 bytes} [Mon Aug 1 16:37:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 16:37:05.706016 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-213cdeb3-a3ba-4f91-9105-1dc9e49ecc82 None None] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:37:05.721107 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe52e555-59de-4e59-bf18-6a842dd0a401 None admin] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:37:05.758437 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3aa6a41-115c-497b-be1a-5e503c864d4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:37:05.759464 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3aa6a41-115c-497b-be1a-5e503c864d4a None admin] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:37:05.759973 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3aa6a41-115c-497b-be1a-5e503c864d4a None admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:37:05.760443 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3aa6a41-115c-497b-be1a-5e503c864d4a None admin] PATH_INFO: `/v3/users/nova` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:37:05.768747 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3aa6a41-115c-497b-be1a-5e503c864d4a None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:37:05.770502 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3aa6a41-115c-497b-be1a-5e503c864d4a None admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:37:05.783595 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application [None req-c3aa6a41-115c-497b-be1a-5e503c864d4a None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 01 16:37:05.783595 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 16:37:05.783595 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 16:37:05.783595 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 16:37:05.783595 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 16:37:05.783595 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 16:37:05.783595 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 16:37:05.783595 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 16:37:05.783595 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 16:37:05.783595 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 16:37:05.783595 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 16:37:05.783595 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 16:37:05.783595 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 01 16:37:05.783595 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 01 16:37:05.783595 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 01 16:37:05.783595 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 01 16:37:05.784606 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 16:37:05.784606 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 16:37:05.784606 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 01 16:37:05.784606 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 16:37:05.784606 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 01 16:37:05.784606 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 16:37:05.784606 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 16:37:05.784606 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 16:37:05.784606 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 16:37:05.784606 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 16:37:05.784606 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 16:37:05.784606 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application with Lock( Aug 01 16:37:05.784606 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 16:37:05.784606 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self._enter() Aug 01 16:37:05.784606 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 16:37:05.784606 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 16:37:05.784606 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 16:37:05.784606 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self.creator() Aug 01 16:37:05.785557 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 16:37:05.785557 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application created_value = creator( Aug 01 16:37:05.785557 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 01 16:37:05.785557 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 01 16:37:05.785557 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 01 16:37:05.785557 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 01 16:37:05.785557 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 01 16:37:05.785557 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 01 16:37:05.785557 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 01 16:37:05.785557 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application Aug 01 16:37:05.788747 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 369/732] 10.222.0.20 () {56 vars in 1093 bytes} [Mon Aug 1 16:37:05 2022] GET /identity/v3/users/nova => generated 82 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 16:37:05.794231 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3aa6a41-115c-497b-be1a-5e503c864d4a None admin] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:37:05.826257 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-213cdeb3-a3ba-4f91-9105-1dc9e49ecc82 None None] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:37:05.863962 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb0f543e-27db-495f-ba40-854a06bcea2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:37:05.864715 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0f543e-27db-495f-ba40-854a06bcea2b None admin] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:37:05.865001 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0f543e-27db-495f-ba40-854a06bcea2b None admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:37:05.865957 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0f543e-27db-495f-ba40-854a06bcea2b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:37:05.877524 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb0f543e-27db-495f-ba40-854a06bcea2b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:37:05.879025 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb0f543e-27db-495f-ba40-854a06bcea2b None admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:37:05.894543 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG dbcounter [-] [87058] Writing DB stats keystone:SELECT=100 {{(pid=87058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:37:05.896872 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application [None req-eb0f543e-27db-495f-ba40-854a06bcea2b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 16:37:05.896872 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 16:37:05.896872 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 16:37:05.896872 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 16:37:05.896872 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 16:37:05.896872 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 16:37:05.896872 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 16:37:05.896872 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 16:37:05.896872 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 16:37:05.896872 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 16:37:05.896872 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 16:37:05.896872 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 16:37:05.896872 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 16:37:05.896872 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 16:37:05.896872 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 16:37:05.897785 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 16:37:05.897785 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 16:37:05.897785 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 16:37:05.897785 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 16:37:05.897785 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 16:37:05.897785 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 16:37:05.897785 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 16:37:05.897785 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 16:37:05.897785 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application with Lock( Aug 01 16:37:05.897785 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 16:37:05.897785 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self._enter() Aug 01 16:37:05.897785 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 16:37:05.897785 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 16:37:05.897785 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 16:37:05.897785 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self.creator() Aug 01 16:37:05.897785 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 16:37:05.897785 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application created_value = creator( Aug 01 16:37:05.899011 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 16:37:05.899011 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 16:37:05.899011 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 16:37:05.899011 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application Aug 01 16:37:05.899011 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 364/733] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Aug 1 16:37:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 16:37:05.899011 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb0f543e-27db-495f-ba40-854a06bcea2b None admin] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:37:05.913993 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3aa6a41-115c-497b-be1a-5e503c864d4a None admin] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:37:05.917518 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG dbcounter [-] [87057] Writing DB stats keystone:SELECT=100 {{(pid=87057) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:37:05.951148 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e086c9-7778-4047-8930-00fc2a305110 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:37:05.951890 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e086c9-7778-4047-8930-00fc2a305110 None admin] REQUEST_METHOD: `POST` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:37:05.952224 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e086c9-7778-4047-8930-00fc2a305110 None admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:37:05.952552 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e086c9-7778-4047-8930-00fc2a305110 None admin] PATH_INFO: `/v3/services` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:37:05.959519 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4e086c9-7778-4047-8930-00fc2a305110 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:37:05.959519 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4e086c9-7778-4047-8930-00fc2a305110 None admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:37:06.018231 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb0f543e-27db-495f-ba40-854a06bcea2b None admin] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:37:06.041456 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 370/734] 10.222.0.20 () {60 vars in 1141 bytes} [Mon Aug 1 16:37:05 2022] POST /identity/v3/services => generated 251 bytes in 252 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 16:37:06.042963 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e086c9-7778-4047-8930-00fc2a305110 None admin] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:37:06.072103 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-932e2b97-0686-4e8b-9f2f-b6028decc035 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:37:06.072905 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932e2b97-0686-4e8b-9f2f-b6028decc035 None admin] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:37:06.073257 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932e2b97-0686-4e8b-9f2f-b6028decc035 None admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:37:06.074060 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932e2b97-0686-4e8b-9f2f-b6028decc035 None admin] PATH_INFO: `/v3/users` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:37:06.074630 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-932e2b97-0686-4e8b-9f2f-b6028decc035 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=87057) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:37:06.075030 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-932e2b97-0686-4e8b-9f2f-b6028decc035 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:37:06.076763 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-932e2b97-0686-4e8b-9f2f-b6028decc035 None admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:37:06.101246 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 365/735] 10.222.0.20 () {56 vars in 1133 bytes} [Mon Aug 1 16:37:05 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:37:06.107423 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-932e2b97-0686-4e8b-9f2f-b6028decc035 None admin] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:37:06.186866 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e086c9-7778-4047-8930-00fc2a305110 None admin] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:37:06.204075 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG dbcounter [-] [87057] Writing DB stats keystone:SELECT=100 {{(pid=87057) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:37:06.225971 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e40c830-138a-4dab-9272-27bdfbc49238 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:37:06.226630 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e40c830-138a-4dab-9272-27bdfbc49238 None admin] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:37:06.226908 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e40c830-138a-4dab-9272-27bdfbc49238 None admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:37:06.227179 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e40c830-138a-4dab-9272-27bdfbc49238 None admin] PATH_INFO: `/v3/domains` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:37:06.227887 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e40c830-138a-4dab-9272-27bdfbc49238 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87058) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:37:06.228190 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e40c830-138a-4dab-9272-27bdfbc49238 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:37:06.229696 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e40c830-138a-4dab-9272-27bdfbc49238 None admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:37:06.243138 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 371/736] 10.222.0.20 () {56 vars in 1109 bytes} [Mon Aug 1 16:37:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:37:06.244441 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48638081-5746-4eb7-9a54-fec275d97837 None None] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:37:06.244728 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48638081-5746-4eb7-9a54-fec275d97837 None None] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:37:06.245002 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48638081-5746-4eb7-9a54-fec275d97837 None None] PATH_INFO: `/` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:37:06.250258 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 372/737] 10.222.0.20 () {54 vars in 926 bytes} [Mon Aug 1 16:37:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 16:37:06.250258 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f2ebd0-9b6b-4c8e-9572-5dd5c31e4563 None None] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:37:06.250258 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f2ebd0-9b6b-4c8e-9572-5dd5c31e4563 None None] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:37:06.250258 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f2ebd0-9b6b-4c8e-9572-5dd5c31e4563 None None] PATH_INFO: `/` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:37:06.250258 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 373/738] 10.222.0.20 () {54 vars in 926 bytes} [Mon Aug 1 16:37:06 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 16:37:06.250695 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0f2ebd0-9b6b-4c8e-9572-5dd5c31e4563 None None] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:37:06.263235 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-932e2b97-0686-4e8b-9f2f-b6028decc035 None admin] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:37:06.263426 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG dbcounter [-] [87058] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:37:06.305330 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3844eab3-23cb-457c-82ba-e24b7c5ab850 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:37:06.305986 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3844eab3-23cb-457c-82ba-e24b7c5ab850 None admin] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:37:06.306352 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3844eab3-23cb-457c-82ba-e24b7c5ab850 None admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:37:06.306633 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3844eab3-23cb-457c-82ba-e24b7c5ab850 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:37:06.317750 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3844eab3-23cb-457c-82ba-e24b7c5ab850 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:37:06.320106 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3844eab3-23cb-457c-82ba-e24b7c5ab850 None admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:37:06.335536 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application [None req-3844eab3-23cb-457c-82ba-e24b7c5ab850 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 16:37:06.335536 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 16:37:06.335536 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 16:37:06.335536 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 16:37:06.335536 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 16:37:06.335536 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 16:37:06.335536 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 16:37:06.335536 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 16:37:06.335536 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 16:37:06.335536 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 16:37:06.335536 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 16:37:06.335536 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 16:37:06.335536 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 16:37:06.335536 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 16:37:06.335536 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 16:37:06.336475 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 16:37:06.336475 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 16:37:06.336475 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 16:37:06.336475 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 16:37:06.336475 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 16:37:06.336475 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 16:37:06.336475 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 16:37:06.336475 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 16:37:06.336475 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application with Lock( Aug 01 16:37:06.336475 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 16:37:06.336475 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self._enter() Aug 01 16:37:06.336475 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 16:37:06.336475 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 16:37:06.336475 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 16:37:06.336475 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self.creator() Aug 01 16:37:06.336475 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 16:37:06.336475 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application created_value = creator( Aug 01 16:37:06.337982 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 16:37:06.337982 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 16:37:06.337982 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 16:37:06.337982 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application Aug 01 16:37:06.337982 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 366/739] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Aug 1 16:37:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 01 16:37:06.338293 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b632f70-2fab-4f21-a4e6-c0218e14c035 None None] REQUEST_METHOD: `POST` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:37:06.339122 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b632f70-2fab-4f21-a4e6-c0218e14c035 None None] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:37:06.339122 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b632f70-2fab-4f21-a4e6-c0218e14c035 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:37:06.357546 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.auth.core [None req-9b632f70-2fab-4f21-a4e6-c0218e14c035 None None] MFA Rules not processed for user `1007f8e379b74e169b98fb7c3c0ba5b4`. Rule list: `[]` (Enabled: `True`). {{(pid=87057) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:37:06.358781 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.fernet_utils [None req-9b632f70-2fab-4f21-a4e6-c0218e14c035 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87057) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:37:06.366892 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0f2ebd0-9b6b-4c8e-9572-5dd5c31e4563 None None] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:37:06.404385 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba132ce-d988-4d8a-9cfc-c8784800167d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:37:06.405035 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba132ce-d988-4d8a-9cfc-c8784800167d None admin] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:37:06.405320 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba132ce-d988-4d8a-9cfc-c8784800167d None admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:37:06.405594 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba132ce-d988-4d8a-9cfc-c8784800167d None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:37:06.415108 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ba132ce-d988-4d8a-9cfc-c8784800167d None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:37:06.416565 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ba132ce-d988-4d8a-9cfc-c8784800167d None admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:37:06.429316 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application [None req-0ba132ce-d988-4d8a-9cfc-c8784800167d None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 01 16:37:06.429316 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 16:37:06.429316 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 16:37:06.429316 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 16:37:06.429316 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 16:37:06.429316 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 16:37:06.429316 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 16:37:06.429316 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 16:37:06.429316 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 16:37:06.429316 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 16:37:06.429316 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 16:37:06.429316 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 16:37:06.429316 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 01 16:37:06.429316 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 01 16:37:06.429316 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 01 16:37:06.429316 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 01 16:37:06.430529 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 16:37:06.430529 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 16:37:06.430529 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 01 16:37:06.430529 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 16:37:06.430529 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 01 16:37:06.430529 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 16:37:06.430529 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 16:37:06.430529 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 16:37:06.430529 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 16:37:06.430529 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 16:37:06.430529 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 16:37:06.430529 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application with Lock( Aug 01 16:37:06.430529 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 16:37:06.430529 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self._enter() Aug 01 16:37:06.430529 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 16:37:06.430529 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 16:37:06.430529 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 16:37:06.430529 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self.creator() Aug 01 16:37:06.431839 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 16:37:06.431839 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application created_value = creator( Aug 01 16:37:06.431839 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 01 16:37:06.431839 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 01 16:37:06.431839 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 01 16:37:06.431839 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 01 16:37:06.431839 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 01 16:37:06.431839 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 01 16:37:06.431839 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 01 16:37:06.431839 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application Aug 01 16:37:06.431839 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 374/740] 10.222.0.20 () {56 vars in 1117 bytes} [Mon Aug 1 16:37:06 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 16:37:06.431839 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f27972-0a4f-4fb7-b3d3-7f3014184111 None None] REQUEST_METHOD: `POST` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:37:06.432515 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f27972-0a4f-4fb7-b3d3-7f3014184111 None None] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:37:06.432515 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f27972-0a4f-4fb7-b3d3-7f3014184111 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:37:06.440608 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 367/741] 10.222.0.20 () {58 vars in 1023 bytes} [Mon Aug 1 16:37:06 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 16:37:06.441315 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b632f70-2fab-4f21-a4e6-c0218e14c035 None None] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:37:06.468011 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.auth.core [None req-95f27972-0a4f-4fb7-b3d3-7f3014184111 None None] MFA Rules not processed for user `1007f8e379b74e169b98fb7c3c0ba5b4`. Rule list: `[]` (Enabled: `True`). {{(pid=87058) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:37:06.469219 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.fernet_utils [None req-95f27972-0a4f-4fb7-b3d3-7f3014184111 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:37:06.498062 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG dbcounter [-] [87057] Writing DB stats keystone:SELECT=100 {{(pid=87057) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:37:06.540752 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG dbcounter [-] [87058] Writing DB stats keystone:SELECT=100 {{(pid=87058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:37:06.546400 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 375/742] 10.222.0.20 () {58 vars in 1023 bytes} [Mon Aug 1 16:37:06 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 16:37:06.547217 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95f27972-0a4f-4fb7-b3d3-7f3014184111 None None] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:37:06.569389 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b632f70-2fab-4f21-a4e6-c0218e14c035 None None] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:37:06.609822 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4065fa10-4ccf-47e6-9a32-d0edccdaa0eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:37:06.610379 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4065fa10-4ccf-47e6-9a32-d0edccdaa0eb None admin] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:37:06.610657 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4065fa10-4ccf-47e6-9a32-d0edccdaa0eb None admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:37:06.610926 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4065fa10-4ccf-47e6-9a32-d0edccdaa0eb None admin] PATH_INFO: `/v3/domains` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:37:06.611617 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4065fa10-4ccf-47e6-9a32-d0edccdaa0eb None admin] RBAC: Adding query filter params (name=Default) {{(pid=87057) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:37:06.611912 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4065fa10-4ccf-47e6-9a32-d0edccdaa0eb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:37:06.613272 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4065fa10-4ccf-47e6-9a32-d0edccdaa0eb None admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:37:06.624463 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 368/743] 10.222.0.20 () {56 vars in 1109 bytes} [Mon Aug 1 16:37:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:37:06.625240 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4065fa10-4ccf-47e6-9a32-d0edccdaa0eb None admin] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:37:06.665590 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95f27972-0a4f-4fb7-b3d3-7f3014184111 None None] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:37:06.703008 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe3e763-558a-4f6b-98ba-7226d27b4d51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:37:06.703948 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe3e763-558a-4f6b-98ba-7226d27b4d51 None admin] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:37:06.704413 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe3e763-558a-4f6b-98ba-7226d27b4d51 None admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:37:06.704861 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe3e763-558a-4f6b-98ba-7226d27b4d51 None admin] PATH_INFO: `/v3/users` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:37:06.705846 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbe3e763-558a-4f6b-98ba-7226d27b4d51 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=87058) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:37:06.706356 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbe3e763-558a-4f6b-98ba-7226d27b4d51 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:37:06.708370 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbe3e763-558a-4f6b-98ba-7226d27b4d51 None admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:37:06.731971 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 376/744] 10.222.0.20 () {56 vars in 1149 bytes} [Mon Aug 1 16:37:06 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:37:06.732792 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe3e763-558a-4f6b-98ba-7226d27b4d51 None admin] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:37:06.732992 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4065fa10-4ccf-47e6-9a32-d0edccdaa0eb None admin] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:37:06.734789 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG dbcounter [-] [87057] Writing DB stats keystone:SELECT=100 {{(pid=87057) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:37:06.769690 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e66510-976c-449a-89ec-58dcaa80fea8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:37:06.770347 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e66510-976c-449a-89ec-58dcaa80fea8 None admin] REQUEST_METHOD: `POST` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:37:06.770630 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e66510-976c-449a-89ec-58dcaa80fea8 None admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:37:06.770907 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e66510-976c-449a-89ec-58dcaa80fea8 None admin] PATH_INFO: `/v3/services` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:37:06.772130 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03e66510-976c-449a-89ec-58dcaa80fea8 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:37:06.773515 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03e66510-976c-449a-89ec-58dcaa80fea8 None admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:37:06.792509 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG dbcounter [-] [87058] Writing DB stats keystone:SELECT=100 {{(pid=87058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:37:06.835986 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe3e763-558a-4f6b-98ba-7226d27b4d51 None admin] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:37:06.866157 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 369/745] 10.222.0.20 () {60 vars in 1141 bytes} [Mon Aug 1 16:37:06 2022] POST /identity/v3/services => generated 240 bytes in 231 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 16:37:06.866157 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e66510-976c-449a-89ec-58dcaa80fea8 None admin] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:37:06.892193 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ec7e99d-3a21-448a-97d9-eba0810bb4f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:37:06.892967 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec7e99d-3a21-448a-97d9-eba0810bb4f7 None admin] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:37:06.893248 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec7e99d-3a21-448a-97d9-eba0810bb4f7 None admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:37:06.893520 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec7e99d-3a21-448a-97d9-eba0810bb4f7 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:37:06.909538 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ec7e99d-3a21-448a-97d9-eba0810bb4f7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 16:37:06.915840 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application [None req-2ec7e99d-3a21-448a-97d9-eba0810bb4f7 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 16:37:06.915840 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 16:37:06.915840 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 16:37:06.915840 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 16:37:06.915840 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 16:37:06.915840 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 16:37:06.915840 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 16:37:06.915840 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 16:37:06.915840 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 16:37:06.915840 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 16:37:06.915840 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 16:37:06.915840 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 16:37:06.915840 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 16:37:06.915840 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 16:37:06.915840 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 16:37:06.915840 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 16:37:06.916992 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 16:37:06.916992 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 16:37:06.916992 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 16:37:06.916992 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 16:37:06.916992 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 16:37:06.916992 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 16:37:06.916992 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 16:37:06.916992 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 16:37:06.916992 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 16:37:06.916992 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 16:37:06.916992 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 16:37:06.916992 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application with Lock( Aug 01 16:37:06.916992 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 16:37:06.916992 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self._enter() Aug 01 16:37:06.916992 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 16:37:06.916992 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 16:37:06.918015 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 16:37:06.918015 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self.creator() Aug 01 16:37:06.918015 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 16:37:06.918015 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application created_value = creator( Aug 01 16:37:06.918015 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 16:37:06.918015 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 16:37:06.918015 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 16:37:06.918015 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 16:37:06.918015 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 16:37:06.918015 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 16:37:06.918015 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 16:37:06.918015 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application Aug 01 16:37:06.922718 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 377/746] 10.222.0.20 () {56 vars in 1096 bytes} [Mon Aug 1 16:37:06 2022] GET /identity/v3/roles/admin => generated 83 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 16:37:06.923655 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ec7e99d-3a21-448a-97d9-eba0810bb4f7 None admin] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:37:06.999978 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e66510-976c-449a-89ec-58dcaa80fea8 None admin] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:37:07.053306 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d23c03c2-c910-497b-9aa1-b729ce71ff42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:37:07.054373 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23c03c2-c910-497b-9aa1-b729ce71ff42 None admin] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:37:07.054484 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ec7e99d-3a21-448a-97d9-eba0810bb4f7 None admin] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:37:07.054985 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23c03c2-c910-497b-9aa1-b729ce71ff42 None admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:37:07.055560 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23c03c2-c910-497b-9aa1-b729ce71ff42 None admin] PATH_INFO: `/v3/projects/service` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:37:07.066252 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d23c03c2-c910-497b-9aa1-b729ce71ff42 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:37:07.067736 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d23c03c2-c910-497b-9aa1-b729ce71ff42 None admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:37:07.069962 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG dbcounter [-] [87058] Writing DB stats keystone:SELECT=100 {{(pid=87058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:37:07.081035 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application [None req-d23c03c2-c910-497b-9aa1-b729ce71ff42 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 16:37:07.081035 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 16:37:07.081035 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 16:37:07.081035 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 16:37:07.081035 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 16:37:07.081035 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 16:37:07.081035 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 16:37:07.081035 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 16:37:07.081035 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 16:37:07.081035 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 16:37:07.081035 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 16:37:07.081035 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 16:37:07.081035 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 01 16:37:07.081035 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 01 16:37:07.081035 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 01 16:37:07.088372 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 01 16:37:07.088372 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 16:37:07.088372 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 16:37:07.088372 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 16:37:07.088372 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 16:37:07.088372 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 16:37:07.088372 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 16:37:07.088372 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 16:37:07.088372 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application with Lock( Aug 01 16:37:07.088372 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 16:37:07.088372 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self._enter() Aug 01 16:37:07.088372 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 16:37:07.088372 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 16:37:07.088372 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 16:37:07.088372 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self.creator() Aug 01 16:37:07.088372 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 16:37:07.088372 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application created_value = creator( Aug 01 16:37:07.089267 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 01 16:37:07.089267 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 01 16:37:07.089267 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 01 16:37:07.089267 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 01 16:37:07.089267 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 01 16:37:07.089267 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 01 16:37:07.089267 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 16:37:07.089267 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application Aug 01 16:37:07.089267 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 370/747] 10.222.0.20 () {56 vars in 1111 bytes} [Mon Aug 1 16:37:06 2022] GET /identity/v3/projects/service => generated 88 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 16:37:07.089267 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d23c03c2-c910-497b-9aa1-b729ce71ff42 None admin] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:37:07.100504 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d922b1-4ab4-466b-8baa-3f098761aa07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:37:07.101140 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d922b1-4ab4-466b-8baa-3f098761aa07 None admin] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:37:07.101418 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d922b1-4ab4-466b-8baa-3f098761aa07 None admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:37:07.101689 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d922b1-4ab4-466b-8baa-3f098761aa07 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:37:07.117827 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG dbcounter [-] [87057] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87057) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:37:07.185330 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8d922b1-4ab4-466b-8baa-3f098761aa07 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:37:07.186996 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8d922b1-4ab4-466b-8baa-3f098761aa07 None admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:37:07.210933 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d23c03c2-c910-497b-9aa1-b729ce71ff42 None admin] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:37:07.250473 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9662a08c-28fd-44ef-94b2-6fcd2e440ab4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:37:07.251394 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9662a08c-28fd-44ef-94b2-6fcd2e440ab4 None admin] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:37:07.251843 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9662a08c-28fd-44ef-94b2-6fcd2e440ab4 None admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:37:07.252374 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9662a08c-28fd-44ef-94b2-6fcd2e440ab4 None admin] PATH_INFO: `/v3/roles` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:37:07.253373 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9662a08c-28fd-44ef-94b2-6fcd2e440ab4 None admin] RBAC: Adding query filter params (name=admin) {{(pid=87057) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:37:07.253880 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9662a08c-28fd-44ef-94b2-6fcd2e440ab4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:37:07.255503 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9662a08c-28fd-44ef-94b2-6fcd2e440ab4 None admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:37:07.266889 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 371/748] 10.222.0.20 () {56 vars in 1099 bytes} [Mon Aug 1 16:37:07 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:37:07.268024 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9662a08c-28fd-44ef-94b2-6fcd2e440ab4 None admin] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:37:07.306242 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 378/749] 10.222.0.20 () {58 vars in 1282 bytes} [Mon Aug 1 16:37:06 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 379 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 01 16:37:07.308016 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d922b1-4ab4-466b-8baa-3f098761aa07 None admin] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:37:07.323923 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG dbcounter [-] [87058] Writing DB stats keystone:SELECT=100 {{(pid=87058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:37:07.361036 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG dbcounter [-] [87057] Writing DB stats keystone:SELECT=100 {{(pid=87057) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:37:07.378457 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9662a08c-28fd-44ef-94b2-6fcd2e440ab4 None admin] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:37:07.414604 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d922b1-4ab4-466b-8baa-3f098761aa07 None admin] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:37:07.417530 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea5a591-bb79-46ae-91b7-49cb41d581d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:37:07.418376 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea5a591-bb79-46ae-91b7-49cb41d581d4 None admin] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:37:07.418829 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea5a591-bb79-46ae-91b7-49cb41d581d4 None admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:37:07.419306 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea5a591-bb79-46ae-91b7-49cb41d581d4 None admin] PATH_INFO: `/v3/projects` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:37:07.420277 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cea5a591-bb79-46ae-91b7-49cb41d581d4 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87057) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:37:07.420761 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cea5a591-bb79-46ae-91b7-49cb41d581d4 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:37:07.422390 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cea5a591-bb79-46ae-91b7-49cb41d581d4 None admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:37:07.436494 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 372/750] 10.222.0.20 () {56 vars in 1148 bytes} [Mon Aug 1 16:37:07 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:37:07.437557 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea5a591-bb79-46ae-91b7-49cb41d581d4 None admin] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:37:07.457900 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c34a28d-554f-45a6-b770-df445dd720f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:37:07.458626 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c34a28d-554f-45a6-b770-df445dd720f8 None admin] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:37:07.459520 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c34a28d-554f-45a6-b770-df445dd720f8 None admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:37:07.459955 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c34a28d-554f-45a6-b770-df445dd720f8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:37:07.470705 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c34a28d-554f-45a6-b770-df445dd720f8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:37:07.472313 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c34a28d-554f-45a6-b770-df445dd720f8 None admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:37:07.488016 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application [None req-2c34a28d-554f-45a6-b770-df445dd720f8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 16:37:07.488016 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 16:37:07.488016 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 16:37:07.488016 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 16:37:07.488016 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 16:37:07.488016 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 16:37:07.488016 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 16:37:07.488016 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 16:37:07.488016 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 16:37:07.488016 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 16:37:07.488016 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 16:37:07.488016 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 16:37:07.488016 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 16:37:07.488016 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 16:37:07.488016 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 16:37:07.488941 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 16:37:07.488941 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 16:37:07.488941 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 16:37:07.488941 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 16:37:07.488941 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 16:37:07.488941 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 16:37:07.488941 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 16:37:07.488941 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 16:37:07.488941 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application with Lock( Aug 01 16:37:07.488941 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 16:37:07.488941 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self._enter() Aug 01 16:37:07.488941 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 16:37:07.488941 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 16:37:07.488941 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 16:37:07.488941 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self.creator() Aug 01 16:37:07.488941 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 16:37:07.488941 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application created_value = creator( Aug 01 16:37:07.489840 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 16:37:07.489840 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 16:37:07.489840 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 16:37:07.489840 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application Aug 01 16:37:07.494623 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 379/751] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Aug 1 16:37:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 16:37:07.495503 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c34a28d-554f-45a6-b770-df445dd720f8 None admin] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:37:07.551989 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea5a591-bb79-46ae-91b7-49cb41d581d4 None admin] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:37:07.560772 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG dbcounter [-] [87058] Writing DB stats keystone:SELECT=100 {{(pid=87058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:37:07.589460 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd6f2ed5-7622-41a7-b5e3-d586ab244ec5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:37:07.590880 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6f2ed5-7622-41a7-b5e3-d586ab244ec5 None admin] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:37:07.591381 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6f2ed5-7622-41a7-b5e3-d586ab244ec5 None admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:37:07.591874 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6f2ed5-7622-41a7-b5e3-d586ab244ec5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:37:07.601396 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c34a28d-554f-45a6-b770-df445dd720f8 None admin] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:37:07.604679 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd6f2ed5-7622-41a7-b5e3-d586ab244ec5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:37:07.606387 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd6f2ed5-7622-41a7-b5e3-d586ab244ec5 None admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:37:07.610329 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG dbcounter [-] [87057] Writing DB stats keystone:SELECT=100 {{(pid=87057) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:37:07.624044 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application [None req-dd6f2ed5-7622-41a7-b5e3-d586ab244ec5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 16:37:07.624044 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 16:37:07.624044 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 16:37:07.624044 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 16:37:07.624044 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 16:37:07.624044 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 16:37:07.624044 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 16:37:07.624044 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 16:37:07.624044 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 16:37:07.624044 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 16:37:07.624044 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 16:37:07.624044 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 16:37:07.624044 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 16:37:07.624044 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 16:37:07.624044 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 16:37:07.627602 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 16:37:07.627602 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 16:37:07.627602 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 16:37:07.627602 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 16:37:07.627602 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 16:37:07.627602 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 16:37:07.627602 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 16:37:07.627602 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 16:37:07.627602 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application with Lock( Aug 01 16:37:07.627602 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 16:37:07.627602 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self._enter() Aug 01 16:37:07.627602 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 16:37:07.627602 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 16:37:07.627602 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 16:37:07.627602 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self.creator() Aug 01 16:37:07.627602 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 16:37:07.627602 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application created_value = creator( Aug 01 16:37:07.628565 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 16:37:07.628565 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 16:37:07.628565 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 16:37:07.628565 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application Aug 01 16:37:07.630198 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 373/752] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Aug 1 16:37:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 16:37:07.631737 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd6f2ed5-7622-41a7-b5e3-d586ab244ec5 None admin] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:37:07.643849 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d4f9ae1-bf28-49e8-bcdf-b511381fd009 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:37:07.644802 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4f9ae1-bf28-49e8-bcdf-b511381fd009 None admin] REQUEST_METHOD: `PUT` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:37:07.645262 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4f9ae1-bf28-49e8-bcdf-b511381fd009 None admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:37:07.645753 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4f9ae1-bf28-49e8-bcdf-b511381fd009 None admin] PATH_INFO: `/v3/projects/7b992d6a65df4bf9b27c13192d57c5f0/users/6533106d03eb4ca185a8839336d2aff9/roles/440bb943dd5b46fea038b903b6eaaf81` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:37:07.648701 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d4f9ae1-bf28-49e8-bcdf-b511381fd009 None admin] RBAC: Authorizing `identity:create_grant(project_id=7b992d6a65df4bf9b27c13192d57c5f0, user_id=6533106d03eb4ca185a8839336d2aff9, role_id=440bb943dd5b46fea038b903b6eaaf81)` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:37:07.650380 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d4f9ae1-bf28-49e8-bcdf-b511381fd009 None admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:37:07.732378 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 380/753] 10.222.0.20 () {58 vars in 1439 bytes} [Mon Aug 1 16:37:07 2022] PUT /identity/v3/projects/7b992d6a65df4bf9b27c13192d57c5f0/users/6533106d03eb4ca185a8839336d2aff9/roles/440bb943dd5b46fea038b903b6eaaf81 => generated 0 bytes in 231 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 16:37:07.734245 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d4f9ae1-bf28-49e8-bcdf-b511381fd009 None admin] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:37:07.760097 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd6f2ed5-7622-41a7-b5e3-d586ab244ec5 None admin] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:37:07.811913 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c8fa14-4333-45b6-90d2-11a1a3ad343d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:37:07.812553 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c8fa14-4333-45b6-90d2-11a1a3ad343d None admin] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:37:07.812829 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c8fa14-4333-45b6-90d2-11a1a3ad343d None admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:37:07.814392 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c8fa14-4333-45b6-90d2-11a1a3ad343d None admin] PATH_INFO: `/v3/domains` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:37:07.815121 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0c8fa14-4333-45b6-90d2-11a1a3ad343d None admin] RBAC: Adding query filter params (name=Default) {{(pid=87057) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:37:07.815424 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0c8fa14-4333-45b6-90d2-11a1a3ad343d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:37:07.822192 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0c8fa14-4333-45b6-90d2-11a1a3ad343d None admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:37:07.847601 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 374/754] 10.222.0.20 () {56 vars in 1109 bytes} [Mon Aug 1 16:37:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:37:07.854169 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c8fa14-4333-45b6-90d2-11a1a3ad343d None admin] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:37:07.918152 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d4f9ae1-bf28-49e8-bcdf-b511381fd009 None admin] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:37:07.961302 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG dbcounter [-] [87057] Writing DB stats keystone:SELECT=100 {{(pid=87057) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:37:07.962750 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG dbcounter [-] [87058] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:37:07.975553 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82bf9f32-bac1-4eb7-be37-d6fc4e2ac1ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:37:07.976390 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82bf9f32-bac1-4eb7-be37-d6fc4e2ac1ab None admin] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:37:07.976666 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82bf9f32-bac1-4eb7-be37-d6fc4e2ac1ab None admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:37:07.977650 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82bf9f32-bac1-4eb7-be37-d6fc4e2ac1ab None admin] PATH_INFO: `/v3/domains` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:37:07.977650 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82bf9f32-bac1-4eb7-be37-d6fc4e2ac1ab None admin] RBAC: Adding query filter params (name=Default) {{(pid=87058) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:37:07.980502 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82bf9f32-bac1-4eb7-be37-d6fc4e2ac1ab None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:37:07.981536 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82bf9f32-bac1-4eb7-be37-d6fc4e2ac1ab None admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:37:07.995654 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 381/755] 10.222.0.20 () {56 vars in 1109 bytes} [Mon Aug 1 16:37:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:37:08.001352 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82bf9f32-bac1-4eb7-be37-d6fc4e2ac1ab None admin] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:37:08.014096 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c8fa14-4333-45b6-90d2-11a1a3ad343d None admin] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:37:08.054670 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9756d614-0764-4614-ac29-bedbdfd22697 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:37:08.055660 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9756d614-0764-4614-ac29-bedbdfd22697 None admin] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:37:08.056127 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9756d614-0764-4614-ac29-bedbdfd22697 None admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:37:08.056612 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9756d614-0764-4614-ac29-bedbdfd22697 None admin] PATH_INFO: `/v3/users/swift` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:37:08.065223 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9756d614-0764-4614-ac29-bedbdfd22697 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:37:08.066705 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9756d614-0764-4614-ac29-bedbdfd22697 None admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:37:08.079305 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application [None req-9756d614-0764-4614-ac29-bedbdfd22697 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 01 16:37:08.079305 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 16:37:08.079305 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 16:37:08.079305 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 16:37:08.079305 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 16:37:08.079305 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 16:37:08.079305 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 16:37:08.079305 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 16:37:08.079305 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 16:37:08.079305 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 16:37:08.079305 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 16:37:08.079305 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 16:37:08.079305 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 01 16:37:08.079305 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 01 16:37:08.079305 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 01 16:37:08.079305 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 01 16:37:08.080332 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 16:37:08.080332 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 16:37:08.080332 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 01 16:37:08.080332 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 16:37:08.080332 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 01 16:37:08.080332 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 16:37:08.080332 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 16:37:08.080332 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 16:37:08.080332 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 16:37:08.080332 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 16:37:08.080332 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 16:37:08.080332 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application with Lock( Aug 01 16:37:08.080332 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 16:37:08.080332 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self._enter() Aug 01 16:37:08.080332 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 16:37:08.080332 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 16:37:08.080332 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 16:37:08.080332 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self.creator() Aug 01 16:37:08.081396 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 16:37:08.081396 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application created_value = creator( Aug 01 16:37:08.081396 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 01 16:37:08.081396 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 01 16:37:08.081396 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 01 16:37:08.081396 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 01 16:37:08.081396 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 01 16:37:08.081396 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 01 16:37:08.081396 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 01 16:37:08.081396 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application Aug 01 16:37:08.081396 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 375/756] 10.222.0.20 () {56 vars in 1096 bytes} [Mon Aug 1 16:37:07 2022] GET /identity/v3/users/swift => generated 83 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 01 16:37:08.091276 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9756d614-0764-4614-ac29-bedbdfd22697 None admin] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:37:08.157529 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82bf9f32-bac1-4eb7-be37-d6fc4e2ac1ab None admin] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:37:08.229793 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a426986-19ee-426c-9476-003b41d656ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:37:08.230469 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a426986-19ee-426c-9476-003b41d656ee None admin] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:37:08.230747 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a426986-19ee-426c-9476-003b41d656ee None admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:37:08.231018 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a426986-19ee-426c-9476-003b41d656ee None admin] PATH_INFO: `/v3/projects/service` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:37:08.244221 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a426986-19ee-426c-9476-003b41d656ee None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:37:08.245786 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a426986-19ee-426c-9476-003b41d656ee None admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:37:08.258564 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application [None req-1a426986-19ee-426c-9476-003b41d656ee None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 16:37:08.258564 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 16:37:08.258564 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 16:37:08.258564 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 16:37:08.258564 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 16:37:08.258564 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 16:37:08.258564 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 16:37:08.258564 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 16:37:08.258564 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 16:37:08.258564 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 16:37:08.258564 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 16:37:08.258564 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 16:37:08.258564 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 01 16:37:08.258564 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 01 16:37:08.258564 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 01 16:37:08.260644 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 01 16:37:08.260644 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 16:37:08.260644 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 16:37:08.260644 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 16:37:08.260644 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 16:37:08.260644 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 16:37:08.260644 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 16:37:08.260644 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 16:37:08.260644 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application with Lock( Aug 01 16:37:08.260644 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 16:37:08.260644 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self._enter() Aug 01 16:37:08.260644 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 16:37:08.260644 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 16:37:08.260644 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 16:37:08.260644 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self.creator() Aug 01 16:37:08.260644 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 16:37:08.260644 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application created_value = creator( Aug 01 16:37:08.262847 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 01 16:37:08.262847 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 01 16:37:08.262847 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 01 16:37:08.262847 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 01 16:37:08.262847 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 01 16:37:08.262847 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 01 16:37:08.262847 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 16:37:08.262847 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application Aug 01 16:37:08.262847 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 382/757] 10.222.0.20 () {56 vars in 1111 bytes} [Mon Aug 1 16:37:08 2022] GET /identity/v3/projects/service => generated 88 bytes in 259 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 01 16:37:08.262847 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524b1ccf-69dd-4353-a833-689e680f1776 None None] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:37:08.262847 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524b1ccf-69dd-4353-a833-689e680f1776 None None] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:37:08.264514 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524b1ccf-69dd-4353-a833-689e680f1776 None None] PATH_INFO: `/` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:37:08.264514 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 383/758] 10.222.0.20 () {54 vars in 926 bytes} [Mon Aug 1 16:37:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 16:37:08.269227 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104cfa59-7b9b-43a7-8075-920838e1a400 None None] REQUEST_METHOD: `POST` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:37:08.269622 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104cfa59-7b9b-43a7-8075-920838e1a400 None None] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:37:08.269783 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104cfa59-7b9b-43a7-8075-920838e1a400 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:37:08.272849 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9756d614-0764-4614-ac29-bedbdfd22697 None admin] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:37:08.288771 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.auth.core [None req-104cfa59-7b9b-43a7-8075-920838e1a400 None None] MFA Rules not processed for user `1007f8e379b74e169b98fb7c3c0ba5b4`. Rule list: `[]` (Enabled: `True`). {{(pid=87058) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:37:08.290160 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG dbcounter [-] [87057] Writing DB stats keystone:SELECT=100 {{(pid=87057) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:37:08.290984 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.fernet_utils [None req-104cfa59-7b9b-43a7-8075-920838e1a400 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:37:08.309990 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c420b9-4ce6-4eb4-acab-7099b40da734 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:37:08.310629 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c420b9-4ce6-4eb4-acab-7099b40da734 None admin] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:37:08.310905 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c420b9-4ce6-4eb4-acab-7099b40da734 None admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:37:08.311174 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c420b9-4ce6-4eb4-acab-7099b40da734 None admin] PATH_INFO: `/v3/users` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:37:08.311961 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01c420b9-4ce6-4eb4-acab-7099b40da734 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=87057) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:37:08.312264 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01c420b9-4ce6-4eb4-acab-7099b40da734 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:37:08.313682 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01c420b9-4ce6-4eb4-acab-7099b40da734 None admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:37:08.333012 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 376/759] 10.222.0.20 () {56 vars in 1135 bytes} [Mon Aug 1 16:37:08 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:37:08.339142 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c420b9-4ce6-4eb4-acab-7099b40da734 None admin] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:37:08.343616 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG dbcounter [-] [87058] Writing DB stats keystone:SELECT=100 {{(pid=87058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:37:08.386055 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 384/760] 10.222.0.20 () {58 vars in 1023 bytes} [Mon Aug 1 16:37:08 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 01 16:37:08.386055 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-104cfa59-7b9b-43a7-8075-920838e1a400 None None] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:37:08.473767 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c420b9-4ce6-4eb4-acab-7099b40da734 None admin] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:37:08.489878 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-104cfa59-7b9b-43a7-8075-920838e1a400 None None] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:37:08.512093 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78319f44-7173-4e7f-a69b-52966da50180 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:37:08.512977 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78319f44-7173-4e7f-a69b-52966da50180 None admin] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:37:08.513407 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78319f44-7173-4e7f-a69b-52966da50180 None admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:37:08.513825 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78319f44-7173-4e7f-a69b-52966da50180 None admin] PATH_INFO: `/v3/projects` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:37:08.514915 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78319f44-7173-4e7f-a69b-52966da50180 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87057) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:37:08.515385 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78319f44-7173-4e7f-a69b-52966da50180 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:37:08.517334 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78319f44-7173-4e7f-a69b-52966da50180 None admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:37:08.526545 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b29261d1-dc8c-443f-89a0-e13990b1151c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:37:08.527247 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29261d1-dc8c-443f-89a0-e13990b1151c None admin] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:37:08.527524 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29261d1-dc8c-443f-89a0-e13990b1151c None admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:37:08.527795 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29261d1-dc8c-443f-89a0-e13990b1151c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:37:08.536446 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 377/761] 10.222.0.20 () {56 vars in 1148 bytes} [Mon Aug 1 16:37:08 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:37:08.538095 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78319f44-7173-4e7f-a69b-52966da50180 None admin] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:37:08.575988 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG dbcounter [-] [87058] Writing DB stats keystone:SELECT=100 {{(pid=87058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:37:08.577507 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG dbcounter [-] [87057] Writing DB stats keystone:SELECT=100 {{(pid=87057) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:37:08.603641 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b29261d1-dc8c-443f-89a0-e13990b1151c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:37:08.605085 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b29261d1-dc8c-443f-89a0-e13990b1151c None admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:37:08.651804 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78319f44-7173-4e7f-a69b-52966da50180 None admin] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:37:08.690244 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c998b6fe-f9dc-4889-9537-de40d6b9edb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:37:08.691234 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c998b6fe-f9dc-4889-9537-de40d6b9edb5 None admin] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:37:08.691661 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c998b6fe-f9dc-4889-9537-de40d6b9edb5 None admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:37:08.692083 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c998b6fe-f9dc-4889-9537-de40d6b9edb5 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:37:08.692888 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c998b6fe-f9dc-4889-9537-de40d6b9edb5 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:37:08.695375 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c998b6fe-f9dc-4889-9537-de40d6b9edb5 None admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:37:08.707267 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application [None req-c998b6fe-f9dc-4889-9537-de40d6b9edb5 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 01 16:37:08.707267 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 16:37:08.707267 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 01 16:37:08.707267 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 01 16:37:08.707267 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 01 16:37:08.707267 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 01 16:37:08.707267 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 01 16:37:08.707267 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 01 16:37:08.707267 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 01 16:37:08.707267 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application Aug 01 16:37:08.707267 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 01 16:37:08.707267 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application Aug 01 16:37:08.707267 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 16:37:08.707267 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 16:37:08.707267 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 16:37:08.707267 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 16:37:08.708445 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 16:37:08.708445 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 16:37:08.708445 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 16:37:08.708445 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 16:37:08.708445 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 16:37:08.708445 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 16:37:08.708445 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 16:37:08.708445 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 01 16:37:08.708445 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 01 16:37:08.708445 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 01 16:37:08.708445 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 01 16:37:08.708445 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 16:37:08.708445 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 16:37:08.708445 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 16:37:08.708445 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 16:37:08.708445 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 16:37:08.709440 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 16:37:08.709440 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 16:37:08.709440 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application with Lock( Aug 01 16:37:08.709440 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 16:37:08.709440 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self._enter() Aug 01 16:37:08.709440 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 16:37:08.709440 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 16:37:08.709440 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 16:37:08.709440 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self.creator() Aug 01 16:37:08.709440 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 16:37:08.709440 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application created_value = creator( Aug 01 16:37:08.709440 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 01 16:37:08.709440 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 01 16:37:08.709440 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 01 16:37:08.709440 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application Aug 01 16:37:08.715647 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 378/763] 10.222.0.20 () {56 vars in 1129 bytes} [Mon Aug 1 16:37:08 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 16:37:08.716660 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c998b6fe-f9dc-4889-9537-de40d6b9edb5 None admin] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:37:08.719532 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 385/763] 10.222.0.20 () {58 vars in 1282 bytes} [Mon Aug 1 16:37:08 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 336 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 01 16:37:08.719998 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b29261d1-dc8c-443f-89a0-e13990b1151c None admin] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:37:08.798862 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG dbcounter [-] [87058] Writing DB stats keystone:SELECT=100 {{(pid=87058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:37:08.826162 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b29261d1-dc8c-443f-89a0-e13990b1151c None admin] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:37:08.848291 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG dbcounter [-] [87057] Writing DB stats keystone:SELECT=100 {{(pid=87057) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:37:08.852615 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c998b6fe-f9dc-4889-9537-de40d6b9edb5 None admin] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:37:08.867951 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be3822c3-aa0c-41ac-b516-44aed96e1c76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:37:08.868324 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3822c3-aa0c-41ac-b516-44aed96e1c76 None admin] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:37:08.868599 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3822c3-aa0c-41ac-b516-44aed96e1c76 None admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:37:08.868867 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3822c3-aa0c-41ac-b516-44aed96e1c76 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:37:08.881378 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be3822c3-aa0c-41ac-b516-44aed96e1c76 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:37:08.883166 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be3822c3-aa0c-41ac-b516-44aed96e1c76 None admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:37:08.891517 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3125941a-dd76-4587-a8d9-0717db128245 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:37:08.892705 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3125941a-dd76-4587-a8d9-0717db128245 None admin] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:37:08.893275 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3125941a-dd76-4587-a8d9-0717db128245 None admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:37:08.893754 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3125941a-dd76-4587-a8d9-0717db128245 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:37:08.894796 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3125941a-dd76-4587-a8d9-0717db128245 None admin] RBAC: Adding query filter params (role.id=51bb93e277c446b49bb80a32afcd3af7, scope.project.id=7b992d6a65df4bf9b27c13192d57c5f0, user.id=b6a93d1c6c82489099adc8b413280fc0) {{(pid=87057) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:37:08.895407 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3125941a-dd76-4587-a8d9-0717db128245 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:37:08.898127 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3125941a-dd76-4587-a8d9-0717db128245 None admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:37:08.899659 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application [None req-be3822c3-aa0c-41ac-b516-44aed96e1c76 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 16:37:08.899659 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 16:37:08.899659 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 16:37:08.899659 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 16:37:08.899659 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 16:37:08.899659 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 16:37:08.899659 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 16:37:08.899659 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 16:37:08.899659 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 16:37:08.899659 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 16:37:08.899659 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 16:37:08.899659 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 16:37:08.899659 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 16:37:08.899659 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 16:37:08.899659 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 16:37:08.900749 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 16:37:08.900749 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 16:37:08.900749 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 16:37:08.900749 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 16:37:08.900749 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 16:37:08.900749 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 16:37:08.900749 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 16:37:08.900749 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 16:37:08.900749 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application with Lock( Aug 01 16:37:08.900749 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 16:37:08.900749 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self._enter() Aug 01 16:37:08.900749 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 16:37:08.900749 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 16:37:08.900749 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 16:37:08.900749 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self.creator() Aug 01 16:37:08.900749 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 16:37:08.900749 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application created_value = creator( Aug 01 16:37:08.901657 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 16:37:08.901657 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 16:37:08.901657 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 16:37:08.901657 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application Aug 01 16:37:08.905708 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 386/764] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Aug 1 16:37:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 01 16:37:08.906557 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be3822c3-aa0c-41ac-b516-44aed96e1c76 None admin] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:37:08.908191 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 379/765] 10.222.0.20 () {56 vars in 1374 bytes} [Mon Aug 1 16:37:08 2022] GET /identity/v3/role_assignments?user.id=b6a93d1c6c82489099adc8b413280fc0&scope.project.id=7b992d6a65df4bf9b27c13192d57c5f0&role.id=51bb93e277c446b49bb80a32afcd3af7 => generated 259 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:37:08.909210 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3125941a-dd76-4587-a8d9-0717db128245 None admin] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:37:09.020511 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3125941a-dd76-4587-a8d9-0717db128245 None admin] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:37:09.025904 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be3822c3-aa0c-41ac-b516-44aed96e1c76 None admin] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:37:09.057219 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG dbcounter [-] [87058] Writing DB stats keystone:SELECT=100 {{(pid=87058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:37:09.061964 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-917b2b21-c4e1-4c34-9a01-2f32541fe095 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:37:09.063000 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917b2b21-c4e1-4c34-9a01-2f32541fe095 None admin] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:37:09.063607 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917b2b21-c4e1-4c34-9a01-2f32541fe095 None admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:37:09.064203 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917b2b21-c4e1-4c34-9a01-2f32541fe095 None admin] PATH_INFO: `/v3/domains` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:37:09.065132 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-917b2b21-c4e1-4c34-9a01-2f32541fe095 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87057) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:37:09.065822 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-917b2b21-c4e1-4c34-9a01-2f32541fe095 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:37:09.067044 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c926f07-9109-4165-bea0-c473d515076e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:37:09.067679 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c926f07-9109-4165-bea0-c473d515076e None admin] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:37:09.067956 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c926f07-9109-4165-bea0-c473d515076e None admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:37:09.068281 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-917b2b21-c4e1-4c34-9a01-2f32541fe095 None admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:37:09.068443 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c926f07-9109-4165-bea0-c473d515076e None admin] PATH_INFO: `/v3/services` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:37:09.068993 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c926f07-9109-4165-bea0-c473d515076e None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=87058) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:37:09.069294 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c926f07-9109-4165-bea0-c473d515076e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:37:09.070879 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c926f07-9109-4165-bea0-c473d515076e None admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:37:09.076263 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 387/766] 10.222.0.20 () {56 vars in 1124 bytes} [Mon Aug 1 16:37:08 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:37:09.081709 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c926f07-9109-4165-bea0-c473d515076e None admin] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:37:09.083585 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 380/767] 10.222.0.20 () {56 vars in 1109 bytes} [Mon Aug 1 16:37:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:37:09.089866 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-917b2b21-c4e1-4c34-9a01-2f32541fe095 None admin] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:37:09.119599 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG dbcounter [-] [87057] Writing DB stats keystone:SELECT=100 {{(pid=87057) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:37:09.217831 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-917b2b21-c4e1-4c34-9a01-2f32541fe095 None admin] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:37:09.248971 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c926f07-9109-4165-bea0-c473d515076e None admin] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:37:09.253759 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c7fc3b0-6113-44c0-b2c3-731e756fcfe5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:37:09.254669 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7fc3b0-6113-44c0-b2c3-731e756fcfe5 None admin] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:37:09.255177 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7fc3b0-6113-44c0-b2c3-731e756fcfe5 None admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:37:09.255698 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7fc3b0-6113-44c0-b2c3-731e756fcfe5 None admin] PATH_INFO: `/v3/projects/service` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:37:09.263630 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c7fc3b0-6113-44c0-b2c3-731e756fcfe5 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:37:09.265347 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c7fc3b0-6113-44c0-b2c3-731e756fcfe5 None admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:37:09.278958 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application [None req-3c7fc3b0-6113-44c0-b2c3-731e756fcfe5 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 16:37:09.278958 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 16:37:09.278958 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 16:37:09.278958 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 16:37:09.278958 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 16:37:09.278958 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 16:37:09.278958 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 16:37:09.278958 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 16:37:09.278958 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 16:37:09.278958 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 16:37:09.278958 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 16:37:09.278958 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 16:37:09.278958 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 01 16:37:09.278958 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 01 16:37:09.278958 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 01 16:37:09.279867 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 01 16:37:09.279867 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 16:37:09.279867 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 16:37:09.279867 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 16:37:09.279867 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 16:37:09.279867 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 16:37:09.279867 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 16:37:09.279867 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 16:37:09.279867 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application with Lock( Aug 01 16:37:09.279867 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 16:37:09.279867 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self._enter() Aug 01 16:37:09.279867 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 16:37:09.279867 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 16:37:09.279867 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 16:37:09.279867 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self.creator() Aug 01 16:37:09.279867 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 16:37:09.279867 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application created_value = creator( Aug 01 16:37:09.280746 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 01 16:37:09.280746 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 01 16:37:09.280746 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 01 16:37:09.280746 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 01 16:37:09.280746 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 01 16:37:09.280746 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 01 16:37:09.280746 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 16:37:09.280746 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application Aug 01 16:37:09.283288 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 381/768] 10.222.0.20 () {56 vars in 1111 bytes} [Mon Aug 1 16:37:09 2022] GET /identity/v3/projects/service => generated 88 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 01 16:37:09.285153 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a343e9-7017-4754-814a-69120de0f50b None None] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:37:09.286067 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a343e9-7017-4754-814a-69120de0f50b None None] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:37:09.288151 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a343e9-7017-4754-814a-69120de0f50b None None] PATH_INFO: `/` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:37:09.289620 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 382/769] 10.222.0.20 () {54 vars in 926 bytes} [Mon Aug 1 16:37:09 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 16:37:09.290759 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a343e9-7017-4754-814a-69120de0f50b None None] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:37:09.291565 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c17043e-4136-4578-a567-133549389ad8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:37:09.292359 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c17043e-4136-4578-a567-133549389ad8 None admin] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:37:09.292776 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c17043e-4136-4578-a567-133549389ad8 None admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:37:09.293186 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c17043e-4136-4578-a567-133549389ad8 None admin] PATH_INFO: `/v3/services` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:37:09.294386 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c17043e-4136-4578-a567-133549389ad8 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=87058) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:37:09.295193 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c17043e-4136-4578-a567-133549389ad8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:37:09.296867 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c17043e-4136-4578-a567-133549389ad8 None admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:37:09.302767 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 388/770] 10.222.0.20 () {56 vars in 1124 bytes} [Mon Aug 1 16:37:09 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:37:09.304443 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981dd221-0fba-4188-a6c7-b20ae815f7bf None None] REQUEST_METHOD: `POST` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:37:09.304870 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981dd221-0fba-4188-a6c7-b20ae815f7bf None None] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:37:09.305287 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981dd221-0fba-4188-a6c7-b20ae815f7bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:37:09.325283 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.auth.core [None req-981dd221-0fba-4188-a6c7-b20ae815f7bf None None] MFA Rules not processed for user `1007f8e379b74e169b98fb7c3c0ba5b4`. Rule list: `[]` (Enabled: `True`). {{(pid=87058) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:37:09.326850 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.fernet_utils [None req-981dd221-0fba-4188-a6c7-b20ae815f7bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:37:09.383427 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG dbcounter [-] [87058] Writing DB stats keystone:SELECT=100 {{(pid=87058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:37:09.399940 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG dbcounter [-] [87057] Writing DB stats keystone:SELECT=100 {{(pid=87057) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:37:09.404924 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 389/771] 10.222.0.20 () {58 vars in 1023 bytes} [Mon Aug 1 16:37:09 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 01 16:37:09.405918 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-981dd221-0fba-4188-a6c7-b20ae815f7bf None None] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:37:09.438629 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a343e9-7017-4754-814a-69120de0f50b None None] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:37:09.484428 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ffe1a4-1028-4b82-a63a-94365c9d15b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:37:09.485143 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ffe1a4-1028-4b82-a63a-94365c9d15b3 None admin] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:37:09.485423 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ffe1a4-1028-4b82-a63a-94365c9d15b3 None admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:37:09.485695 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ffe1a4-1028-4b82-a63a-94365c9d15b3 None admin] PATH_INFO: `/v3/projects` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:37:09.486644 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15ffe1a4-1028-4b82-a63a-94365c9d15b3 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87057) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:37:09.486952 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15ffe1a4-1028-4b82-a63a-94365c9d15b3 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:37:09.488392 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15ffe1a4-1028-4b82-a63a-94365c9d15b3 None admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:37:09.506207 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 383/772] 10.222.0.20 () {56 vars in 1148 bytes} [Mon Aug 1 16:37:09 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:37:09.506207 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ffe1a4-1028-4b82-a63a-94365c9d15b3 None admin] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:37:09.539557 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-981dd221-0fba-4188-a6c7-b20ae815f7bf None None] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:37:09.577205 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a765c4e0-6c8b-413e-a210-11ad689df992 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:37:09.577830 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a765c4e0-6c8b-413e-a210-11ad689df992 None admin] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:37:09.577980 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a765c4e0-6c8b-413e-a210-11ad689df992 None admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:37:09.578333 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a765c4e0-6c8b-413e-a210-11ad689df992 None admin] PATH_INFO: `/v3/endpoints` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:37:09.579096 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a765c4e0-6c8b-413e-a210-11ad689df992 None admin] RBAC: Adding query filter params (interface=public, service_id=56e6ab2309c24afdbf240e7f1979a20a, region_id=RegionOne) {{(pid=87058) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:37:09.579456 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a765c4e0-6c8b-413e-a210-11ad689df992 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:37:09.581051 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a765c4e0-6c8b-413e-a210-11ad689df992 None admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:37:09.590757 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 390/773] 10.222.0.20 () {56 vars in 1251 bytes} [Mon Aug 1 16:37:09 2022] GET /identity/v3/endpoints?service_id=56e6ab2309c24afdbf240e7f1979a20a&interface=public®ion_id=RegionOne => generated 194 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:37:09.591019 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a765c4e0-6c8b-413e-a210-11ad689df992 None admin] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:37:09.626988 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ffe1a4-1028-4b82-a63a-94365c9d15b3 None admin] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:37:09.644601 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG dbcounter [-] [87058] Writing DB stats keystone:SELECT=100 {{(pid=87058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:37:09.673357 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad52dc67-84c3-4be6-b224-bcb4f29e927f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:37:09.674002 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad52dc67-84c3-4be6-b224-bcb4f29e927f None admin] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:37:09.674348 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad52dc67-84c3-4be6-b224-bcb4f29e927f None admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:37:09.674620 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad52dc67-84c3-4be6-b224-bcb4f29e927f None admin] PATH_INFO: `/v3/services/network` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:37:09.675992 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad52dc67-84c3-4be6-b224-bcb4f29e927f None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:37:09.677433 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad52dc67-84c3-4be6-b224-bcb4f29e927f None admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:37:09.687662 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application [None req-ad52dc67-84c3-4be6-b224-bcb4f29e927f None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Aug 01 16:37:09.687662 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 16:37:09.687662 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 01 16:37:09.687662 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 01 16:37:09.687662 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 01 16:37:09.687662 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 01 16:37:09.687662 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 01 16:37:09.687662 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 01 16:37:09.687662 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 01 16:37:09.687662 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application Aug 01 16:37:09.687662 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 01 16:37:09.687662 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application Aug 01 16:37:09.687662 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 16:37:09.687662 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 16:37:09.687662 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 16:37:09.687662 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 16:37:09.687662 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 16:37:09.688711 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 16:37:09.688711 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 16:37:09.688711 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 16:37:09.688711 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 16:37:09.688711 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 16:37:09.688711 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 16:37:09.688711 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 01 16:37:09.688711 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 01 16:37:09.688711 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 01 16:37:09.688711 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 01 16:37:09.688711 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 16:37:09.688711 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 16:37:09.688711 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 16:37:09.688711 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 16:37:09.688711 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 16:37:09.688711 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 16:37:09.689540 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 16:37:09.689540 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application with Lock( Aug 01 16:37:09.689540 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 16:37:09.689540 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self._enter() Aug 01 16:37:09.689540 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 16:37:09.689540 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 16:37:09.689540 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 16:37:09.689540 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self.creator() Aug 01 16:37:09.689540 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 16:37:09.689540 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application created_value = creator( Aug 01 16:37:09.689540 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 01 16:37:09.689540 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 01 16:37:09.689540 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 01 16:37:09.689540 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application Aug 01 16:37:09.694483 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 384/774] 10.222.0.20 () {56 vars in 1111 bytes} [Mon Aug 1 16:37:09 2022] GET /identity/v3/services/network => generated 88 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 16:37:09.695622 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad52dc67-84c3-4be6-b224-bcb4f29e927f None admin] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:37:09.705325 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG dbcounter [-] [87057] Writing DB stats keystone:SELECT=100 {{(pid=87057) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:37:09.713662 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a765c4e0-6c8b-413e-a210-11ad689df992 None admin] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:37:09.756153 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e69001a-54a8-4f85-99b6-39ff0a90effd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:37:09.757098 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e69001a-54a8-4f85-99b6-39ff0a90effd None admin] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:37:09.757527 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e69001a-54a8-4f85-99b6-39ff0a90effd None admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:37:09.757946 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e69001a-54a8-4f85-99b6-39ff0a90effd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:37:09.758916 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e69001a-54a8-4f85-99b6-39ff0a90effd None admin] RBAC: Adding query filter params (role.id=440bb943dd5b46fea038b903b6eaaf81, scope.project.id=7b992d6a65df4bf9b27c13192d57c5f0, user.id=9ab4add1faa04faa811ef7a9341cc35c) {{(pid=87058) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:37:09.759457 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e69001a-54a8-4f85-99b6-39ff0a90effd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:37:09.761226 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e69001a-54a8-4f85-99b6-39ff0a90effd None admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:37:09.768074 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 391/775] 10.222.0.20 () {56 vars in 1374 bytes} [Mon Aug 1 16:37:09 2022] GET /identity/v3/role_assignments?user.id=9ab4add1faa04faa811ef7a9341cc35c&scope.project.id=7b992d6a65df4bf9b27c13192d57c5f0&role.id=440bb943dd5b46fea038b903b6eaaf81 => generated 259 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:37:09.769214 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e69001a-54a8-4f85-99b6-39ff0a90effd None admin] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:37:09.816646 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad52dc67-84c3-4be6-b224-bcb4f29e927f None admin] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:37:09.865877 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-975eef8b-59ac-4b22-9524-b73a5ca1b3b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:37:09.869271 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975eef8b-59ac-4b22-9524-b73a5ca1b3b7 None admin] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:37:09.871269 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975eef8b-59ac-4b22-9524-b73a5ca1b3b7 None admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:37:09.872643 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975eef8b-59ac-4b22-9524-b73a5ca1b3b7 None admin] PATH_INFO: `/v3/services` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:37:09.873756 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-975eef8b-59ac-4b22-9524-b73a5ca1b3b7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:37:09.877195 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-975eef8b-59ac-4b22-9524-b73a5ca1b3b7 None admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:37:09.890782 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 385/776] 10.222.0.20 () {56 vars in 1087 bytes} [Mon Aug 1 16:37:09 2022] GET /identity/v3/services => generated 772 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:37:09.907887 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e69001a-54a8-4f85-99b6-39ff0a90effd None admin] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:37:09.943472 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG dbcounter [-] [87058] Writing DB stats keystone:SELECT=100 {{(pid=87058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:37:09.978378 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e7c89b0-8f91-4c37-83b3-9497c3739fd3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:37:09.979463 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7c89b0-8f91-4c37-83b3-9497c3739fd3 None admin] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:37:09.979997 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7c89b0-8f91-4c37-83b3-9497c3739fd3 None admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:37:09.980477 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7c89b0-8f91-4c37-83b3-9497c3739fd3 None admin] PATH_INFO: `/v3/services` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:37:09.981571 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e7c89b0-8f91-4c37-83b3-9497c3739fd3 None admin] RBAC: Adding query filter params (name=network) {{(pid=87058) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:37:09.982123 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e7c89b0-8f91-4c37-83b3-9497c3739fd3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:37:09.984730 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e7c89b0-8f91-4c37-83b3-9497c3739fd3 None admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:37:09.997729 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 392/777] 10.222.0.20 () {56 vars in 1112 bytes} [Mon Aug 1 16:37:09 2022] GET /identity/v3/services?name=network => generated 124 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:37:10.004045 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-975eef8b-59ac-4b22-9524-b73a5ca1b3b7 None admin] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:37:10.101636 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9590fc0e-220d-4b77-9d9f-24ed5042f128 None None] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:37:10.101963 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9590fc0e-220d-4b77-9d9f-24ed5042f128 None None] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:37:10.102269 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9590fc0e-220d-4b77-9d9f-24ed5042f128 None None] PATH_INFO: `/` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:37:10.103933 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 393/778] 10.222.0.20 () {54 vars in 926 bytes} [Mon Aug 1 16:37:10 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 16:37:10.111616 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49276ce-bbe7-4e1d-af91-d83b936f3731 None None] REQUEST_METHOD: `POST` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:37:10.112235 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49276ce-bbe7-4e1d-af91-d83b936f3731 None None] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:37:10.112732 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49276ce-bbe7-4e1d-af91-d83b936f3731 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:37:10.133906 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG dbcounter [-] [87057] Writing DB stats keystone:SELECT=100 {{(pid=87057) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:37:10.155093 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.auth.core [None req-d49276ce-bbe7-4e1d-af91-d83b936f3731 None None] MFA Rules not processed for user `1007f8e379b74e169b98fb7c3c0ba5b4`. Rule list: `[]` (Enabled: `True`). {{(pid=87058) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:37:10.156891 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.fernet_utils [None req-d49276ce-bbe7-4e1d-af91-d83b936f3731 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:37:10.179146 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-975eef8b-59ac-4b22-9524-b73a5ca1b3b7 None admin] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:37:10.247376 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f617eab7-7070-4776-863e-734d16217449 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:37:10.248678 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f617eab7-7070-4776-863e-734d16217449 None admin] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:37:10.249177 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f617eab7-7070-4776-863e-734d16217449 None admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:37:10.249681 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f617eab7-7070-4776-863e-734d16217449 None admin] PATH_INFO: `/v3/services` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:37:10.250682 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f617eab7-7070-4776-863e-734d16217449 None admin] RBAC: Adding query filter params (type=network) {{(pid=87057) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:37:10.251193 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f617eab7-7070-4776-863e-734d16217449 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:37:10.252801 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f617eab7-7070-4776-863e-734d16217449 None admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:37:10.266560 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 386/779] 10.222.0.20 () {56 vars in 1112 bytes} [Mon Aug 1 16:37:10 2022] GET /identity/v3/services?type=network => generated 350 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:37:10.274430 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f617eab7-7070-4776-863e-734d16217449 None admin] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:37:10.297834 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 394/780] 10.222.0.20 () {58 vars in 1023 bytes} [Mon Aug 1 16:37:10 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 01 16:37:10.342566 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d49276ce-bbe7-4e1d-af91-d83b936f3731 None None] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:37:10.453817 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f617eab7-7070-4776-863e-734d16217449 None admin] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:37:10.497059 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d49276ce-bbe7-4e1d-af91-d83b936f3731 None None] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:37:10.509427 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG dbcounter [-] [87058] Writing DB stats keystone:SELECT=100 {{(pid=87058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:37:10.525896 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-870bfd56-2f61-4877-a688-1ed77283e4b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:37:10.531381 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870bfd56-2f61-4877-a688-1ed77283e4b9 None admin] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:37:10.531381 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870bfd56-2f61-4877-a688-1ed77283e4b9 None admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:37:10.531381 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870bfd56-2f61-4877-a688-1ed77283e4b9 None admin] PATH_INFO: `/v3/endpoints` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:37:10.531381 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-870bfd56-2f61-4877-a688-1ed77283e4b9 None admin] RBAC: Adding query filter params (interface=public, service_id=ec62ddd8985a4d7d989f4b42a224ab56, region_id=RegionOne) {{(pid=87057) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:37:10.531381 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-870bfd56-2f61-4877-a688-1ed77283e4b9 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:37:10.531970 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-870bfd56-2f61-4877-a688-1ed77283e4b9 None admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:37:10.539461 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG dbcounter [-] [87057] Writing DB stats keystone:SELECT=100 {{(pid=87057) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:37:10.561669 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 387/781] 10.222.0.20 () {56 vars in 1251 bytes} [Mon Aug 1 16:37:10 2022] GET /identity/v3/endpoints?service_id=ec62ddd8985a4d7d989f4b42a224ab56&interface=public®ion_id=RegionOne => generated 194 bytes in 287 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:37:10.567067 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-870bfd56-2f61-4877-a688-1ed77283e4b9 None admin] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:37:10.584902 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5563d60-660b-45c2-8b65-53472723792c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:37:10.585543 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5563d60-660b-45c2-8b65-53472723792c None admin] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:37:10.585849 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5563d60-660b-45c2-8b65-53472723792c None admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:37:10.586121 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5563d60-660b-45c2-8b65-53472723792c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:37:10.609131 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5563d60-660b-45c2-8b65-53472723792c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 16:37:10.615861 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application [None req-e5563d60-660b-45c2-8b65-53472723792c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 16:37:10.615861 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 16:37:10.615861 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 16:37:10.615861 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 16:37:10.615861 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 16:37:10.615861 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 16:37:10.615861 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 16:37:10.615861 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 16:37:10.615861 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 16:37:10.615861 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 16:37:10.615861 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 16:37:10.615861 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 16:37:10.615861 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 16:37:10.615861 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 16:37:10.615861 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 16:37:10.615861 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 16:37:10.616857 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 16:37:10.616857 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 16:37:10.616857 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 16:37:10.616857 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 16:37:10.616857 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 16:37:10.616857 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 16:37:10.616857 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 16:37:10.616857 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 16:37:10.616857 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 16:37:10.616857 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 16:37:10.616857 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 16:37:10.616857 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application with Lock( Aug 01 16:37:10.616857 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 16:37:10.616857 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self._enter() Aug 01 16:37:10.616857 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 16:37:10.616857 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 16:37:10.617717 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 16:37:10.617717 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self.creator() Aug 01 16:37:10.617717 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 16:37:10.617717 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application created_value = creator( Aug 01 16:37:10.617717 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 16:37:10.617717 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 16:37:10.617717 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 16:37:10.617717 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 16:37:10.617717 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 16:37:10.617717 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 16:37:10.617717 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 16:37:10.617717 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application Aug 01 16:37:10.620521 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 395/782] 10.222.0.20 () {56 vars in 1096 bytes} [Mon Aug 1 16:37:10 2022] GET /identity/v3/roles/admin => generated 83 bytes in 277 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 01 16:37:10.625692 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5563d60-660b-45c2-8b65-53472723792c None admin] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:37:10.762917 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-870bfd56-2f61-4877-a688-1ed77283e4b9 None admin] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:37:10.812244 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd2d6c8-cfbe-4e69-8c58-7d751929730b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:37:10.813692 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd2d6c8-cfbe-4e69-8c58-7d751929730b None admin] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:37:10.813981 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd2d6c8-cfbe-4e69-8c58-7d751929730b None admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:37:10.814285 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd2d6c8-cfbe-4e69-8c58-7d751929730b None admin] PATH_INFO: `/v3/services` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:37:10.815725 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbd2d6c8-cfbe-4e69-8c58-7d751929730b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:37:10.816831 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbd2d6c8-cfbe-4e69-8c58-7d751929730b None admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:37:10.827210 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 388/783] 10.222.0.20 () {56 vars in 1087 bytes} [Mon Aug 1 16:37:10 2022] GET /identity/v3/services => generated 772 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 16:37:10.840557 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5563d60-660b-45c2-8b65-53472723792c None admin] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:37:10.888230 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e36b3c7e-4a36-41cc-b497-f21f7fa5fc18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:37:10.888975 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36b3c7e-4a36-41cc-b497-f21f7fa5fc18 None admin] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:37:10.889255 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36b3c7e-4a36-41cc-b497-f21f7fa5fc18 None admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:37:10.889523 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36b3c7e-4a36-41cc-b497-f21f7fa5fc18 None admin] PATH_INFO: `/v3/roles` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:37:10.890294 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e36b3c7e-4a36-41cc-b497-f21f7fa5fc18 None admin] RBAC: Adding query filter params (name=admin) {{(pid=87058) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:37:10.890596 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e36b3c7e-4a36-41cc-b497-f21f7fa5fc18 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:37:10.892405 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e36b3c7e-4a36-41cc-b497-f21f7fa5fc18 None admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:37:10.911187 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 396/784] 10.222.0.20 () {56 vars in 1099 bytes} [Mon Aug 1 16:37:10 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 286 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:37:10.918255 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd2d6c8-cfbe-4e69-8c58-7d751929730b None admin] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:37:11.069185 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG dbcounter [-] [87057] Writing DB stats keystone:SELECT=100 {{(pid=87057) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:37:11.111419 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd2d6c8-cfbe-4e69-8c58-7d751929730b None admin] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:37:11.150927 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-896c505c-4bb6-4991-a9fa-1e7a3d04f600 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:37:11.152519 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896c505c-4bb6-4991-a9fa-1e7a3d04f600 None admin] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:37:11.152988 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896c505c-4bb6-4991-a9fa-1e7a3d04f600 None admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:37:11.153471 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896c505c-4bb6-4991-a9fa-1e7a3d04f600 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:37:11.166585 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-896c505c-4bb6-4991-a9fa-1e7a3d04f600 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:37:11.168304 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-896c505c-4bb6-4991-a9fa-1e7a3d04f600 None admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:37:11.184369 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application [None req-896c505c-4bb6-4991-a9fa-1e7a3d04f600 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 16:37:11.184369 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 16:37:11.184369 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 16:37:11.184369 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 16:37:11.184369 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 16:37:11.184369 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 16:37:11.184369 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 16:37:11.184369 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 16:37:11.184369 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 16:37:11.184369 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 16:37:11.184369 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 16:37:11.184369 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 16:37:11.184369 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 16:37:11.184369 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 16:37:11.184369 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 16:37:11.185381 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 16:37:11.185381 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 16:37:11.185381 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 16:37:11.185381 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 16:37:11.185381 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 16:37:11.185381 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 16:37:11.185381 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 16:37:11.185381 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 16:37:11.185381 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application with Lock( Aug 01 16:37:11.185381 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 16:37:11.185381 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self._enter() Aug 01 16:37:11.185381 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 16:37:11.185381 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 16:37:11.185381 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 16:37:11.185381 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self.creator() Aug 01 16:37:11.185381 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 16:37:11.185381 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application created_value = creator( Aug 01 16:37:11.186312 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 16:37:11.186312 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 16:37:11.186312 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 16:37:11.186312 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application Aug 01 16:37:11.188462 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 389/785] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Aug 1 16:37:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 271 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 16:37:11.199447 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e36b3c7e-4a36-41cc-b497-f21f7fa5fc18 None admin] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:37:11.236070 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG dbcounter [-] [87058] Writing DB stats keystone:SELECT=100 {{(pid=87058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:37:11.329994 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e36b3c7e-4a36-41cc-b497-f21f7fa5fc18 None admin] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:37:11.374193 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52693d72-3045-4a1e-9792-18f5f2275f50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:37:11.374820 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52693d72-3045-4a1e-9792-18f5f2275f50 None admin] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:37:11.375100 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52693d72-3045-4a1e-9792-18f5f2275f50 None admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:37:11.375375 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52693d72-3045-4a1e-9792-18f5f2275f50 None admin] PATH_INFO: `/v3/domains` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:37:11.376072 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52693d72-3045-4a1e-9792-18f5f2275f50 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87058) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:37:11.376378 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52693d72-3045-4a1e-9792-18f5f2275f50 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:37:11.377738 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52693d72-3045-4a1e-9792-18f5f2275f50 None admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:37:11.393420 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 397/786] 10.222.0.20 () {56 vars in 1109 bytes} [Mon Aug 1 16:37:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:37:11.399050 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-896c505c-4bb6-4991-a9fa-1e7a3d04f600 None admin] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:37:11.444982 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d798fa09-74f2-4e79-8208-24ed5c8dbb63 None None] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:37:11.445289 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d798fa09-74f2-4e79-8208-24ed5c8dbb63 None None] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:37:11.445542 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d798fa09-74f2-4e79-8208-24ed5c8dbb63 None None] PATH_INFO: `/` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:37:11.447166 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 398/787] 10.222.0.20 () {54 vars in 926 bytes} [Mon Aug 1 16:37:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 01 16:37:11.452359 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743914c1-d0bc-45d6-b718-f06a3e4036e9 None None] REQUEST_METHOD: `POST` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:37:11.452643 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743914c1-d0bc-45d6-b718-f06a3e4036e9 None None] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:37:11.452920 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743914c1-d0bc-45d6-b718-f06a3e4036e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:37:11.481744 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.auth.core [None req-743914c1-d0bc-45d6-b718-f06a3e4036e9 None None] MFA Rules not processed for user `1007f8e379b74e169b98fb7c3c0ba5b4`. Rule list: `[]` (Enabled: `True`). {{(pid=87058) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:37:11.482977 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.fernet_utils [None req-743914c1-d0bc-45d6-b718-f06a3e4036e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:37:11.574021 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-896c505c-4bb6-4991-a9fa-1e7a3d04f600 None admin] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:37:11.601599 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG dbcounter [-] [87057] Writing DB stats keystone:SELECT=100 {{(pid=87057) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:37:11.606282 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 399/788] 10.222.0.20 () {58 vars in 1023 bytes} [Mon Aug 1 16:37:11 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 01 16:37:11.635751 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a022f4b-8cd0-4e34-bc12-3ab1becbea94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:37:11.636403 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a022f4b-8cd0-4e34-bc12-3ab1becbea94 None admin] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:37:11.636692 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a022f4b-8cd0-4e34-bc12-3ab1becbea94 None admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:37:11.636969 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a022f4b-8cd0-4e34-bc12-3ab1becbea94 None admin] PATH_INFO: `/v3/users/nova` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:37:11.648151 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a022f4b-8cd0-4e34-bc12-3ab1becbea94 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:37:11.649684 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a022f4b-8cd0-4e34-bc12-3ab1becbea94 None admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:37:11.666475 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application [None req-0a022f4b-8cd0-4e34-bc12-3ab1becbea94 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 01 16:37:11.666475 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 16:37:11.666475 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 16:37:11.666475 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 16:37:11.666475 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 16:37:11.666475 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 16:37:11.666475 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 16:37:11.666475 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 16:37:11.666475 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 16:37:11.666475 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 16:37:11.666475 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 16:37:11.666475 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 16:37:11.666475 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 01 16:37:11.666475 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 01 16:37:11.666475 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 01 16:37:11.666475 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 01 16:37:11.667454 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 16:37:11.667454 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 16:37:11.667454 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 01 16:37:11.667454 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 16:37:11.667454 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 01 16:37:11.667454 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 16:37:11.667454 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 16:37:11.667454 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 16:37:11.667454 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 16:37:11.667454 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 16:37:11.667454 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 16:37:11.667454 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application with Lock( Aug 01 16:37:11.667454 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 16:37:11.667454 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self._enter() Aug 01 16:37:11.667454 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 16:37:11.667454 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 16:37:11.667454 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 16:37:11.667454 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self.creator() Aug 01 16:37:11.668982 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 16:37:11.668982 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application created_value = creator( Aug 01 16:37:11.668982 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 01 16:37:11.668982 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 01 16:37:11.668982 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 01 16:37:11.668982 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 01 16:37:11.668982 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 01 16:37:11.668982 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 01 16:37:11.668982 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 01 16:37:11.668982 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application Aug 01 16:37:11.668982 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 390/789] 10.222.0.20 () {56 vars in 1093 bytes} [Mon Aug 1 16:37:11 2022] GET /identity/v3/users/nova => generated 82 bytes in 269 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 16:37:11.670622 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-743914c1-d0bc-45d6-b718-f06a3e4036e9 None None] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:37:11.674324 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a022f4b-8cd0-4e34-bc12-3ab1becbea94 None admin] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:37:11.699089 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG dbcounter [-] [87058] Writing DB stats keystone:SELECT=100 {{(pid=87058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:37:11.863123 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-743914c1-d0bc-45d6-b718-f06a3e4036e9 None None] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:37:11.883944 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a022f4b-8cd0-4e34-bc12-3ab1becbea94 None admin] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:37:11.940333 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e2f3352-31fb-4daf-b390-478bc2d8fcbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:37:11.940991 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2f3352-31fb-4daf-b390-478bc2d8fcbf None admin] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:37:11.941332 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2f3352-31fb-4daf-b390-478bc2d8fcbf None admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:37:11.941613 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2f3352-31fb-4daf-b390-478bc2d8fcbf None admin] PATH_INFO: `/v3/roles/service` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:37:11.952848 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e2f3352-31fb-4daf-b390-478bc2d8fcbf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 16:37:11.958240 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application [None req-0e2f3352-31fb-4daf-b390-478bc2d8fcbf None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 01 16:37:11.958240 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 16:37:11.958240 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 16:37:11.958240 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 16:37:11.958240 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 16:37:11.958240 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 16:37:11.958240 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 16:37:11.958240 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 16:37:11.958240 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 16:37:11.958240 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 16:37:11.958240 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 16:37:11.958240 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 16:37:11.958240 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 16:37:11.958240 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 16:37:11.958240 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 16:37:11.958240 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 16:37:11.959883 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 16:37:11.959883 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 16:37:11.959883 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 16:37:11.959883 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 16:37:11.959883 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 16:37:11.959883 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 16:37:11.959883 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 16:37:11.959883 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 16:37:11.959883 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 16:37:11.959883 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 16:37:11.959883 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 16:37:11.959883 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application with Lock( Aug 01 16:37:11.959883 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 16:37:11.959883 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self._enter() Aug 01 16:37:11.959883 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 16:37:11.959883 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 16:37:11.961137 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 16:37:11.961137 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self.creator() Aug 01 16:37:11.961137 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 16:37:11.961137 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application created_value = creator( Aug 01 16:37:11.961137 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 16:37:11.961137 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 16:37:11.961137 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 16:37:11.961137 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 16:37:11.961137 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 16:37:11.961137 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 16:37:11.961137 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 01 16:37:11.961137 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application Aug 01 16:37:11.961137 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 400/790] 10.222.0.20 () {56 vars in 1102 bytes} [Mon Aug 1 16:37:11 2022] GET /identity/v3/roles/service => generated 85 bytes in 290 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 16:37:11.966420 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e2f3352-31fb-4daf-b390-478bc2d8fcbf None admin] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:37:12.027399 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7468bafa-0a08-4fad-9d96-cfce494b7f72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:37:12.028038 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7468bafa-0a08-4fad-9d96-cfce494b7f72 None admin] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:37:12.028312 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7468bafa-0a08-4fad-9d96-cfce494b7f72 None admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:37:12.028577 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7468bafa-0a08-4fad-9d96-cfce494b7f72 None admin] PATH_INFO: `/v3/users` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:37:12.029382 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7468bafa-0a08-4fad-9d96-cfce494b7f72 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=87057) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:37:12.029685 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7468bafa-0a08-4fad-9d96-cfce494b7f72 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:37:12.031153 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7468bafa-0a08-4fad-9d96-cfce494b7f72 None admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:37:12.056303 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG dbcounter [-] [87058] Writing DB stats keystone:SELECT=100 {{(pid=87058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:37:12.069668 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 391/791] 10.222.0.20 () {56 vars in 1133 bytes} [Mon Aug 1 16:37:11 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 396 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:37:12.074873 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7468bafa-0a08-4fad-9d96-cfce494b7f72 None admin] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:37:12.080079 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e2f3352-31fb-4daf-b390-478bc2d8fcbf None admin] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:37:12.122385 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60de0948-d94d-42e9-95a0-0f624ca1218c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:37:12.122927 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60de0948-d94d-42e9-95a0-0f624ca1218c None admin] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:37:12.123205 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60de0948-d94d-42e9-95a0-0f624ca1218c None admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:37:12.123478 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60de0948-d94d-42e9-95a0-0f624ca1218c None admin] PATH_INFO: `/v3/roles` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:37:12.124762 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60de0948-d94d-42e9-95a0-0f624ca1218c None admin] RBAC: Adding query filter params (name=service) {{(pid=87058) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:37:12.125067 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60de0948-d94d-42e9-95a0-0f624ca1218c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:37:12.126466 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60de0948-d94d-42e9-95a0-0f624ca1218c None admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:37:12.136010 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 401/792] 10.222.0.20 () {56 vars in 1103 bytes} [Mon Aug 1 16:37:11 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 16:37:12.142063 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60de0948-d94d-42e9-95a0-0f624ca1218c None admin] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:37:12.159157 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG dbcounter [-] [87057] Writing DB stats keystone:SELECT=100 {{(pid=87057) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:37:12.233137 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7468bafa-0a08-4fad-9d96-cfce494b7f72 None admin] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:37:12.274761 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60de0948-d94d-42e9-95a0-0f624ca1218c None admin] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:37:12.281555 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-580a95f8-80f1-49c4-a0f6-bce27fa7e2e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:37:12.282717 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580a95f8-80f1-49c4-a0f6-bce27fa7e2e3 None admin] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:37:12.282996 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580a95f8-80f1-49c4-a0f6-bce27fa7e2e3 None admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:37:12.283265 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580a95f8-80f1-49c4-a0f6-bce27fa7e2e3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:37:12.321974 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe033ea5-a72a-4999-a222-d9ae93ec5267 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:37:12.322921 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe033ea5-a72a-4999-a222-d9ae93ec5267 None admin] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:37:12.323202 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe033ea5-a72a-4999-a222-d9ae93ec5267 None admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:37:12.323476 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe033ea5-a72a-4999-a222-d9ae93ec5267 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:37:12.336148 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG dbcounter [-] [87058] Writing DB stats keystone:SELECT=100 {{(pid=87058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:37:12.340143 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe033ea5-a72a-4999-a222-d9ae93ec5267 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:37:12.342020 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe033ea5-a72a-4999-a222-d9ae93ec5267 None admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:37:12.361281 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-580a95f8-80f1-49c4-a0f6-bce27fa7e2e3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:37:12.362359 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application [None req-fe033ea5-a72a-4999-a222-d9ae93ec5267 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 16:37:12.362359 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 16:37:12.362359 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 16:37:12.362359 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 16:37:12.362359 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 16:37:12.362359 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 16:37:12.362359 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 16:37:12.362359 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 16:37:12.362359 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 16:37:12.362359 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 16:37:12.362359 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 16:37:12.362359 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 16:37:12.362359 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 16:37:12.362359 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 16:37:12.362359 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 16:37:12.364324 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 16:37:12.364324 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 16:37:12.364324 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 16:37:12.364324 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 16:37:12.364324 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 16:37:12.364324 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 16:37:12.364324 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 16:37:12.364324 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 16:37:12.364324 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application with Lock( Aug 01 16:37:12.364324 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 16:37:12.364324 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self._enter() Aug 01 16:37:12.364324 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 16:37:12.364324 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 16:37:12.364324 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 16:37:12.364324 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self.creator() Aug 01 16:37:12.364324 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 16:37:12.364324 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application created_value = creator( Aug 01 16:37:12.367248 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 16:37:12.367248 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 16:37:12.367248 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 16:37:12.367248 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application Aug 01 16:37:12.367788 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-580a95f8-80f1-49c4-a0f6-bce27fa7e2e3 None admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:37:12.373097 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 402/793] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Aug 1 16:37:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 16:37:12.373097 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eac9789-96f2-43d9-a6c2-f8d9d6b2a4e2 None None] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:37:12.373097 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eac9789-96f2-43d9-a6c2-f8d9d6b2a4e2 None None] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:37:12.373097 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eac9789-96f2-43d9-a6c2-f8d9d6b2a4e2 None None] PATH_INFO: `/` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:37:12.373097 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 403/794] 10.222.0.20 () {54 vars in 926 bytes} [Mon Aug 1 16:37:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 16:37:12.373866 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eac9789-96f2-43d9-a6c2-f8d9d6b2a4e2 None None] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:37:12.403778 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG dbcounter [-] [87057] Writing DB stats keystone:SELECT=100 {{(pid=87057) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:37:12.485921 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 392/795] 10.222.0.20 () {58 vars in 1282 bytes} [Mon Aug 1 16:37:12 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 411 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 01 16:37:12.487638 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ae7715-8aba-41d6-b173-3859ccec3f02 None None] REQUEST_METHOD: `POST` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:37:12.488153 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ae7715-8aba-41d6-b173-3859ccec3f02 None None] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:37:12.488649 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ae7715-8aba-41d6-b173-3859ccec3f02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:37:12.506009 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eac9789-96f2-43d9-a6c2-f8d9d6b2a4e2 None None] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:37:12.507360 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.auth.core [None req-33ae7715-8aba-41d6-b173-3859ccec3f02 None None] MFA Rules not processed for user `1007f8e379b74e169b98fb7c3c0ba5b4`. Rule list: `[]` (Enabled: `True`). {{(pid=87057) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:37:12.509673 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.fernet_utils [None req-33ae7715-8aba-41d6-b173-3859ccec3f02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87057) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:37:12.541045 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de6ffe92-21b1-4320-8050-4586818fb98b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:37:12.541663 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6ffe92-21b1-4320-8050-4586818fb98b None admin] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:37:12.541943 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6ffe92-21b1-4320-8050-4586818fb98b None admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:37:12.542242 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6ffe92-21b1-4320-8050-4586818fb98b None admin] PATH_INFO: `/v3/domains` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:37:12.542929 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de6ffe92-21b1-4320-8050-4586818fb98b None admin] RBAC: Adding query filter params (name=Default) {{(pid=87058) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:37:12.543240 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de6ffe92-21b1-4320-8050-4586818fb98b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:37:12.544583 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de6ffe92-21b1-4320-8050-4586818fb98b None admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:37:12.557111 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 404/796] 10.222.0.20 () {56 vars in 1109 bytes} [Mon Aug 1 16:37:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:37:12.558324 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-655b7095-db86-4c30-a173-7ebc3fcd0214 None None] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:37:12.558606 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-655b7095-db86-4c30-a173-7ebc3fcd0214 None None] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:37:12.558875 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-655b7095-db86-4c30-a173-7ebc3fcd0214 None None] PATH_INFO: `/` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:37:12.564116 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 405/797] 10.222.0.20 () {54 vars in 926 bytes} [Mon Aug 1 16:37:12 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 16:37:12.564866 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-655b7095-db86-4c30-a173-7ebc3fcd0214 None None] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:37:12.598176 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 393/798] 10.222.0.20 () {58 vars in 1023 bytes} [Mon Aug 1 16:37:12 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 01 16:37:12.599255 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ae7715-8aba-41d6-b173-3859ccec3f02 None None] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:37:12.624171 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG dbcounter [-] [87058] Writing DB stats keystone:SELECT=100 {{(pid=87058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:37:12.671879 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG dbcounter [-] [87057] Writing DB stats keystone:SELECT=100 {{(pid=87057) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:37:12.675452 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-655b7095-db86-4c30-a173-7ebc3fcd0214 None None] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:37:12.712487 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-407d269c-567a-4c11-b088-4fd5c3b71dcd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:37:12.713126 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407d269c-567a-4c11-b088-4fd5c3b71dcd None admin] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:37:12.713405 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407d269c-567a-4c11-b088-4fd5c3b71dcd None admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:37:12.713681 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407d269c-567a-4c11-b088-4fd5c3b71dcd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:37:12.714825 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ae7715-8aba-41d6-b173-3859ccec3f02 None None] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:37:12.726130 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-407d269c-567a-4c11-b088-4fd5c3b71dcd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:37:12.727589 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-407d269c-567a-4c11-b088-4fd5c3b71dcd None admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:37:12.743231 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application [None req-407d269c-567a-4c11-b088-4fd5c3b71dcd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 16:37:12.743231 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 16:37:12.743231 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 16:37:12.743231 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 16:37:12.743231 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 16:37:12.743231 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 16:37:12.743231 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 16:37:12.743231 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 16:37:12.743231 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 16:37:12.743231 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 16:37:12.743231 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 16:37:12.743231 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 16:37:12.743231 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 16:37:12.743231 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 16:37:12.743231 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 16:37:12.744182 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 16:37:12.744182 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 16:37:12.744182 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 16:37:12.744182 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 16:37:12.744182 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 16:37:12.744182 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 16:37:12.744182 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 16:37:12.744182 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 16:37:12.744182 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application with Lock( Aug 01 16:37:12.744182 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 16:37:12.744182 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self._enter() Aug 01 16:37:12.744182 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 16:37:12.744182 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 16:37:12.744182 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 16:37:12.744182 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self.creator() Aug 01 16:37:12.744182 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 16:37:12.744182 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application created_value = creator( Aug 01 16:37:12.745068 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 16:37:12.745068 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 16:37:12.745068 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 16:37:12.745068 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application Aug 01 16:37:12.749188 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 406/799] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Aug 1 16:37:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 16:37:12.750658 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f08e3c-7179-4ba3-91d8-935d28e1ed3b None None] REQUEST_METHOD: `POST` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:37:12.750954 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f08e3c-7179-4ba3-91d8-935d28e1ed3b None None] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:37:12.751284 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f08e3c-7179-4ba3-91d8-935d28e1ed3b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:37:12.752010 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ef51a6-830e-4f2b-ac04-6cef367cf29d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:37:12.752678 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ef51a6-830e-4f2b-ac04-6cef367cf29d None admin] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:37:12.752958 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ef51a6-830e-4f2b-ac04-6cef367cf29d None admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:37:12.753229 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ef51a6-830e-4f2b-ac04-6cef367cf29d None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:37:12.762805 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6ef51a6-830e-4f2b-ac04-6cef367cf29d None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:37:12.764243 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6ef51a6-830e-4f2b-ac04-6cef367cf29d None admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:37:12.770829 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.auth.core [None req-b2f08e3c-7179-4ba3-91d8-935d28e1ed3b None None] MFA Rules not processed for user `1007f8e379b74e169b98fb7c3c0ba5b4`. Rule list: `[]` (Enabled: `True`). {{(pid=87058) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:37:12.772054 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.fernet_utils [None req-b2f08e3c-7179-4ba3-91d8-935d28e1ed3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:37:12.776488 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application [None req-b6ef51a6-830e-4f2b-ac04-6cef367cf29d None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 01 16:37:12.776488 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 16:37:12.776488 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 16:37:12.776488 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 16:37:12.776488 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 16:37:12.776488 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 16:37:12.776488 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 16:37:12.776488 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 16:37:12.776488 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 16:37:12.776488 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 16:37:12.776488 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 16:37:12.776488 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 16:37:12.776488 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 01 16:37:12.776488 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 01 16:37:12.776488 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 01 16:37:12.776488 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 01 16:37:12.777435 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 16:37:12.777435 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 16:37:12.777435 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 01 16:37:12.777435 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 16:37:12.777435 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 01 16:37:12.777435 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 16:37:12.777435 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 16:37:12.777435 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 16:37:12.777435 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 16:37:12.777435 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 16:37:12.777435 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 16:37:12.777435 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application with Lock( Aug 01 16:37:12.777435 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 16:37:12.777435 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self._enter() Aug 01 16:37:12.777435 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 16:37:12.777435 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 16:37:12.777435 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 16:37:12.777435 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self.creator() Aug 01 16:37:12.783884 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 16:37:12.783884 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application created_value = creator( Aug 01 16:37:12.783884 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 01 16:37:12.783884 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 01 16:37:12.783884 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 01 16:37:12.783884 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 01 16:37:12.783884 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 01 16:37:12.783884 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 01 16:37:12.783884 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 01 16:37:12.783884 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application Aug 01 16:37:12.783884 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 394/800] 10.222.0.20 () {56 vars in 1117 bytes} [Mon Aug 1 16:37:12 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 16:37:12.783884 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ef51a6-830e-4f2b-ac04-6cef367cf29d None admin] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:37:12.848218 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 407/801] 10.222.0.20 () {58 vars in 1023 bytes} [Mon Aug 1 16:37:12 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 01 16:37:12.850110 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f08e3c-7179-4ba3-91d8-935d28e1ed3b None None] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:37:12.884653 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG dbcounter [-] [87058] Writing DB stats keystone:SELECT=100 {{(pid=87058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:37:12.896694 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ef51a6-830e-4f2b-ac04-6cef367cf29d None admin] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:37:12.927640 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG dbcounter [-] [87057] Writing DB stats keystone:SELECT=100 {{(pid=87057) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:37:12.943644 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb63e32-13e5-49ce-9fdf-c34f027f3b09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:37:12.944296 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb63e32-13e5-49ce-9fdf-c34f027f3b09 None admin] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:37:12.944581 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb63e32-13e5-49ce-9fdf-c34f027f3b09 None admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:37:12.944855 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb63e32-13e5-49ce-9fdf-c34f027f3b09 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:37:12.954507 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f08e3c-7179-4ba3-91d8-935d28e1ed3b None None] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:37:12.957166 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fb63e32-13e5-49ce-9fdf-c34f027f3b09 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 16:37:12.962744 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application [None req-2fb63e32-13e5-49ce-9fdf-c34f027f3b09 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 16:37:12.962744 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 16:37:12.962744 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 16:37:12.962744 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 16:37:12.962744 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 16:37:12.962744 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 16:37:12.962744 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 16:37:12.962744 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 16:37:12.962744 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 16:37:12.962744 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 16:37:12.962744 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 16:37:12.962744 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 16:37:12.962744 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 16:37:12.962744 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 16:37:12.962744 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 16:37:12.962744 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 16:37:12.963706 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 16:37:12.963706 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 16:37:12.963706 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 16:37:12.963706 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 16:37:12.963706 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 16:37:12.963706 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 16:37:12.963706 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 16:37:12.963706 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 16:37:12.963706 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 16:37:12.963706 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 16:37:12.963706 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 16:37:12.963706 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application with Lock( Aug 01 16:37:12.963706 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 16:37:12.963706 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self._enter() Aug 01 16:37:12.963706 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 16:37:12.963706 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 16:37:12.964542 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 16:37:12.964542 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self.creator() Aug 01 16:37:12.964542 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 16:37:12.964542 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application created_value = creator( Aug 01 16:37:12.964542 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 16:37:12.964542 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 16:37:12.964542 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 16:37:12.964542 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 16:37:12.964542 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 16:37:12.964542 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 16:37:12.964542 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 16:37:12.964542 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application Aug 01 16:37:12.964542 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 395/802] 10.222.0.20 () {56 vars in 1096 bytes} [Mon Aug 1 16:37:12 2022] GET /identity/v3/roles/admin => generated 83 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 16:37:12.965448 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb63e32-13e5-49ce-9fdf-c34f027f3b09 None admin] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:37:13.003163 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17af916a-64e1-442b-a33a-d68e5256bb43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:37:13.004392 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17af916a-64e1-442b-a33a-d68e5256bb43 None admin] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:37:13.004392 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17af916a-64e1-442b-a33a-d68e5256bb43 None admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:37:13.004392 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17af916a-64e1-442b-a33a-d68e5256bb43 None admin] PATH_INFO: `/v3/domains` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:37:13.004994 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17af916a-64e1-442b-a33a-d68e5256bb43 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87058) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:37:13.005296 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17af916a-64e1-442b-a33a-d68e5256bb43 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:37:13.006815 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17af916a-64e1-442b-a33a-d68e5256bb43 None admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:37:13.018795 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 408/803] 10.222.0.20 () {56 vars in 1109 bytes} [Mon Aug 1 16:37:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:37:13.019650 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17af916a-64e1-442b-a33a-d68e5256bb43 None admin] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:37:13.080159 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb63e32-13e5-49ce-9fdf-c34f027f3b09 None admin] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:37:13.115907 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-485f9002-b317-4cac-a741-88430c70a8af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:37:13.116516 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485f9002-b317-4cac-a741-88430c70a8af None admin] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:37:13.116792 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485f9002-b317-4cac-a741-88430c70a8af None admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:37:13.117060 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485f9002-b317-4cac-a741-88430c70a8af None admin] PATH_INFO: `/v3/users` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:37:13.117846 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-485f9002-b317-4cac-a741-88430c70a8af None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=87057) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:37:13.118186 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-485f9002-b317-4cac-a741-88430c70a8af None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:37:13.119611 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-485f9002-b317-4cac-a741-88430c70a8af None admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:37:13.124472 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG dbcounter [-] [87058] Writing DB stats keystone:SELECT=100 {{(pid=87058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:37:13.132221 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17af916a-64e1-442b-a33a-d68e5256bb43 None admin] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:37:13.143415 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 396/804] 10.222.0.20 () {56 vars in 1149 bytes} [Mon Aug 1 16:37:12 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:37:13.143415 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-485f9002-b317-4cac-a741-88430c70a8af None admin] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:37:13.173671 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f39a6f8-d814-43a7-94bb-4d6a0781abea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:37:13.174342 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f39a6f8-d814-43a7-94bb-4d6a0781abea None admin] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:37:13.174620 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f39a6f8-d814-43a7-94bb-4d6a0781abea None admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:37:13.174893 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f39a6f8-d814-43a7-94bb-4d6a0781abea None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:37:13.175515 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f39a6f8-d814-43a7-94bb-4d6a0781abea None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:37:13.176873 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f39a6f8-d814-43a7-94bb-4d6a0781abea None admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:37:13.186773 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application [None req-7f39a6f8-d814-43a7-94bb-4d6a0781abea None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 01 16:37:13.186773 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 16:37:13.186773 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 01 16:37:13.186773 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 01 16:37:13.186773 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 01 16:37:13.186773 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 01 16:37:13.186773 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 01 16:37:13.186773 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 01 16:37:13.186773 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 01 16:37:13.186773 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application Aug 01 16:37:13.186773 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 01 16:37:13.186773 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application Aug 01 16:37:13.186773 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 16:37:13.186773 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 16:37:13.186773 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 16:37:13.186773 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 16:37:13.187768 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 16:37:13.187768 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 16:37:13.187768 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 16:37:13.187768 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 16:37:13.187768 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 16:37:13.187768 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 16:37:13.187768 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 16:37:13.187768 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 01 16:37:13.187768 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 01 16:37:13.187768 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 01 16:37:13.187768 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 01 16:37:13.187768 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 16:37:13.187768 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 16:37:13.187768 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 16:37:13.187768 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 16:37:13.187768 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 16:37:13.188629 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 16:37:13.188629 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 16:37:13.188629 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application with Lock( Aug 01 16:37:13.188629 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 16:37:13.188629 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self._enter() Aug 01 16:37:13.188629 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 16:37:13.188629 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 16:37:13.188629 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 16:37:13.188629 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self.creator() Aug 01 16:37:13.188629 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 16:37:13.188629 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application created_value = creator( Aug 01 16:37:13.188629 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 01 16:37:13.188629 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 01 16:37:13.188629 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 01 16:37:13.188629 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application Aug 01 16:37:13.188629 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 409/805] 10.222.0.20 () {56 vars in 1129 bytes} [Mon Aug 1 16:37:13 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 16:37:13.189489 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f39a6f8-d814-43a7-94bb-4d6a0781abea None admin] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:37:13.197864 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG dbcounter [-] [87057] Writing DB stats keystone:SELECT=100 {{(pid=87057) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:37:13.268992 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-485f9002-b317-4cac-a741-88430c70a8af None admin] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:37:13.297035 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f39a6f8-d814-43a7-94bb-4d6a0781abea None admin] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:37:13.305724 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a5b9da8-a956-4039-bda3-ea49ec6db07a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:37:13.306387 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5b9da8-a956-4039-bda3-ea49ec6db07a None admin] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:37:13.306666 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5b9da8-a956-4039-bda3-ea49ec6db07a None admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:37:13.306940 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5b9da8-a956-4039-bda3-ea49ec6db07a None admin] PATH_INFO: `/v3/roles` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:37:13.307663 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a5b9da8-a956-4039-bda3-ea49ec6db07a None admin] RBAC: Adding query filter params (name=admin) {{(pid=87057) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:37:13.308131 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a5b9da8-a956-4039-bda3-ea49ec6db07a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:37:13.309513 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a5b9da8-a956-4039-bda3-ea49ec6db07a None admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:37:13.318883 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 397/806] 10.222.0.20 () {56 vars in 1099 bytes} [Mon Aug 1 16:37:13 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:37:13.319573 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a5b9da8-a956-4039-bda3-ea49ec6db07a None admin] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:37:13.333579 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9deadf65-db13-436f-99b6-2350936ff913 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:37:13.334876 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9deadf65-db13-436f-99b6-2350936ff913 None admin] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:37:13.335158 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9deadf65-db13-436f-99b6-2350936ff913 None admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:37:13.335434 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9deadf65-db13-436f-99b6-2350936ff913 None admin] PATH_INFO: `/v3/projects/service` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:37:13.342221 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9deadf65-db13-436f-99b6-2350936ff913 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:37:13.343671 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9deadf65-db13-436f-99b6-2350936ff913 None admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:37:13.355522 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application [None req-9deadf65-db13-436f-99b6-2350936ff913 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 16:37:13.355522 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 16:37:13.355522 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 16:37:13.355522 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 16:37:13.355522 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 16:37:13.355522 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 16:37:13.355522 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 16:37:13.355522 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 16:37:13.355522 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 16:37:13.355522 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 16:37:13.355522 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 16:37:13.355522 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 16:37:13.355522 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 01 16:37:13.355522 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 01 16:37:13.355522 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 01 16:37:13.361520 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 01 16:37:13.361520 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 16:37:13.361520 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 16:37:13.361520 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 16:37:13.361520 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 16:37:13.361520 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 16:37:13.361520 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 16:37:13.361520 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 16:37:13.361520 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application with Lock( Aug 01 16:37:13.361520 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 16:37:13.361520 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self._enter() Aug 01 16:37:13.361520 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 16:37:13.361520 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 16:37:13.361520 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 16:37:13.361520 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self.creator() Aug 01 16:37:13.361520 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 16:37:13.361520 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application created_value = creator( Aug 01 16:37:13.362457 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 01 16:37:13.362457 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 01 16:37:13.362457 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 01 16:37:13.362457 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 01 16:37:13.362457 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 01 16:37:13.362457 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 01 16:37:13.362457 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 16:37:13.362457 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application Aug 01 16:37:13.362457 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 410/807] 10.222.0.20 () {56 vars in 1111 bytes} [Mon Aug 1 16:37:13 2022] GET /identity/v3/projects/service => generated 88 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 16:37:13.362457 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9deadf65-db13-436f-99b6-2350936ff913 None admin] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:37:13.379862 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG dbcounter [-] [87058] Writing DB stats keystone:SELECT=100 {{(pid=87058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:37:13.424985 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a5b9da8-a956-4039-bda3-ea49ec6db07a None admin] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:37:13.426460 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG dbcounter [-] [87057] Writing DB stats keystone:SELECT=100 {{(pid=87057) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:37:13.459331 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9deadf65-db13-436f-99b6-2350936ff913 None admin] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:37:13.460440 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe9f694e-0d3d-4555-bcb5-32b3528567d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:37:13.461076 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9f694e-0d3d-4555-bcb5-32b3528567d5 None admin] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:37:13.461351 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9f694e-0d3d-4555-bcb5-32b3528567d5 None admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:37:13.461617 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9f694e-0d3d-4555-bcb5-32b3528567d5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:37:13.471419 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe9f694e-0d3d-4555-bcb5-32b3528567d5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:37:13.472849 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe9f694e-0d3d-4555-bcb5-32b3528567d5 None admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:37:13.487711 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application [None req-fe9f694e-0d3d-4555-bcb5-32b3528567d5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 16:37:13.487711 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 16:37:13.487711 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 16:37:13.487711 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 16:37:13.487711 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 16:37:13.487711 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 16:37:13.487711 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 16:37:13.487711 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 16:37:13.487711 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 16:37:13.487711 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 16:37:13.487711 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 16:37:13.487711 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 16:37:13.487711 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 16:37:13.487711 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 16:37:13.487711 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 16:37:13.488738 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 16:37:13.488738 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 16:37:13.488738 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 16:37:13.488738 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 16:37:13.488738 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 16:37:13.488738 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 16:37:13.488738 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 16:37:13.488738 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 16:37:13.488738 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application with Lock( Aug 01 16:37:13.488738 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 16:37:13.488738 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self._enter() Aug 01 16:37:13.488738 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 16:37:13.488738 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 16:37:13.488738 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 16:37:13.488738 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self.creator() Aug 01 16:37:13.488738 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 16:37:13.488738 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application created_value = creator( Aug 01 16:37:13.489639 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 16:37:13.489639 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 16:37:13.489639 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 16:37:13.489639 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application Aug 01 16:37:13.489639 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 398/808] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Aug 1 16:37:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 16:37:13.489639 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe9f694e-0d3d-4555-bcb5-32b3528567d5 None admin] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:37:13.498252 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffdd4f12-ab13-4eb3-97cf-afd502a348d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:37:13.498877 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffdd4f12-ab13-4eb3-97cf-afd502a348d7 None admin] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:37:13.499159 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffdd4f12-ab13-4eb3-97cf-afd502a348d7 None admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:37:13.499430 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffdd4f12-ab13-4eb3-97cf-afd502a348d7 None admin] PATH_INFO: `/v3/services` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:37:13.500112 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffdd4f12-ab13-4eb3-97cf-afd502a348d7 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=87058) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:37:13.500417 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffdd4f12-ab13-4eb3-97cf-afd502a348d7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:37:13.501779 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffdd4f12-ab13-4eb3-97cf-afd502a348d7 None admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:37:13.507287 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 411/809] 10.222.0.20 () {56 vars in 1124 bytes} [Mon Aug 1 16:37:13 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:37:13.508441 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9493082-8d94-4826-88bd-d2aacf900c6f None None] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:37:13.508725 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9493082-8d94-4826-88bd-d2aacf900c6f None None] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:37:13.509003 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9493082-8d94-4826-88bd-d2aacf900c6f None None] PATH_INFO: `/` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:37:13.512633 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 412/810] 10.222.0.20 () {54 vars in 926 bytes} [Mon Aug 1 16:37:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 16:37:13.512633 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9493082-8d94-4826-88bd-d2aacf900c6f None None] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:37:13.588457 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe9f694e-0d3d-4555-bcb5-32b3528567d5 None admin] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:37:13.594758 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG dbcounter [-] [87058] Writing DB stats keystone:SELECT=100 {{(pid=87058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:37:13.613268 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9493082-8d94-4826-88bd-d2aacf900c6f None None] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:37:13.621999 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f1e8cef-2113-4c51-ac4f-70554299ac01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:37:13.622667 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f1e8cef-2113-4c51-ac4f-70554299ac01 None admin] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:37:13.622958 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f1e8cef-2113-4c51-ac4f-70554299ac01 None admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:37:13.623234 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f1e8cef-2113-4c51-ac4f-70554299ac01 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:37:13.634371 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f1e8cef-2113-4c51-ac4f-70554299ac01 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:37:13.635823 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f1e8cef-2113-4c51-ac4f-70554299ac01 None admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:37:13.648030 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-558fe56f-31b5-4db4-8e88-2b78b2b39130 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:37:13.648638 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558fe56f-31b5-4db4-8e88-2b78b2b39130 None admin] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:37:13.648915 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558fe56f-31b5-4db4-8e88-2b78b2b39130 None admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:37:13.649192 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558fe56f-31b5-4db4-8e88-2b78b2b39130 None admin] PATH_INFO: `/v3/projects` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:37:13.649935 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-558fe56f-31b5-4db4-8e88-2b78b2b39130 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87058) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:37:13.650275 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-558fe56f-31b5-4db4-8e88-2b78b2b39130 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:37:13.650476 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application [None req-8f1e8cef-2113-4c51-ac4f-70554299ac01 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 16:37:13.650476 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 16:37:13.650476 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 16:37:13.650476 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 16:37:13.650476 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 16:37:13.650476 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 16:37:13.650476 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 16:37:13.650476 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 16:37:13.650476 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 16:37:13.650476 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 16:37:13.650476 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 16:37:13.650476 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 16:37:13.650476 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 16:37:13.650476 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 16:37:13.650476 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 16:37:13.651445 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 16:37:13.651445 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 16:37:13.651445 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 16:37:13.651445 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 16:37:13.651445 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 16:37:13.651445 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 16:37:13.651445 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 16:37:13.651445 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 16:37:13.651445 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application with Lock( Aug 01 16:37:13.651445 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 16:37:13.651445 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self._enter() Aug 01 16:37:13.651445 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 16:37:13.651445 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 16:37:13.651445 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 16:37:13.651445 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self.creator() Aug 01 16:37:13.651445 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 16:37:13.651445 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application created_value = creator( Aug 01 16:37:13.652358 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 16:37:13.652358 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 16:37:13.652358 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 16:37:13.652358 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application Aug 01 16:37:13.652358 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 399/811] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Aug 1 16:37:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 16:37:13.652644 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-558fe56f-31b5-4db4-8e88-2b78b2b39130 None admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:37:13.652716 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f1e8cef-2113-4c51-ac4f-70554299ac01 None admin] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:37:13.663155 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 413/812] 10.222.0.20 () {56 vars in 1148 bytes} [Mon Aug 1 16:37:13 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:37:13.664023 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-558fe56f-31b5-4db4-8e88-2b78b2b39130 None admin] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:37:13.666426 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG dbcounter [-] [87057] Writing DB stats keystone:SELECT=100 {{(pid=87057) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:37:13.756130 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f1e8cef-2113-4c51-ac4f-70554299ac01 None admin] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:37:13.764454 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-558fe56f-31b5-4db4-8e88-2b78b2b39130 None admin] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:37:13.788441 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-881d5a05-e0ac-4830-b020-26932a5fd770 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:37:13.789051 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881d5a05-e0ac-4830-b020-26932a5fd770 None admin] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:37:13.789336 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881d5a05-e0ac-4830-b020-26932a5fd770 None admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:37:13.789616 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881d5a05-e0ac-4830-b020-26932a5fd770 None admin] PATH_INFO: `/v3/domains` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:37:13.790340 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-881d5a05-e0ac-4830-b020-26932a5fd770 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87057) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:37:13.790649 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-881d5a05-e0ac-4830-b020-26932a5fd770 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:37:13.792009 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-881d5a05-e0ac-4830-b020-26932a5fd770 None admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:37:13.797969 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a186a06-8fa9-453b-8905-00dddbcf4f3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:37:13.798627 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a186a06-8fa9-453b-8905-00dddbcf4f3d None admin] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:37:13.798910 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a186a06-8fa9-453b-8905-00dddbcf4f3d None admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:37:13.799186 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a186a06-8fa9-453b-8905-00dddbcf4f3d None admin] PATH_INFO: `/v3/services` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:37:13.799863 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a186a06-8fa9-453b-8905-00dddbcf4f3d None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=87058) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:37:13.800167 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a186a06-8fa9-453b-8905-00dddbcf4f3d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:37:13.801517 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a186a06-8fa9-453b-8905-00dddbcf4f3d None admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:37:13.804328 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 400/813] 10.222.0.20 () {56 vars in 1109 bytes} [Mon Aug 1 16:37:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:37:13.805512 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07861f9b-5bc9-4088-a55c-bce03a12c323 None None] REQUEST_METHOD: `POST` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:37:13.805798 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07861f9b-5bc9-4088-a55c-bce03a12c323 None None] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:37:13.806091 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07861f9b-5bc9-4088-a55c-bce03a12c323 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:37:13.807046 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 414/814] 10.222.0.20 () {56 vars in 1124 bytes} [Mon Aug 1 16:37:13 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:37:13.807667 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a186a06-8fa9-453b-8905-00dddbcf4f3d None admin] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:37:13.816893 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG dbcounter [-] [87058] Writing DB stats keystone:SELECT=100 {{(pid=87058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:37:13.825820 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.auth.core [None req-07861f9b-5bc9-4088-a55c-bce03a12c323 None None] MFA Rules not processed for user `1007f8e379b74e169b98fb7c3c0ba5b4`. Rule list: `[]` (Enabled: `True`). {{(pid=87057) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:37:13.827055 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.fernet_utils [None req-07861f9b-5bc9-4088-a55c-bce03a12c323 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87057) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:37:13.885560 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG dbcounter [-] [87057] Writing DB stats keystone:SELECT=100 {{(pid=87057) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:37:13.892870 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 401/815] 10.222.0.20 () {58 vars in 1023 bytes} [Mon Aug 1 16:37:13 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 01 16:37:13.893673 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07861f9b-5bc9-4088-a55c-bce03a12c323 None None] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:37:13.922813 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a186a06-8fa9-453b-8905-00dddbcf4f3d None admin] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:37:13.957592 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0537cbe2-2723-4ea7-a35a-22f6f12cd7b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:37:13.958236 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0537cbe2-2723-4ea7-a35a-22f6f12cd7b0 None admin] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:37:13.958515 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0537cbe2-2723-4ea7-a35a-22f6f12cd7b0 None admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:37:13.958959 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0537cbe2-2723-4ea7-a35a-22f6f12cd7b0 None admin] PATH_INFO: `/v3/domains` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:37:13.959718 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0537cbe2-2723-4ea7-a35a-22f6f12cd7b0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87058) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:37:13.960023 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0537cbe2-2723-4ea7-a35a-22f6f12cd7b0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:37:13.961386 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0537cbe2-2723-4ea7-a35a-22f6f12cd7b0 None admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:37:13.973102 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 415/816] 10.222.0.20 () {56 vars in 1109 bytes} [Mon Aug 1 16:37:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:37:13.973902 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0537cbe2-2723-4ea7-a35a-22f6f12cd7b0 None admin] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:37:13.991259 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07861f9b-5bc9-4088-a55c-bce03a12c323 None None] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:37:14.025924 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffdc55bc-09b9-4a5f-855e-b9e0495ffa39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:37:14.026572 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffdc55bc-09b9-4a5f-855e-b9e0495ffa39 None admin] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:37:14.026855 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffdc55bc-09b9-4a5f-855e-b9e0495ffa39 None admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:37:14.027129 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffdc55bc-09b9-4a5f-855e-b9e0495ffa39 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:37:14.027915 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffdc55bc-09b9-4a5f-855e-b9e0495ffa39 None admin] RBAC: Adding query filter params (role.id=440bb943dd5b46fea038b903b6eaaf81, scope.project.id=7b992d6a65df4bf9b27c13192d57c5f0, user.id=6533106d03eb4ca185a8839336d2aff9) {{(pid=87057) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:37:14.028222 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffdc55bc-09b9-4a5f-855e-b9e0495ffa39 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:37:14.029647 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffdc55bc-09b9-4a5f-855e-b9e0495ffa39 None admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:37:14.035246 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 402/817] 10.222.0.20 () {56 vars in 1374 bytes} [Mon Aug 1 16:37:13 2022] GET /identity/v3/role_assignments?user.id=6533106d03eb4ca185a8839336d2aff9&scope.project.id=7b992d6a65df4bf9b27c13192d57c5f0&role.id=440bb943dd5b46fea038b903b6eaaf81 => generated 608 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:37:14.035998 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffdc55bc-09b9-4a5f-855e-b9e0495ffa39 None admin] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:37:14.040856 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG dbcounter [-] [87058] Writing DB stats keystone:SELECT=100 {{(pid=87058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:37:14.079576 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0537cbe2-2723-4ea7-a35a-22f6f12cd7b0 None admin] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:37:14.108653 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG dbcounter [-] [87057] Writing DB stats keystone:SELECT=100 {{(pid=87057) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:37:14.119990 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1cc06fd-c8c1-402d-ba12-ab1ca83c98d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:37:14.120864 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1cc06fd-c8c1-402d-ba12-ab1ca83c98d7 None admin] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:37:14.120964 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1cc06fd-c8c1-402d-ba12-ab1ca83c98d7 None admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:37:14.121179 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1cc06fd-c8c1-402d-ba12-ab1ca83c98d7 None admin] PATH_INFO: `/v3/projects/service` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:37:14.128894 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1cc06fd-c8c1-402d-ba12-ab1ca83c98d7 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:37:14.130402 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1cc06fd-c8c1-402d-ba12-ab1ca83c98d7 None admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:37:14.144833 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application [None req-a1cc06fd-c8c1-402d-ba12-ab1ca83c98d7 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 16:37:14.144833 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 16:37:14.144833 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 16:37:14.144833 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 16:37:14.144833 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 16:37:14.144833 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 16:37:14.144833 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 16:37:14.144833 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 16:37:14.144833 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 16:37:14.144833 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 16:37:14.144833 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 16:37:14.144833 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 16:37:14.144833 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 01 16:37:14.144833 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 01 16:37:14.144833 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 01 16:37:14.145841 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 01 16:37:14.145841 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 16:37:14.145841 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 16:37:14.145841 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 16:37:14.145841 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 16:37:14.145841 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 16:37:14.145841 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 16:37:14.145841 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 16:37:14.145841 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application with Lock( Aug 01 16:37:14.145841 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 16:37:14.145841 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self._enter() Aug 01 16:37:14.145841 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 16:37:14.145841 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 16:37:14.145841 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 16:37:14.145841 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self.creator() Aug 01 16:37:14.145841 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 16:37:14.145841 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application created_value = creator( Aug 01 16:37:14.147570 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 01 16:37:14.147570 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 01 16:37:14.147570 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 01 16:37:14.147570 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 01 16:37:14.147570 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 01 16:37:14.147570 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 01 16:37:14.147570 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 16:37:14.147570 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application Aug 01 16:37:14.147570 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 416/818] 10.222.0.20 () {56 vars in 1111 bytes} [Mon Aug 1 16:37:13 2022] GET /identity/v3/projects/service => generated 88 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 16:37:14.148098 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1cc06fd-c8c1-402d-ba12-ab1ca83c98d7 None admin] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:37:14.162637 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffdc55bc-09b9-4a5f-855e-b9e0495ffa39 None admin] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:37:14.203662 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06fa6932-0343-4fad-9a7f-8edd84b38fcb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:37:14.204284 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06fa6932-0343-4fad-9a7f-8edd84b38fcb None admin] REQUEST_METHOD: `POST` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:37:14.204562 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06fa6932-0343-4fad-9a7f-8edd84b38fcb None admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:37:14.204830 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06fa6932-0343-4fad-9a7f-8edd84b38fcb None admin] PATH_INFO: `/v3/endpoints` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:37:14.205548 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06fa6932-0343-4fad-9a7f-8edd84b38fcb None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:37:14.206944 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06fa6932-0343-4fad-9a7f-8edd84b38fcb None admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:37:14.260097 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1cc06fd-c8c1-402d-ba12-ab1ca83c98d7 None admin] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:37:14.286291 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG dbcounter [-] [87058] Writing DB stats keystone:SELECT=100 {{(pid=87058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:37:14.298072 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 403/819] 10.222.0.20 () {60 vars in 1144 bytes} [Mon Aug 1 16:37:14 2022] POST /identity/v3/endpoints => generated 346 bytes in 255 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 16:37:14.298072 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06fa6932-0343-4fad-9a7f-8edd84b38fcb None admin] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:37:14.310795 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-115df185-3b50-49dd-99be-adad101da6c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:37:14.311664 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115df185-3b50-49dd-99be-adad101da6c8 None admin] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:37:14.311940 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115df185-3b50-49dd-99be-adad101da6c8 None admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:37:14.312207 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115df185-3b50-49dd-99be-adad101da6c8 None admin] PATH_INFO: `/v3/services/network` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:37:14.312889 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-115df185-3b50-49dd-99be-adad101da6c8 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:37:14.314517 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-115df185-3b50-49dd-99be-adad101da6c8 None admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:37:14.328853 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application [None req-115df185-3b50-49dd-99be-adad101da6c8 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Aug 01 16:37:14.328853 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 16:37:14.328853 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 01 16:37:14.328853 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 01 16:37:14.328853 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 01 16:37:14.328853 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 01 16:37:14.328853 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 01 16:37:14.328853 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 01 16:37:14.328853 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 01 16:37:14.328853 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application Aug 01 16:37:14.328853 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 01 16:37:14.328853 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application Aug 01 16:37:14.328853 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 16:37:14.328853 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 16:37:14.328853 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 16:37:14.328853 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 16:37:14.328853 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 16:37:14.329934 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 16:37:14.329934 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 16:37:14.329934 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 16:37:14.329934 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 16:37:14.329934 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 16:37:14.329934 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 16:37:14.329934 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 01 16:37:14.329934 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 01 16:37:14.329934 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 01 16:37:14.329934 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 01 16:37:14.329934 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 16:37:14.329934 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 16:37:14.329934 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 16:37:14.329934 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 16:37:14.329934 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 16:37:14.329934 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 16:37:14.330922 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 16:37:14.330922 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application with Lock( Aug 01 16:37:14.330922 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 16:37:14.330922 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self._enter() Aug 01 16:37:14.330922 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 16:37:14.330922 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 16:37:14.330922 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 16:37:14.330922 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self.creator() Aug 01 16:37:14.330922 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 16:37:14.330922 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application created_value = creator( Aug 01 16:37:14.330922 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 01 16:37:14.330922 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 01 16:37:14.330922 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 01 16:37:14.330922 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application Aug 01 16:37:14.336708 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 417/820] 10.222.0.20 () {56 vars in 1111 bytes} [Mon Aug 1 16:37:14 2022] GET /identity/v3/services/network => generated 88 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 16:37:14.338379 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-115df185-3b50-49dd-99be-adad101da6c8 None admin] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:37:14.428293 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06fa6932-0343-4fad-9a7f-8edd84b38fcb None admin] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:37:14.447625 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG dbcounter [-] [87057] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87057) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:37:14.467049 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9856c8bc-ac34-4867-9b5a-1ba36ba8ff38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:37:14.467764 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9856c8bc-ac34-4867-9b5a-1ba36ba8ff38 None admin] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:37:14.468257 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9856c8bc-ac34-4867-9b5a-1ba36ba8ff38 None admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:37:14.468532 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9856c8bc-ac34-4867-9b5a-1ba36ba8ff38 None admin] PATH_INFO: `/v3/users/swift` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:37:14.480818 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9856c8bc-ac34-4867-9b5a-1ba36ba8ff38 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:37:14.482913 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9856c8bc-ac34-4867-9b5a-1ba36ba8ff38 None admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:37:14.491423 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-115df185-3b50-49dd-99be-adad101da6c8 None admin] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:37:14.498981 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application [None req-9856c8bc-ac34-4867-9b5a-1ba36ba8ff38 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 01 16:37:14.498981 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 16:37:14.498981 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 16:37:14.498981 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 16:37:14.498981 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 16:37:14.498981 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 16:37:14.498981 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 16:37:14.498981 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 16:37:14.498981 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 16:37:14.498981 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 16:37:14.498981 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 16:37:14.498981 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 16:37:14.498981 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 01 16:37:14.498981 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 01 16:37:14.498981 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 01 16:37:14.498981 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 01 16:37:14.500928 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 16:37:14.500928 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 16:37:14.500928 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 01 16:37:14.500928 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 16:37:14.500928 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 01 16:37:14.500928 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 16:37:14.500928 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 16:37:14.500928 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 16:37:14.500928 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 16:37:14.500928 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 16:37:14.500928 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 16:37:14.500928 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application with Lock( Aug 01 16:37:14.500928 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 16:37:14.500928 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self._enter() Aug 01 16:37:14.500928 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 16:37:14.500928 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 16:37:14.500928 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 16:37:14.500928 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self.creator() Aug 01 16:37:14.501867 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 16:37:14.501867 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application created_value = creator( Aug 01 16:37:14.501867 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 01 16:37:14.501867 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 01 16:37:14.501867 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 01 16:37:14.501867 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 01 16:37:14.501867 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 01 16:37:14.501867 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 01 16:37:14.501867 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 01 16:37:14.501867 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application Aug 01 16:37:14.501867 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 404/821] 10.222.0.20 () {56 vars in 1096 bytes} [Mon Aug 1 16:37:14 2022] GET /identity/v3/users/swift => generated 83 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 16:37:14.501867 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9856c8bc-ac34-4867-9b5a-1ba36ba8ff38 None admin] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:37:14.544855 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e50350-7777-4886-a167-eddad821b852 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:37:14.545514 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e50350-7777-4886-a167-eddad821b852 None admin] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:37:14.545798 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e50350-7777-4886-a167-eddad821b852 None admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:37:14.546097 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e50350-7777-4886-a167-eddad821b852 None admin] PATH_INFO: `/v3/projects` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:37:14.546862 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6e50350-7777-4886-a167-eddad821b852 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87058) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:37:14.547168 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6e50350-7777-4886-a167-eddad821b852 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:37:14.548822 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6e50350-7777-4886-a167-eddad821b852 None admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:37:14.562536 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 418/822] 10.222.0.20 () {56 vars in 1148 bytes} [Mon Aug 1 16:37:14 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:37:14.563385 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e50350-7777-4886-a167-eddad821b852 None admin] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:37:14.618463 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG dbcounter [-] [87058] Writing DB stats keystone:SELECT=100 {{(pid=87058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:37:14.645872 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9856c8bc-ac34-4867-9b5a-1ba36ba8ff38 None admin] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:37:14.679552 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e50350-7777-4886-a167-eddad821b852 None admin] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:37:14.691887 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af0ed0a2-ac7d-42d2-830d-f932e672235a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:37:14.692629 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0ed0a2-ac7d-42d2-830d-f932e672235a None admin] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:37:14.692908 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0ed0a2-ac7d-42d2-830d-f932e672235a None admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:37:14.693747 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0ed0a2-ac7d-42d2-830d-f932e672235a None admin] PATH_INFO: `/v3/services` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:37:14.694721 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af0ed0a2-ac7d-42d2-830d-f932e672235a None admin] RBAC: Adding query filter params (name=network) {{(pid=87057) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:37:14.695022 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af0ed0a2-ac7d-42d2-830d-f932e672235a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:37:14.697204 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af0ed0a2-ac7d-42d2-830d-f932e672235a None admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:37:14.703562 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 405/823] 10.222.0.20 () {56 vars in 1112 bytes} [Mon Aug 1 16:37:14 2022] GET /identity/v3/services?name=network => generated 124 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:37:14.704561 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af0ed0a2-ac7d-42d2-830d-f932e672235a None admin] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:37:14.718640 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4bb137a-ddef-40af-9bd0-1eca42fd6494 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:37:14.719479 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4bb137a-ddef-40af-9bd0-1eca42fd6494 None admin] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:37:14.719856 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4bb137a-ddef-40af-9bd0-1eca42fd6494 None admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:37:14.720235 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4bb137a-ddef-40af-9bd0-1eca42fd6494 None admin] PATH_INFO: `/v3/users` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:37:14.721176 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4bb137a-ddef-40af-9bd0-1eca42fd6494 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=87058) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:37:14.721719 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4bb137a-ddef-40af-9bd0-1eca42fd6494 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:37:14.723402 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4bb137a-ddef-40af-9bd0-1eca42fd6494 None admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:37:14.744458 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 419/824] 10.222.0.20 () {56 vars in 1135 bytes} [Mon Aug 1 16:37:14 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:37:14.745439 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4bb137a-ddef-40af-9bd0-1eca42fd6494 None admin] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:37:14.749051 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG dbcounter [-] [87057] Writing DB stats keystone:SELECT=100 {{(pid=87057) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:37:14.811155 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af0ed0a2-ac7d-42d2-830d-f932e672235a None admin] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:37:14.845877 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62b05a44-9aa8-44ca-9757-c77dcd627898 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:37:14.846588 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b05a44-9aa8-44ca-9757-c77dcd627898 None admin] REQUEST_METHOD: `PUT` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:37:14.846863 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b05a44-9aa8-44ca-9757-c77dcd627898 None admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:37:14.847133 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b05a44-9aa8-44ca-9757-c77dcd627898 None admin] PATH_INFO: `/v3/projects/7b992d6a65df4bf9b27c13192d57c5f0/users/b6a93d1c6c82489099adc8b413280fc0/roles/51bb93e277c446b49bb80a32afcd3af7` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:37:14.863468 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4bb137a-ddef-40af-9bd0-1eca42fd6494 None admin] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:37:14.863813 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62b05a44-9aa8-44ca-9757-c77dcd627898 None admin] RBAC: Authorizing `identity:create_grant(project_id=7b992d6a65df4bf9b27c13192d57c5f0, user_id=b6a93d1c6c82489099adc8b413280fc0, role_id=51bb93e277c446b49bb80a32afcd3af7)` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:37:14.865946 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62b05a44-9aa8-44ca-9757-c77dcd627898 None admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:37:14.867371 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG dbcounter [-] [87058] Writing DB stats keystone:SELECT=100 {{(pid=87058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:37:14.901694 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-373559c1-bfdb-4fc5-86a4-364a5c465289 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:37:14.902599 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373559c1-bfdb-4fc5-86a4-364a5c465289 None admin] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:37:14.903026 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373559c1-bfdb-4fc5-86a4-364a5c465289 None admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:37:14.903452 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373559c1-bfdb-4fc5-86a4-364a5c465289 None admin] PATH_INFO: `/v3/services` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:37:14.904464 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-373559c1-bfdb-4fc5-86a4-364a5c465289 None admin] RBAC: Adding query filter params (type=network) {{(pid=87058) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:37:14.905162 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-373559c1-bfdb-4fc5-86a4-364a5c465289 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:37:14.906752 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-373559c1-bfdb-4fc5-86a4-364a5c465289 None admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:37:14.917627 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 420/825] 10.222.0.20 () {56 vars in 1112 bytes} [Mon Aug 1 16:37:14 2022] GET /identity/v3/services?type=network => generated 350 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:37:14.917627 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-373559c1-bfdb-4fc5-86a4-364a5c465289 None admin] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:37:14.942680 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 406/826] 10.222.0.20 () {58 vars in 1439 bytes} [Mon Aug 1 16:37:14 2022] PUT /identity/v3/projects/7b992d6a65df4bf9b27c13192d57c5f0/users/b6a93d1c6c82489099adc8b413280fc0/roles/51bb93e277c446b49bb80a32afcd3af7 => generated 0 bytes in 239 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 16:37:14.944106 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62b05a44-9aa8-44ca-9757-c77dcd627898 None admin] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:37:15.066413 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG dbcounter [-] [87057] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87057) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:37:15.079753 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62b05a44-9aa8-44ca-9757-c77dcd627898 None admin] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:37:15.124208 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd7f4f53-65d6-4a3f-8fdf-52afdab5ca7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:37:15.125392 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7f4f53-65d6-4a3f-8fdf-52afdab5ca7d None admin] REQUEST_METHOD: `POST` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:37:15.126207 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7f4f53-65d6-4a3f-8fdf-52afdab5ca7d None admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:37:15.126671 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7f4f53-65d6-4a3f-8fdf-52afdab5ca7d None admin] PATH_INFO: `/v3/endpoints` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:37:15.127609 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd7f4f53-65d6-4a3f-8fdf-52afdab5ca7d None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:37:15.129205 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd7f4f53-65d6-4a3f-8fdf-52afdab5ca7d None admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:37:15.139815 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-373559c1-bfdb-4fc5-86a4-364a5c465289 None admin] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:37:15.182692 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4700824e-74f8-47e9-a370-5b67c011865c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:37:15.183450 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4700824e-74f8-47e9-a370-5b67c011865c None admin] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:37:15.183621 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4700824e-74f8-47e9-a370-5b67c011865c None admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:37:15.183886 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4700824e-74f8-47e9-a370-5b67c011865c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:37:15.196973 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4700824e-74f8-47e9-a370-5b67c011865c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:37:15.199685 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4700824e-74f8-47e9-a370-5b67c011865c None admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:37:15.221664 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 407/827] 10.222.0.20 () {60 vars in 1144 bytes} [Mon Aug 1 16:37:14 2022] POST /identity/v3/endpoints => generated 327 bytes in 278 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 16:37:15.222590 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application [None req-4700824e-74f8-47e9-a370-5b67c011865c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 16:37:15.222590 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 16:37:15.222590 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 16:37:15.222590 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 16:37:15.222590 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 16:37:15.222590 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 16:37:15.222590 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 16:37:15.222590 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 16:37:15.222590 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 16:37:15.222590 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 16:37:15.222590 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 16:37:15.222590 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 16:37:15.222590 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 16:37:15.222590 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 16:37:15.222590 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 16:37:15.223472 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 16:37:15.223472 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 16:37:15.223472 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 16:37:15.223472 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 16:37:15.223472 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 16:37:15.223472 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 16:37:15.223472 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 16:37:15.223472 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 16:37:15.223472 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application with Lock( Aug 01 16:37:15.223472 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 16:37:15.223472 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self._enter() Aug 01 16:37:15.223472 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 16:37:15.223472 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 16:37:15.223472 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 16:37:15.223472 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self.creator() Aug 01 16:37:15.223472 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 16:37:15.223472 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application created_value = creator( Aug 01 16:37:15.224667 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 16:37:15.224667 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 16:37:15.224667 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 16:37:15.224667 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application Aug 01 16:37:15.224667 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 421/828] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Aug 1 16:37:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 310 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 16:37:15.229561 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd7f4f53-65d6-4a3f-8fdf-52afdab5ca7d None admin] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:37:15.357882 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd7f4f53-65d6-4a3f-8fdf-52afdab5ca7d None admin] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:37:15.407449 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb852d4-1025-4317-b7ce-f8c91d4caf85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:37:15.408082 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb852d4-1025-4317-b7ce-f8c91d4caf85 None admin] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:37:15.408357 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb852d4-1025-4317-b7ce-f8c91d4caf85 None admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:37:15.408626 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb852d4-1025-4317-b7ce-f8c91d4caf85 None admin] PATH_INFO: `/v3/domains` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:37:15.409522 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cb852d4-1025-4317-b7ce-f8c91d4caf85 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87057) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:37:15.409824 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cb852d4-1025-4317-b7ce-f8c91d4caf85 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:37:15.411299 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cb852d4-1025-4317-b7ce-f8c91d4caf85 None admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:37:15.427942 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 408/829] 10.222.0.20 () {56 vars in 1109 bytes} [Mon Aug 1 16:37:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:37:15.434965 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4700824e-74f8-47e9-a370-5b67c011865c None admin] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:37:15.457792 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG dbcounter [-] [87058] Writing DB stats keystone:SELECT=100 {{(pid=87058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:37:15.548439 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4700824e-74f8-47e9-a370-5b67c011865c None admin] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:37:15.586987 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b02f3b-0883-4433-97ba-7d44987f20fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:37:15.587635 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b02f3b-0883-4433-97ba-7d44987f20fa None admin] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:37:15.587913 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b02f3b-0883-4433-97ba-7d44987f20fa None admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:37:15.588188 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b02f3b-0883-4433-97ba-7d44987f20fa None admin] PATH_INFO: `/v3/projects/service` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:37:15.596398 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7b02f3b-0883-4433-97ba-7d44987f20fa None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:37:15.597936 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7b02f3b-0883-4433-97ba-7d44987f20fa None admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:37:15.610558 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application [None req-f7b02f3b-0883-4433-97ba-7d44987f20fa None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 16:37:15.610558 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 16:37:15.610558 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 16:37:15.610558 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 16:37:15.610558 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 16:37:15.610558 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 16:37:15.610558 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 16:37:15.610558 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 16:37:15.610558 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 16:37:15.610558 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 16:37:15.610558 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 16:37:15.610558 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 16:37:15.610558 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 01 16:37:15.610558 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 01 16:37:15.610558 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 01 16:37:15.611542 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 01 16:37:15.611542 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 16:37:15.611542 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 16:37:15.611542 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 16:37:15.611542 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 16:37:15.611542 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 16:37:15.611542 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 16:37:15.611542 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 16:37:15.611542 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application with Lock( Aug 01 16:37:15.611542 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 16:37:15.611542 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self._enter() Aug 01 16:37:15.611542 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 16:37:15.611542 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 16:37:15.611542 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 16:37:15.611542 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self.creator() Aug 01 16:37:15.611542 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 16:37:15.611542 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application created_value = creator( Aug 01 16:37:15.612455 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 01 16:37:15.612455 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 01 16:37:15.612455 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 01 16:37:15.612455 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 01 16:37:15.612455 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 01 16:37:15.612455 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 01 16:37:15.612455 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 16:37:15.612455 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application Aug 01 16:37:15.612455 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 422/830] 10.222.0.20 () {56 vars in 1111 bytes} [Mon Aug 1 16:37:15 2022] GET /identity/v3/projects/service => generated 88 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 16:37:15.619835 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb852d4-1025-4317-b7ce-f8c91d4caf85 None admin] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:37:15.622797 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG dbcounter [-] [87057] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87057) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:37:15.741396 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb852d4-1025-4317-b7ce-f8c91d4caf85 None admin] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:37:15.781147 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe4317d-861c-41ba-8ade-e142e58bd1fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:37:15.781780 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe4317d-861c-41ba-8ade-e142e58bd1fd None admin] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:37:15.782083 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe4317d-861c-41ba-8ade-e142e58bd1fd None admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:37:15.782365 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe4317d-861c-41ba-8ade-e142e58bd1fd None admin] PATH_INFO: `/v3/projects` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:37:15.783132 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fe4317d-861c-41ba-8ade-e142e58bd1fd None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87057) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:37:15.783442 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fe4317d-861c-41ba-8ade-e142e58bd1fd None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:37:15.784878 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fe4317d-861c-41ba-8ade-e142e58bd1fd None admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:37:15.799031 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 409/831] 10.222.0.20 () {56 vars in 1148 bytes} [Mon Aug 1 16:37:15 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:37:15.805697 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b02f3b-0883-4433-97ba-7d44987f20fa None admin] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:37:15.908803 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG dbcounter [-] [87058] Writing DB stats keystone:SELECT=100 {{(pid=87058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:37:15.929496 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b02f3b-0883-4433-97ba-7d44987f20fa None admin] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:37:15.966206 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf722191-4bfd-40b7-8682-6f39393040a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:37:15.967193 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf722191-4bfd-40b7-8682-6f39393040a6 None admin] REQUEST_METHOD: `PUT` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:37:15.967690 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf722191-4bfd-40b7-8682-6f39393040a6 None admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:37:15.968153 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf722191-4bfd-40b7-8682-6f39393040a6 None admin] PATH_INFO: `/v3/projects/7b992d6a65df4bf9b27c13192d57c5f0/users/9ab4add1faa04faa811ef7a9341cc35c/roles/440bb943dd5b46fea038b903b6eaaf81` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:37:15.970370 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf722191-4bfd-40b7-8682-6f39393040a6 None admin] RBAC: Authorizing `identity:create_grant(project_id=7b992d6a65df4bf9b27c13192d57c5f0, user_id=9ab4add1faa04faa811ef7a9341cc35c, role_id=440bb943dd5b46fea038b903b6eaaf81)` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:37:15.972002 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf722191-4bfd-40b7-8682-6f39393040a6 None admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:37:16.044359 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 423/832] 10.222.0.20 () {58 vars in 1439 bytes} [Mon Aug 1 16:37:15 2022] PUT /identity/v3/projects/7b992d6a65df4bf9b27c13192d57c5f0/users/9ab4add1faa04faa811ef7a9341cc35c/roles/440bb943dd5b46fea038b903b6eaaf81 => generated 0 bytes in 239 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 16:37:16.196325 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15bb002-55af-40ce-9316-b3fbaf79d15c None None] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:37:16.196814 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15bb002-55af-40ce-9316-b3fbaf79d15c None None] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:37:16.196814 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15bb002-55af-40ce-9316-b3fbaf79d15c None None] PATH_INFO: `/` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:37:16.198501 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 410/833] 10.222.0.20 () {54 vars in 926 bytes} [Mon Aug 1 16:37:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 16:37:16.203702 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7e463d-2c32-44c4-b600-9aeb1b352ddb None None] REQUEST_METHOD: `POST` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:37:16.203992 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7e463d-2c32-44c4-b600-9aeb1b352ddb None None] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:37:16.204267 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7e463d-2c32-44c4-b600-9aeb1b352ddb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:37:16.225168 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.auth.core [None req-aa7e463d-2c32-44c4-b600-9aeb1b352ddb None None] MFA Rules not processed for user `1007f8e379b74e169b98fb7c3c0ba5b4`. Rule list: `[]` (Enabled: `True`). {{(pid=87058) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:37:16.226387 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.fernet_utils [None req-aa7e463d-2c32-44c4-b600-9aeb1b352ddb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:37:16.298486 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 424/834] 10.222.0.20 () {58 vars in 1023 bytes} [Mon Aug 1 16:37:16 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 01 16:37:16.322155 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b15bb002-55af-40ce-9316-b3fbaf79d15c None None] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:37:16.394986 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG dbcounter [-] [87057] Writing DB stats keystone:SELECT=100 {{(pid=87057) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:37:16.438490 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b15bb002-55af-40ce-9316-b3fbaf79d15c None None] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:37:16.470302 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0a3e37-3fde-4418-83e0-5d61da0e416e None None] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:37:16.470864 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0a3e37-3fde-4418-83e0-5d61da0e416e None None] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:37:16.471377 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0a3e37-3fde-4418-83e0-5d61da0e416e None None] PATH_INFO: `/` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:37:16.473029 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 425/835] 10.222.0.20 () {54 vars in 926 bytes} [Mon Aug 1 16:37:16 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 01 16:37:16.478702 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006921e2-c071-47b4-b401-f355265a784a None None] REQUEST_METHOD: `POST` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:37:16.479014 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006921e2-c071-47b4-b401-f355265a784a None None] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:37:16.479265 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006921e2-c071-47b4-b401-f355265a784a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:37:16.481016 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e9d15b-bf65-4e4d-b3a7-ade436940dae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:37:16.481670 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e9d15b-bf65-4e4d-b3a7-ade436940dae None admin] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:37:16.481953 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e9d15b-bf65-4e4d-b3a7-ade436940dae None admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:37:16.482262 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e9d15b-bf65-4e4d-b3a7-ade436940dae None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:37:16.482891 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17e9d15b-bf65-4e4d-b3a7-ade436940dae None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:37:16.484257 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17e9d15b-bf65-4e4d-b3a7-ade436940dae None admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:37:16.499377 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application [None req-17e9d15b-bf65-4e4d-b3a7-ade436940dae None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 01 16:37:16.499377 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 16:37:16.499377 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 01 16:37:16.499377 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 01 16:37:16.499377 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 01 16:37:16.499377 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 01 16:37:16.499377 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 01 16:37:16.499377 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 01 16:37:16.499377 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 01 16:37:16.499377 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application Aug 01 16:37:16.499377 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 01 16:37:16.499377 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application Aug 01 16:37:16.499377 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 16:37:16.499377 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 16:37:16.499377 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 16:37:16.499377 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 16:37:16.500377 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 16:37:16.500377 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 16:37:16.500377 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 16:37:16.500377 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 16:37:16.500377 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 16:37:16.500377 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 16:37:16.500377 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 16:37:16.500377 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 01 16:37:16.500377 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 01 16:37:16.500377 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 01 16:37:16.500377 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 01 16:37:16.500377 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 16:37:16.500377 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 16:37:16.500377 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 16:37:16.500377 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 16:37:16.500377 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 16:37:16.501231 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 16:37:16.501231 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 16:37:16.501231 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application with Lock( Aug 01 16:37:16.501231 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 16:37:16.501231 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self._enter() Aug 01 16:37:16.501231 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 16:37:16.501231 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 16:37:16.501231 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 16:37:16.501231 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self.creator() Aug 01 16:37:16.501231 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 16:37:16.501231 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application created_value = creator( Aug 01 16:37:16.501231 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 01 16:37:16.501231 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 01 16:37:16.501231 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 01 16:37:16.501231 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application Aug 01 16:37:16.503637 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 411/836] 10.222.0.20 () {56 vars in 1132 bytes} [Mon Aug 1 16:37:16 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 16:37:16.507364 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.auth.core [None req-006921e2-c071-47b4-b401-f355265a784a None None] MFA Rules not processed for user `1007f8e379b74e169b98fb7c3c0ba5b4`. Rule list: `[]` (Enabled: `True`). {{(pid=87058) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:37:16.508799 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.fernet_utils [None req-006921e2-c071-47b4-b401-f355265a784a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:37:16.512016 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e9d15b-bf65-4e4d-b3a7-ade436940dae None admin] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:37:16.602595 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG dbcounter [-] [87058] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:37:16.610078 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 426/837] 10.222.0.20 () {58 vars in 1023 bytes} [Mon Aug 1 16:37:16 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 01 16:37:16.627485 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e9d15b-bf65-4e4d-b3a7-ade436940dae None admin] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:37:16.631329 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-006921e2-c071-47b4-b401-f355265a784a None None] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:37:16.656485 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG dbcounter [-] [87057] Writing DB stats keystone:SELECT=100 {{(pid=87057) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:37:16.665485 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e165ad68-25b0-4669-a863-b99790df4621 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:37:16.666073 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e165ad68-25b0-4669-a863-b99790df4621 None admin] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:37:16.666362 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e165ad68-25b0-4669-a863-b99790df4621 None admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:37:16.666637 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e165ad68-25b0-4669-a863-b99790df4621 None admin] PATH_INFO: `/v3/services` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:37:16.667318 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e165ad68-25b0-4669-a863-b99790df4621 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=87057) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:37:16.667623 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e165ad68-25b0-4669-a863-b99790df4621 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:37:16.669131 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e165ad68-25b0-4669-a863-b99790df4621 None admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:37:16.674072 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 412/838] 10.222.0.20 () {56 vars in 1126 bytes} [Mon Aug 1 16:37:16 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 16:37:16.683510 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e165ad68-25b0-4669-a863-b99790df4621 None admin] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:37:16.764035 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-006921e2-c071-47b4-b401-f355265a784a None None] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:37:16.811713 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b0c2230-a1d7-45a5-bfd3-5b88710cb328 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:37:16.812693 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0c2230-a1d7-45a5-bfd3-5b88710cb328 None admin] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:37:16.812849 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0c2230-a1d7-45a5-bfd3-5b88710cb328 None admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:37:16.813154 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0c2230-a1d7-45a5-bfd3-5b88710cb328 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:37:16.813812 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b0c2230-a1d7-45a5-bfd3-5b88710cb328 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:37:16.815640 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b0c2230-a1d7-45a5-bfd3-5b88710cb328 None admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:37:16.817060 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e165ad68-25b0-4669-a863-b99790df4621 None admin] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:37:16.826881 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application [None req-6b0c2230-a1d7-45a5-bfd3-5b88710cb328 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 01 16:37:16.826881 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 16:37:16.826881 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 01 16:37:16.826881 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 01 16:37:16.826881 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 01 16:37:16.826881 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 01 16:37:16.826881 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 01 16:37:16.826881 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 01 16:37:16.826881 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 01 16:37:16.826881 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application Aug 01 16:37:16.826881 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 01 16:37:16.826881 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application Aug 01 16:37:16.826881 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 16:37:16.826881 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 16:37:16.826881 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 16:37:16.826881 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 16:37:16.826881 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 16:37:16.828100 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 16:37:16.828100 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 16:37:16.828100 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 16:37:16.828100 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 16:37:16.828100 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 16:37:16.828100 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 16:37:16.828100 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 01 16:37:16.828100 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 01 16:37:16.828100 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 01 16:37:16.828100 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 01 16:37:16.828100 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 16:37:16.828100 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 16:37:16.828100 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 16:37:16.828100 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 16:37:16.828100 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 16:37:16.828100 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 16:37:16.829133 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 16:37:16.829133 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application with Lock( Aug 01 16:37:16.829133 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 16:37:16.829133 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self._enter() Aug 01 16:37:16.829133 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 16:37:16.829133 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 16:37:16.829133 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 16:37:16.829133 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self.creator() Aug 01 16:37:16.829133 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 16:37:16.829133 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application created_value = creator( Aug 01 16:37:16.829133 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 01 16:37:16.829133 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 01 16:37:16.829133 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 01 16:37:16.829133 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application Aug 01 16:37:16.831709 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 427/839] 10.222.0.20 () {56 vars in 1114 bytes} [Mon Aug 1 16:37:16 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 01 16:37:16.837698 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b0c2230-a1d7-45a5-bfd3-5b88710cb328 None admin] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:37:16.870064 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1bfb1d5-4d42-4d0f-b657-8af6ea6aee1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:37:16.871098 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1bfb1d5-4d42-4d0f-b657-8af6ea6aee1f None admin] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:37:16.871383 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1bfb1d5-4d42-4d0f-b657-8af6ea6aee1f None admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:37:16.871660 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1bfb1d5-4d42-4d0f-b657-8af6ea6aee1f None admin] PATH_INFO: `/v3/services` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:37:16.872778 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1bfb1d5-4d42-4d0f-b657-8af6ea6aee1f None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=87057) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:37:16.873087 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1bfb1d5-4d42-4d0f-b657-8af6ea6aee1f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:37:16.874854 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1bfb1d5-4d42-4d0f-b657-8af6ea6aee1f None admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:37:16.882067 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 413/840] 10.222.0.20 () {56 vars in 1126 bytes} [Mon Aug 1 16:37:16 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:37:16.916685 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG dbcounter [-] [87058] Writing DB stats keystone:SELECT=100 {{(pid=87058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:37:16.962534 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b0c2230-a1d7-45a5-bfd3-5b88710cb328 None admin] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:37:17.007384 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45d7414a-9c72-4c63-bbdc-3ebe6bc95e0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:37:17.008456 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d7414a-9c72-4c63-bbdc-3ebe6bc95e0c None admin] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:37:17.009000 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d7414a-9c72-4c63-bbdc-3ebe6bc95e0c None admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:37:17.009481 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d7414a-9c72-4c63-bbdc-3ebe6bc95e0c None admin] PATH_INFO: `/v3/services` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:37:17.010542 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45d7414a-9c72-4c63-bbdc-3ebe6bc95e0c None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=87058) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:37:17.011056 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45d7414a-9c72-4c63-bbdc-3ebe6bc95e0c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:37:17.012747 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45d7414a-9c72-4c63-bbdc-3ebe6bc95e0c None admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:37:17.019182 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 428/841] 10.222.0.20 () {56 vars in 1114 bytes} [Mon Aug 1 16:37:16 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 16:37:17.026293 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1bfb1d5-4d42-4d0f-b657-8af6ea6aee1f None admin] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:37:17.088683 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG dbcounter [-] [87057] Writing DB stats keystone:SELECT=100 {{(pid=87057) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:37:17.137395 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1bfb1d5-4d42-4d0f-b657-8af6ea6aee1f None admin] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:37:17.174327 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b62381a-559e-4dad-a732-5a2fd53ed0ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:37:17.175324 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b62381a-559e-4dad-a732-5a2fd53ed0ad None admin] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:37:17.175866 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b62381a-559e-4dad-a732-5a2fd53ed0ad None admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:37:17.176359 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b62381a-559e-4dad-a732-5a2fd53ed0ad None admin] PATH_INFO: `/v3/services` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:37:17.177055 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b62381a-559e-4dad-a732-5a2fd53ed0ad None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=87057) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:37:17.177476 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b62381a-559e-4dad-a732-5a2fd53ed0ad None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:37:17.178945 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b62381a-559e-4dad-a732-5a2fd53ed0ad None admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:37:17.184054 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 414/842] 10.222.0.20 () {56 vars in 1114 bytes} [Mon Aug 1 16:37:17 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:37:17.392319 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032d197f-5a13-4359-8906-12555d402e8d None None] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:37:17.392514 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032d197f-5a13-4359-8906-12555d402e8d None None] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:37:17.392797 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032d197f-5a13-4359-8906-12555d402e8d None None] PATH_INFO: `/` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:37:17.394433 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 429/843] 10.222.0.20 () {54 vars in 926 bytes} [Mon Aug 1 16:37:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 16:37:17.400390 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ded9d52-cdcd-4276-8e60-6275cbed5116 None None] REQUEST_METHOD: `POST` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:37:17.401239 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ded9d52-cdcd-4276-8e60-6275cbed5116 None None] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:37:17.410018 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ded9d52-cdcd-4276-8e60-6275cbed5116 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:37:17.421298 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.auth.core [None req-0ded9d52-cdcd-4276-8e60-6275cbed5116 None None] MFA Rules not processed for user `1007f8e379b74e169b98fb7c3c0ba5b4`. Rule list: `[]` (Enabled: `True`). {{(pid=87057) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:37:17.422785 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.fernet_utils [None req-0ded9d52-cdcd-4276-8e60-6275cbed5116 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87057) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:37:17.495057 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 415/844] 10.222.0.20 () {58 vars in 1023 bytes} [Mon Aug 1 16:37:17 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 01 16:37:17.517973 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-032d197f-5a13-4359-8906-12555d402e8d None None] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:37:17.628307 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-032d197f-5a13-4359-8906-12555d402e8d None None] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:37:17.656261 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG dbcounter [-] [87058] Writing DB stats keystone:SELECT=100 {{(pid=87058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:37:17.665100 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e0e57e-7cce-4c73-bd57-02eed57bfba2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:37:17.665752 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e0e57e-7cce-4c73-bd57-02eed57bfba2 None admin] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:37:17.666054 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e0e57e-7cce-4c73-bd57-02eed57bfba2 None admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:37:17.666335 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e0e57e-7cce-4c73-bd57-02eed57bfba2 None admin] PATH_INFO: `/v3/roles/service` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:37:17.678266 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6e0e57e-7cce-4c73-bd57-02eed57bfba2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 16:37:17.683169 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application [None req-b6e0e57e-7cce-4c73-bd57-02eed57bfba2 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 01 16:37:17.683169 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 16:37:17.683169 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 16:37:17.683169 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 16:37:17.683169 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 16:37:17.683169 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 16:37:17.683169 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 16:37:17.683169 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 16:37:17.683169 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 16:37:17.683169 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 16:37:17.683169 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 16:37:17.683169 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 16:37:17.683169 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 16:37:17.683169 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 16:37:17.683169 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 16:37:17.683169 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 16:37:17.684151 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 16:37:17.684151 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 16:37:17.684151 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 16:37:17.684151 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 16:37:17.684151 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 16:37:17.684151 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 16:37:17.684151 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 16:37:17.684151 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 16:37:17.684151 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 16:37:17.684151 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 16:37:17.684151 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 16:37:17.684151 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application with Lock( Aug 01 16:37:17.684151 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 16:37:17.684151 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self._enter() Aug 01 16:37:17.684151 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 16:37:17.684151 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 16:37:17.684996 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 16:37:17.684996 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self.creator() Aug 01 16:37:17.684996 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 16:37:17.684996 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application created_value = creator( Aug 01 16:37:17.684996 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 16:37:17.684996 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 16:37:17.684996 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 16:37:17.684996 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 16:37:17.684996 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 16:37:17.684996 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 16:37:17.684996 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 01 16:37:17.684996 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application Aug 01 16:37:17.684996 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 430/845] 10.222.0.20 () {56 vars in 1102 bytes} [Mon Aug 1 16:37:17 2022] GET /identity/v3/roles/service => generated 85 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 16:37:17.690570 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ded9d52-cdcd-4276-8e60-6275cbed5116 None None] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:37:17.742914 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG dbcounter [-] [87057] Writing DB stats keystone:SELECT=100 {{(pid=87057) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:37:17.795009 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ded9d52-cdcd-4276-8e60-6275cbed5116 None None] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:37:17.830418 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ce1072a-afa2-4597-8590-7119d6e85632 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:37:17.831044 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce1072a-afa2-4597-8590-7119d6e85632 None admin] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:37:17.831325 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce1072a-afa2-4597-8590-7119d6e85632 None admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:37:17.831603 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce1072a-afa2-4597-8590-7119d6e85632 None admin] PATH_INFO: `/v3/roles` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:37:17.832332 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ce1072a-afa2-4597-8590-7119d6e85632 None admin] RBAC: Adding query filter params (name=service) {{(pid=87057) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:37:17.832636 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ce1072a-afa2-4597-8590-7119d6e85632 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:37:17.834129 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ce1072a-afa2-4597-8590-7119d6e85632 None admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:37:17.842873 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 416/846] 10.222.0.20 () {56 vars in 1103 bytes} [Mon Aug 1 16:37:17 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 16:37:17.848322 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e0e57e-7cce-4c73-bd57-02eed57bfba2 None admin] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:37:17.955820 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e0e57e-7cce-4c73-bd57-02eed57bfba2 None admin] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:37:17.991207 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a8bd5ba-404a-4a05-b0f1-84eee53c3f36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:37:17.991855 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a8bd5ba-404a-4a05-b0f1-84eee53c3f36 None admin] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:37:17.992135 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a8bd5ba-404a-4a05-b0f1-84eee53c3f36 None admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:37:17.992406 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a8bd5ba-404a-4a05-b0f1-84eee53c3f36 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:37:18.003313 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a8bd5ba-404a-4a05-b0f1-84eee53c3f36 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:37:18.004762 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a8bd5ba-404a-4a05-b0f1-84eee53c3f36 None admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:37:18.020064 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application [None req-6a8bd5ba-404a-4a05-b0f1-84eee53c3f36 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 16:37:18.020064 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 16:37:18.020064 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 16:37:18.020064 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 16:37:18.020064 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 16:37:18.020064 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 16:37:18.020064 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 16:37:18.020064 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 16:37:18.020064 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 16:37:18.020064 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 16:37:18.020064 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 16:37:18.020064 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 16:37:18.020064 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 16:37:18.020064 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 16:37:18.020064 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 16:37:18.022440 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 16:37:18.022440 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 16:37:18.022440 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 16:37:18.022440 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 16:37:18.022440 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 16:37:18.022440 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 16:37:18.022440 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 16:37:18.022440 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 16:37:18.022440 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application with Lock( Aug 01 16:37:18.022440 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 16:37:18.022440 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self._enter() Aug 01 16:37:18.022440 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 16:37:18.022440 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 16:37:18.022440 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 16:37:18.022440 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self.creator() Aug 01 16:37:18.022440 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 16:37:18.022440 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application created_value = creator( Aug 01 16:37:18.024614 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 16:37:18.024614 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 16:37:18.024614 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 16:37:18.024614 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application Aug 01 16:37:18.024614 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 431/847] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Aug 1 16:37:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 16:37:18.031410 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ce1072a-afa2-4597-8590-7119d6e85632 None admin] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:37:18.142548 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ce1072a-afa2-4597-8590-7119d6e85632 None admin] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:37:18.153639 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG dbcounter [-] [87057] Writing DB stats keystone:SELECT=100 {{(pid=87057) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:37:18.176464 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d28f457-3df1-4a8f-b3e2-2e966c4cc419 None None] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:37:18.176677 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d28f457-3df1-4a8f-b3e2-2e966c4cc419 None None] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:37:18.176945 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d28f457-3df1-4a8f-b3e2-2e966c4cc419 None None] PATH_INFO: `/` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:37:18.178078 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-573486f1-eed0-448c-9339-4d06826879f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:37:18.178691 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-573486f1-eed0-448c-9339-4d06826879f8 None admin] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:37:18.178967 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-573486f1-eed0-448c-9339-4d06826879f8 None admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:37:18.179232 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-573486f1-eed0-448c-9339-4d06826879f8 None admin] PATH_INFO: `/v3/domains` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:37:18.179945 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-573486f1-eed0-448c-9339-4d06826879f8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87057) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:37:18.180244 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-573486f1-eed0-448c-9339-4d06826879f8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:37:18.181669 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-573486f1-eed0-448c-9339-4d06826879f8 None admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:37:18.183018 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 432/848] 10.222.0.20 () {54 vars in 926 bytes} [Mon Aug 1 16:37:18 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 01 16:37:18.188255 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3ce465-d048-4bda-9ad2-a5cf163b44c4 None None] REQUEST_METHOD: `POST` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:37:18.188775 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3ce465-d048-4bda-9ad2-a5cf163b44c4 None None] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:37:18.189259 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3ce465-d048-4bda-9ad2-a5cf163b44c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:37:18.194973 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 417/849] 10.222.0.20 () {56 vars in 1109 bytes} [Mon Aug 1 16:37:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:37:18.201007 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-573486f1-eed0-448c-9339-4d06826879f8 None admin] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:37:18.210536 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.auth.core [None req-1d3ce465-d048-4bda-9ad2-a5cf163b44c4 None None] MFA Rules not processed for user `1007f8e379b74e169b98fb7c3c0ba5b4`. Rule list: `[]` (Enabled: `True`). {{(pid=87058) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:37:18.212048 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.fernet_utils [None req-1d3ce465-d048-4bda-9ad2-a5cf163b44c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:37:18.257364 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG dbcounter [-] [87058] Writing DB stats keystone:SELECT=100 {{(pid=87058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:37:18.301433 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 433/850] 10.222.0.20 () {58 vars in 1023 bytes} [Mon Aug 1 16:37:18 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 01 16:37:18.306024 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-573486f1-eed0-448c-9339-4d06826879f8 None admin] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:37:18.324426 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d3ce465-d048-4bda-9ad2-a5cf163b44c4 None None] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:37:18.350387 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dfcbaf5-aca1-43e4-8d6d-3e836e457035 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:37:18.355661 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dfcbaf5-aca1-43e4-8d6d-3e836e457035 None admin] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:37:18.355661 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dfcbaf5-aca1-43e4-8d6d-3e836e457035 None admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:37:18.355661 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dfcbaf5-aca1-43e4-8d6d-3e836e457035 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:37:18.363882 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dfcbaf5-aca1-43e4-8d6d-3e836e457035 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:37:18.366294 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dfcbaf5-aca1-43e4-8d6d-3e836e457035 None admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:37:18.382355 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application [None req-4dfcbaf5-aca1-43e4-8d6d-3e836e457035 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 01 16:37:18.382355 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 16:37:18.382355 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 16:37:18.382355 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 16:37:18.382355 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 16:37:18.382355 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 16:37:18.382355 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 16:37:18.382355 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 16:37:18.382355 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 16:37:18.382355 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 16:37:18.382355 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 16:37:18.382355 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 16:37:18.382355 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 01 16:37:18.382355 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 01 16:37:18.382355 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 01 16:37:18.382355 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 01 16:37:18.383398 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 16:37:18.383398 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 16:37:18.383398 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 01 16:37:18.383398 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 16:37:18.383398 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 01 16:37:18.383398 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 16:37:18.383398 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 16:37:18.383398 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 16:37:18.383398 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 16:37:18.383398 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 16:37:18.383398 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 16:37:18.383398 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application with Lock( Aug 01 16:37:18.383398 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 16:37:18.383398 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self._enter() Aug 01 16:37:18.383398 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 16:37:18.383398 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 16:37:18.383398 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 16:37:18.383398 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self.creator() Aug 01 16:37:18.384414 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 16:37:18.384414 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application created_value = creator( Aug 01 16:37:18.384414 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 01 16:37:18.384414 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 01 16:37:18.384414 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 01 16:37:18.384414 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 01 16:37:18.384414 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 01 16:37:18.384414 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 01 16:37:18.384414 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 01 16:37:18.384414 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application Aug 01 16:37:18.384414 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 418/851] 10.222.0.20 () {56 vars in 1117 bytes} [Mon Aug 1 16:37:18 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 01 16:37:18.390576 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dfcbaf5-aca1-43e4-8d6d-3e836e457035 None admin] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:37:18.425158 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG dbcounter [-] [87057] Writing DB stats keystone:SELECT=100 {{(pid=87057) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:37:18.451442 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d3ce465-d048-4bda-9ad2-a5cf163b44c4 None None] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:37:18.486871 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f8b4d21-a0c4-4012-be1b-2f19a9cba6da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:37:18.487524 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8b4d21-a0c4-4012-be1b-2f19a9cba6da None admin] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:37:18.487806 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8b4d21-a0c4-4012-be1b-2f19a9cba6da None admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:37:18.488080 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8b4d21-a0c4-4012-be1b-2f19a9cba6da None admin] PATH_INFO: `/v3/roles/admin` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:37:18.499606 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f8b4d21-a0c4-4012-be1b-2f19a9cba6da None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 16:37:18.504535 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application [None req-5f8b4d21-a0c4-4012-be1b-2f19a9cba6da None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 16:37:18.504535 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 16:37:18.504535 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 16:37:18.504535 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 16:37:18.504535 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 16:37:18.504535 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 16:37:18.504535 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 16:37:18.504535 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 16:37:18.504535 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 16:37:18.504535 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 16:37:18.504535 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 16:37:18.504535 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 16:37:18.504535 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 16:37:18.504535 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 16:37:18.504535 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 16:37:18.504535 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 16:37:18.506077 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 16:37:18.506077 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 16:37:18.506077 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 16:37:18.506077 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 16:37:18.506077 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 16:37:18.506077 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 16:37:18.506077 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 16:37:18.506077 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 16:37:18.506077 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 16:37:18.506077 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 16:37:18.506077 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 16:37:18.506077 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application with Lock( Aug 01 16:37:18.506077 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 16:37:18.506077 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self._enter() Aug 01 16:37:18.506077 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 16:37:18.506077 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 16:37:18.506931 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 16:37:18.506931 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self.creator() Aug 01 16:37:18.506931 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 16:37:18.506931 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application created_value = creator( Aug 01 16:37:18.506931 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 16:37:18.506931 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 16:37:18.506931 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 16:37:18.506931 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 16:37:18.506931 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 16:37:18.506931 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 16:37:18.506931 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 16:37:18.506931 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application Aug 01 16:37:18.509602 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dfcbaf5-aca1-43e4-8d6d-3e836e457035 None admin] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:37:18.510174 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 434/852] 10.222.0.20 () {56 vars in 1096 bytes} [Mon Aug 1 16:37:18 2022] GET /identity/v3/roles/admin => generated 83 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 01 16:37:18.513314 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f8b4d21-a0c4-4012-be1b-2f19a9cba6da None admin] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:37:18.545244 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG dbcounter [-] [87058] Writing DB stats keystone:SELECT=100 {{(pid=87058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:37:18.550647 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb75f439-a03f-4af7-bd73-245fb16c102a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:37:18.551590 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb75f439-a03f-4af7-bd73-245fb16c102a None admin] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:37:18.552082 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb75f439-a03f-4af7-bd73-245fb16c102a None admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:37:18.553471 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb75f439-a03f-4af7-bd73-245fb16c102a None admin] PATH_INFO: `/v3/users` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:37:18.554608 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb75f439-a03f-4af7-bd73-245fb16c102a None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=87057) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:37:18.559161 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb75f439-a03f-4af7-bd73-245fb16c102a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:37:18.559161 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb75f439-a03f-4af7-bd73-245fb16c102a None admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:37:18.581854 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 419/853] 10.222.0.20 () {56 vars in 1149 bytes} [Mon Aug 1 16:37:18 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:37:18.583455 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb75f439-a03f-4af7-bd73-245fb16c102a None admin] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:37:18.618890 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f8b4d21-a0c4-4012-be1b-2f19a9cba6da None admin] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:37:18.655805 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc439b9-1a25-4f67-885a-f355d968a6de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:37:18.656530 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc439b9-1a25-4f67-885a-f355d968a6de None admin] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:37:18.656884 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc439b9-1a25-4f67-885a-f355d968a6de None admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:37:18.657269 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc439b9-1a25-4f67-885a-f355d968a6de None admin] PATH_INFO: `/v3/roles` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:37:18.658023 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbc439b9-1a25-4f67-885a-f355d968a6de None admin] RBAC: Adding query filter params (name=admin) {{(pid=87058) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:37:18.658425 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbc439b9-1a25-4f67-885a-f355d968a6de None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:37:18.659876 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbc439b9-1a25-4f67-885a-f355d968a6de None admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:37:18.674324 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 435/854] 10.222.0.20 () {56 vars in 1099 bytes} [Mon Aug 1 16:37:18 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:37:18.675284 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc439b9-1a25-4f67-885a-f355d968a6de None admin] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:37:18.683913 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG dbcounter [-] [87057] Writing DB stats keystone:SELECT=100 {{(pid=87057) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:37:18.698536 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb75f439-a03f-4af7-bd73-245fb16c102a None admin] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:37:18.738149 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49fe7794-5bbf-4b19-bedf-a4c3884477a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:37:18.739110 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49fe7794-5bbf-4b19-bedf-a4c3884477a7 None admin] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:37:18.739525 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49fe7794-5bbf-4b19-bedf-a4c3884477a7 None admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:37:18.740467 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49fe7794-5bbf-4b19-bedf-a4c3884477a7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:37:18.773745 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG dbcounter [-] [87058] Writing DB stats keystone:SELECT=100 {{(pid=87058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:37:18.782654 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc439b9-1a25-4f67-885a-f355d968a6de None admin] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:37:18.818269 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49fe7794-5bbf-4b19-bedf-a4c3884477a7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:37:18.819219 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e19d63d-b75c-4025-a461-4990d10a9402 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:37:18.819950 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e19d63d-b75c-4025-a461-4990d10a9402 None admin] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:37:18.820303 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e19d63d-b75c-4025-a461-4990d10a9402 None admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:37:18.820540 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49fe7794-5bbf-4b19-bedf-a4c3884477a7 None admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:37:18.820704 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e19d63d-b75c-4025-a461-4990d10a9402 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:37:18.831829 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e19d63d-b75c-4025-a461-4990d10a9402 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:37:18.833384 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e19d63d-b75c-4025-a461-4990d10a9402 None admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:37:18.850041 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application [None req-2e19d63d-b75c-4025-a461-4990d10a9402 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 16:37:18.850041 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 16:37:18.850041 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 16:37:18.850041 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 16:37:18.850041 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 16:37:18.850041 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 16:37:18.850041 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 16:37:18.850041 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 16:37:18.850041 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 16:37:18.850041 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 16:37:18.850041 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 16:37:18.850041 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 16:37:18.850041 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 16:37:18.850041 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 16:37:18.850041 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 16:37:18.851604 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 16:37:18.851604 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 16:37:18.851604 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 16:37:18.851604 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 16:37:18.851604 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 16:37:18.851604 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 16:37:18.851604 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 16:37:18.851604 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 16:37:18.851604 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application with Lock( Aug 01 16:37:18.851604 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 16:37:18.851604 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self._enter() Aug 01 16:37:18.851604 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 16:37:18.851604 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 16:37:18.851604 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 16:37:18.851604 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self.creator() Aug 01 16:37:18.851604 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 16:37:18.851604 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application created_value = creator( Aug 01 16:37:18.852679 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 16:37:18.852679 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 16:37:18.852679 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 16:37:18.852679 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application Aug 01 16:37:18.857810 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 436/855] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Aug 1 16:37:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 01 16:37:18.858782 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e19d63d-b75c-4025-a461-4990d10a9402 None admin] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:37:18.922898 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG dbcounter [-] [87057] Writing DB stats keystone:SELECT=100 {{(pid=87057) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:37:18.942412 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 420/856] 10.222.0.20 () {58 vars in 1282 bytes} [Mon Aug 1 16:37:18 2022] GET /identity/v3/auth/tokens => generated 1151 bytes in 355 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 01 16:37:18.943740 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49fe7794-5bbf-4b19-bedf-a4c3884477a7 None admin] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:37:18.977774 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e19d63d-b75c-4025-a461-4990d10a9402 None admin] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:37:19.012838 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf94ad9d-c95e-4761-822a-aa2b454fd3db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:37:19.013635 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf94ad9d-c95e-4761-822a-aa2b454fd3db None admin] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:37:19.014013 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf94ad9d-c95e-4761-822a-aa2b454fd3db None admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:37:19.014411 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf94ad9d-c95e-4761-822a-aa2b454fd3db None admin] PATH_INFO: `/v3/domains` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:37:19.015208 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf94ad9d-c95e-4761-822a-aa2b454fd3db None admin] RBAC: Adding query filter params (name=Default) {{(pid=87058) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:37:19.015612 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf94ad9d-c95e-4761-822a-aa2b454fd3db None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:37:19.017335 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf94ad9d-c95e-4761-822a-aa2b454fd3db None admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:37:19.034446 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 437/857] 10.222.0.20 () {56 vars in 1109 bytes} [Mon Aug 1 16:37:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:37:19.034446 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9755058b-7425-44c1-848e-604b5ba0d8ea None None] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:37:19.034446 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9755058b-7425-44c1-848e-604b5ba0d8ea None None] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:37:19.034446 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9755058b-7425-44c1-848e-604b5ba0d8ea None None] PATH_INFO: `/` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:37:19.034446 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 438/858] 10.222.0.20 () {54 vars in 926 bytes} [Mon Aug 1 16:37:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 16:37:19.035306 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9755058b-7425-44c1-848e-604b5ba0d8ea None None] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:37:19.040578 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG dbcounter [-] [87058] Writing DB stats keystone:SELECT=100 {{(pid=87058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:37:19.058931 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49fe7794-5bbf-4b19-bedf-a4c3884477a7 None admin] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:37:19.097206 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3bca984-03fe-47d6-9482-16e4171aeaa4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:37:19.097885 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3bca984-03fe-47d6-9482-16e4171aeaa4 None admin] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:37:19.098207 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3bca984-03fe-47d6-9482-16e4171aeaa4 None admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:37:19.098484 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3bca984-03fe-47d6-9482-16e4171aeaa4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:37:19.109858 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3bca984-03fe-47d6-9482-16e4171aeaa4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:37:19.111446 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3bca984-03fe-47d6-9482-16e4171aeaa4 None admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:37:19.127935 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application [None req-d3bca984-03fe-47d6-9482-16e4171aeaa4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 16:37:19.127935 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 16:37:19.127935 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 16:37:19.127935 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 16:37:19.127935 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 16:37:19.127935 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 16:37:19.127935 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 16:37:19.127935 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 16:37:19.127935 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 16:37:19.127935 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 16:37:19.127935 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 16:37:19.127935 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 16:37:19.127935 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 16:37:19.127935 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 16:37:19.127935 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 16:37:19.128900 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 16:37:19.128900 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 16:37:19.128900 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 16:37:19.128900 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 16:37:19.128900 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 16:37:19.128900 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 16:37:19.128900 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 16:37:19.128900 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 16:37:19.128900 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application with Lock( Aug 01 16:37:19.128900 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 16:37:19.128900 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self._enter() Aug 01 16:37:19.128900 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 16:37:19.128900 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 16:37:19.128900 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 16:37:19.128900 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self.creator() Aug 01 16:37:19.128900 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 16:37:19.128900 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application created_value = creator( Aug 01 16:37:19.135261 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 16:37:19.135261 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 16:37:19.135261 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 16:37:19.135261 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application Aug 01 16:37:19.135261 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 421/859] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Aug 1 16:37:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 16:37:19.135261 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56e8292-1634-48a0-b600-83614b73f70c None None] REQUEST_METHOD: `POST` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:37:19.135261 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56e8292-1634-48a0-b600-83614b73f70c None None] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:37:19.135261 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56e8292-1634-48a0-b600-83614b73f70c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:37:19.150484 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9755058b-7425-44c1-848e-604b5ba0d8ea None None] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:37:19.151693 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.auth.core [None req-d56e8292-1634-48a0-b600-83614b73f70c None None] MFA Rules not processed for user `1007f8e379b74e169b98fb7c3c0ba5b4`. Rule list: `[]` (Enabled: `True`). {{(pid=87057) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:37:19.153125 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.fernet_utils [None req-d56e8292-1634-48a0-b600-83614b73f70c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87057) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:37:19.186875 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c473596-74e4-4aad-be23-168c8dcdb9f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:37:19.187533 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c473596-74e4-4aad-be23-168c8dcdb9f0 None admin] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:37:19.187814 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c473596-74e4-4aad-be23-168c8dcdb9f0 None admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:37:19.188087 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c473596-74e4-4aad-be23-168c8dcdb9f0 None admin] PATH_INFO: `/v3/users/swift` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:37:19.189704 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG dbcounter [-] [87057] Writing DB stats keystone:SELECT=100 {{(pid=87057) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:37:19.196445 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c473596-74e4-4aad-be23-168c8dcdb9f0 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:37:19.197892 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c473596-74e4-4aad-be23-168c8dcdb9f0 None admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:37:19.210084 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application [None req-7c473596-74e4-4aad-be23-168c8dcdb9f0 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 01 16:37:19.210084 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 16:37:19.210084 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 16:37:19.210084 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 16:37:19.210084 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 16:37:19.210084 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 16:37:19.210084 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 16:37:19.210084 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 16:37:19.210084 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 16:37:19.210084 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 16:37:19.210084 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 16:37:19.210084 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 16:37:19.210084 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 01 16:37:19.210084 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 01 16:37:19.210084 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 01 16:37:19.210084 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 01 16:37:19.216944 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 16:37:19.216944 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 16:37:19.216944 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 01 16:37:19.216944 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 16:37:19.216944 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 01 16:37:19.216944 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 16:37:19.216944 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 16:37:19.216944 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 16:37:19.216944 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 16:37:19.216944 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 16:37:19.216944 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 16:37:19.216944 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application with Lock( Aug 01 16:37:19.216944 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 16:37:19.216944 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self._enter() Aug 01 16:37:19.216944 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 16:37:19.216944 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 16:37:19.216944 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 16:37:19.216944 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self.creator() Aug 01 16:37:19.218151 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 16:37:19.218151 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application created_value = creator( Aug 01 16:37:19.218151 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 01 16:37:19.218151 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 01 16:37:19.218151 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 01 16:37:19.218151 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 01 16:37:19.218151 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 01 16:37:19.218151 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 01 16:37:19.218151 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 01 16:37:19.218151 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application Aug 01 16:37:19.218151 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 439/860] 10.222.0.20 () {56 vars in 1096 bytes} [Mon Aug 1 16:37:19 2022] GET /identity/v3/users/swift => generated 83 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 16:37:19.218151 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c473596-74e4-4aad-be23-168c8dcdb9f0 None admin] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:37:19.231553 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 422/861] 10.222.0.20 () {58 vars in 1023 bytes} [Mon Aug 1 16:37:19 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 01 16:37:19.233173 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d56e8292-1634-48a0-b600-83614b73f70c None None] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:37:19.288177 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG dbcounter [-] [87058] Writing DB stats keystone:SELECT=100 {{(pid=87058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:37:19.325321 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c473596-74e4-4aad-be23-168c8dcdb9f0 None admin] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:37:19.344129 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d56e8292-1634-48a0-b600-83614b73f70c None None] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:37:19.359334 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ad77f7-fbe0-492d-8a4d-910222e8cfc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:37:19.359954 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ad77f7-fbe0-492d-8a4d-910222e8cfc9 None admin] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:37:19.360241 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ad77f7-fbe0-492d-8a4d-910222e8cfc9 None admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:37:19.360512 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ad77f7-fbe0-492d-8a4d-910222e8cfc9 None admin] PATH_INFO: `/v3/domains` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:37:19.361206 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55ad77f7-fbe0-492d-8a4d-910222e8cfc9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87058) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:37:19.361513 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55ad77f7-fbe0-492d-8a4d-910222e8cfc9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:37:19.362923 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55ad77f7-fbe0-492d-8a4d-910222e8cfc9 None admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:37:19.375714 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 440/862] 10.222.0.20 () {56 vars in 1109 bytes} [Mon Aug 1 16:37:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:37:19.376494 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ad77f7-fbe0-492d-8a4d-910222e8cfc9 None admin] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:37:19.379606 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e01f762-4eea-4076-85ab-5de3cfcda5d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:37:19.380316 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e01f762-4eea-4076-85ab-5de3cfcda5d3 None admin] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:37:19.380655 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e01f762-4eea-4076-85ab-5de3cfcda5d3 None admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:37:19.380933 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e01f762-4eea-4076-85ab-5de3cfcda5d3 None admin] PATH_INFO: `/v3/users` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:37:19.381727 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e01f762-4eea-4076-85ab-5de3cfcda5d3 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=87057) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:37:19.382057 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e01f762-4eea-4076-85ab-5de3cfcda5d3 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:37:19.383487 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e01f762-4eea-4076-85ab-5de3cfcda5d3 None admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:37:19.401326 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 423/863] 10.222.0.20 () {56 vars in 1135 bytes} [Mon Aug 1 16:37:19 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:37:19.402591 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ce4b9d-3dac-4e2f-bc46-2f98896c0b75 None None] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:37:19.402876 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ce4b9d-3dac-4e2f-bc46-2f98896c0b75 None None] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:37:19.403152 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ce4b9d-3dac-4e2f-bc46-2f98896c0b75 None None] PATH_INFO: `/` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:37:19.404367 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 424/864] 10.222.0.20 () {54 vars in 926 bytes} [Mon Aug 1 16:37:19 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 16:37:19.405065 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92ce4b9d-3dac-4e2f-bc46-2f98896c0b75 None None] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:37:19.433913 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG dbcounter [-] [87057] Writing DB stats keystone:SELECT=100 {{(pid=87057) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:37:19.485192 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ad77f7-fbe0-492d-8a4d-910222e8cfc9 None admin] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:37:19.505685 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92ce4b9d-3dac-4e2f-bc46-2f98896c0b75 None None] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:37:19.508912 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG dbcounter [-] [87058] Writing DB stats keystone:SELECT=100 {{(pid=87058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:37:19.519450 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85818e83-0867-41c8-8e4a-4af45419e34f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:37:19.520467 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85818e83-0867-41c8-8e4a-4af45419e34f None admin] REQUEST_METHOD: `POST` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:37:19.520467 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85818e83-0867-41c8-8e4a-4af45419e34f None admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:37:19.520660 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85818e83-0867-41c8-8e4a-4af45419e34f None admin] PATH_INFO: `/v3/services` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:37:19.521787 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85818e83-0867-41c8-8e4a-4af45419e34f None admin] RBAC: Authorizing `identity:create_service()` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:37:19.523200 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85818e83-0867-41c8-8e4a-4af45419e34f None admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:37:19.543352 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2589ade8-23ab-47d7-afc1-dd4d61734743 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:37:19.543352 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2589ade8-23ab-47d7-afc1-dd4d61734743 None admin] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:37:19.543352 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2589ade8-23ab-47d7-afc1-dd4d61734743 None admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:37:19.543352 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2589ade8-23ab-47d7-afc1-dd4d61734743 None admin] PATH_INFO: `/v3/projects/service` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:37:19.548672 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2589ade8-23ab-47d7-afc1-dd4d61734743 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:37:19.550247 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2589ade8-23ab-47d7-afc1-dd4d61734743 None admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:37:19.561273 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application [None req-2589ade8-23ab-47d7-afc1-dd4d61734743 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 16:37:19.561273 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 16:37:19.561273 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 16:37:19.561273 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 16:37:19.561273 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 16:37:19.561273 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 16:37:19.561273 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 16:37:19.561273 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 16:37:19.561273 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 16:37:19.561273 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 16:37:19.561273 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 16:37:19.561273 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 16:37:19.561273 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 01 16:37:19.561273 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 01 16:37:19.561273 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 01 16:37:19.562224 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 01 16:37:19.562224 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 16:37:19.562224 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 16:37:19.562224 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 16:37:19.562224 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 16:37:19.562224 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 16:37:19.562224 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 16:37:19.562224 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 16:37:19.562224 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application with Lock( Aug 01 16:37:19.562224 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 16:37:19.562224 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self._enter() Aug 01 16:37:19.562224 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 16:37:19.562224 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 16:37:19.562224 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 16:37:19.562224 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self.creator() Aug 01 16:37:19.562224 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 16:37:19.562224 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application created_value = creator( Aug 01 16:37:19.563157 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 01 16:37:19.563157 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 01 16:37:19.563157 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 01 16:37:19.563157 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 01 16:37:19.563157 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 01 16:37:19.563157 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 01 16:37:19.563157 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 16:37:19.563157 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application Aug 01 16:37:19.563157 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 425/865] 10.222.0.20 () {56 vars in 1111 bytes} [Mon Aug 1 16:37:19 2022] GET /identity/v3/projects/service => generated 88 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 16:37:19.563642 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2589ade8-23ab-47d7-afc1-dd4d61734743 None admin] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:37:19.602080 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 441/866] 10.222.0.20 () {60 vars in 1141 bytes} [Mon Aug 1 16:37:19 2022] POST /identity/v3/services => generated 269 bytes in 221 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 16:37:19.602080 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8eaffe-f71d-40f8-8da4-17bba8b4e028 None None] REQUEST_METHOD: `POST` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:37:19.602080 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8eaffe-f71d-40f8-8da4-17bba8b4e028 None None] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:37:19.602080 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8eaffe-f71d-40f8-8da4-17bba8b4e028 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:37:19.620045 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.auth.core [None req-9f8eaffe-f71d-40f8-8da4-17bba8b4e028 None None] MFA Rules not processed for user `1007f8e379b74e169b98fb7c3c0ba5b4`. Rule list: `[]` (Enabled: `True`). {{(pid=87058) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:37:19.621313 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.fernet_utils [None req-9f8eaffe-f71d-40f8-8da4-17bba8b4e028 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:37:19.659702 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG dbcounter [-] [87057] Writing DB stats keystone:SELECT=100 {{(pid=87057) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:37:19.674360 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2589ade8-23ab-47d7-afc1-dd4d61734743 None admin] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:37:19.707336 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 442/867] 10.222.0.20 () {58 vars in 1023 bytes} [Mon Aug 1 16:37:19 2022] POST /identity/v3/auth/tokens => generated 1261 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 01 16:37:19.713052 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54978874-829a-4300-8509-9ee999f97ab9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:37:19.713687 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54978874-829a-4300-8509-9ee999f97ab9 None admin] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:37:19.713963 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54978874-829a-4300-8509-9ee999f97ab9 None admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:37:19.714262 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54978874-829a-4300-8509-9ee999f97ab9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:37:19.718246 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f8eaffe-f71d-40f8-8da4-17bba8b4e028 None None] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:37:19.793993 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54978874-829a-4300-8509-9ee999f97ab9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:37:19.795645 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54978874-829a-4300-8509-9ee999f97ab9 None admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:37:19.842278 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f8eaffe-f71d-40f8-8da4-17bba8b4e028 None None] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:37:19.857779 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG dbcounter [-] [87058] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:37:19.882432 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6635cf19-e3fe-44be-ac74-29a2193ddccd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:37:19.883052 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6635cf19-e3fe-44be-ac74-29a2193ddccd None admin] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:37:19.883331 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6635cf19-e3fe-44be-ac74-29a2193ddccd None admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:37:19.883597 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6635cf19-e3fe-44be-ac74-29a2193ddccd None admin] PATH_INFO: `/v3/projects` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:37:19.884348 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6635cf19-e3fe-44be-ac74-29a2193ddccd None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87058) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:37:19.884646 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6635cf19-e3fe-44be-ac74-29a2193ddccd None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:37:19.886119 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6635cf19-e3fe-44be-ac74-29a2193ddccd None admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:37:19.891120 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG dbcounter [-] [87057] Writing DB stats keystone:SELECT=100 {{(pid=87057) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:37:19.904149 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 443/868] 10.222.0.20 () {56 vars in 1148 bytes} [Mon Aug 1 16:37:19 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:37:19.904149 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6635cf19-e3fe-44be-ac74-29a2193ddccd None admin] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:37:19.918061 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 426/869] 10.222.0.20 () {58 vars in 1282 bytes} [Mon Aug 1 16:37:19 2022] GET /identity/v3/auth/tokens => generated 1261 bytes in 355 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 01 16:37:19.919261 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54978874-829a-4300-8509-9ee999f97ab9 None admin] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:37:20.021187 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6635cf19-e3fe-44be-ac74-29a2193ddccd None admin] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:37:20.033059 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54978874-829a-4300-8509-9ee999f97ab9 None admin] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:37:20.057139 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b75657-388d-4742-9775-d00381011264 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:37:20.058160 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b75657-388d-4742-9775-d00381011264 None admin] REQUEST_METHOD: `POST` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:37:20.058450 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b75657-388d-4742-9775-d00381011264 None admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:37:20.058731 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b75657-388d-4742-9775-d00381011264 None admin] PATH_INFO: `/v3/services` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:37:20.059432 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94b75657-388d-4742-9775-d00381011264 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:37:20.060798 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94b75657-388d-4742-9775-d00381011264 None admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:37:20.073148 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b4755c6-2263-4565-9adf-26280a03b21b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:37:20.073767 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4755c6-2263-4565-9adf-26280a03b21b None admin] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:37:20.074066 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4755c6-2263-4565-9adf-26280a03b21b None admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:37:20.074343 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4755c6-2263-4565-9adf-26280a03b21b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:37:20.075129 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b4755c6-2263-4565-9adf-26280a03b21b None admin] RBAC: Adding query filter params (role.id=51bb93e277c446b49bb80a32afcd3af7, scope.project.id=7b992d6a65df4bf9b27c13192d57c5f0, user.id=b6a93d1c6c82489099adc8b413280fc0) {{(pid=87057) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:37:20.075438 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b4755c6-2263-4565-9adf-26280a03b21b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:37:20.076855 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b4755c6-2263-4565-9adf-26280a03b21b None admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:37:20.082649 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 427/870] 10.222.0.20 () {56 vars in 1374 bytes} [Mon Aug 1 16:37:19 2022] GET /identity/v3/role_assignments?user.id=b6a93d1c6c82489099adc8b413280fc0&scope.project.id=7b992d6a65df4bf9b27c13192d57c5f0&role.id=51bb93e277c446b49bb80a32afcd3af7 => generated 608 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:37:20.083471 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b4755c6-2263-4565-9adf-26280a03b21b None admin] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:37:20.144006 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG dbcounter [-] [87057] Writing DB stats keystone:SELECT=100 {{(pid=87057) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:37:20.176031 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 444/871] 10.222.0.20 () {60 vars in 1141 bytes} [Mon Aug 1 16:37:19 2022] POST /identity/v3/services => generated 251 bytes in 276 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 16:37:20.205069 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b4755c6-2263-4565-9adf-26280a03b21b None admin] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:37:20.260050 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d515bae-e435-416f-8578-e5cb665812b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:37:20.261101 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d515bae-e435-416f-8578-e5cb665812b8 None admin] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:37:20.261595 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d515bae-e435-416f-8578-e5cb665812b8 None admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:37:20.262108 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d515bae-e435-416f-8578-e5cb665812b8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:37:20.275958 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d515bae-e435-416f-8578-e5cb665812b8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:37:20.277781 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d515bae-e435-416f-8578-e5cb665812b8 None admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:37:20.294313 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application [None req-6d515bae-e435-416f-8578-e5cb665812b8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 16:37:20.294313 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 16:37:20.294313 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 16:37:20.294313 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 16:37:20.294313 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 16:37:20.294313 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 16:37:20.294313 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 16:37:20.294313 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 16:37:20.294313 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 16:37:20.294313 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 16:37:20.294313 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 16:37:20.294313 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 16:37:20.294313 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 16:37:20.294313 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 16:37:20.294313 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 16:37:20.295435 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 16:37:20.295435 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 16:37:20.295435 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 16:37:20.295435 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 16:37:20.295435 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 16:37:20.295435 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 16:37:20.295435 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 16:37:20.295435 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 16:37:20.295435 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application with Lock( Aug 01 16:37:20.295435 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 16:37:20.295435 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self._enter() Aug 01 16:37:20.295435 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 16:37:20.295435 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 16:37:20.295435 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 16:37:20.295435 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self.creator() Aug 01 16:37:20.295435 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 16:37:20.295435 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application created_value = creator( Aug 01 16:37:20.296558 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 16:37:20.296558 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 16:37:20.296558 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 16:37:20.296558 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application Aug 01 16:37:20.298232 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 428/872] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Aug 1 16:37:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 16:37:20.304549 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b75657-388d-4742-9775-d00381011264 None admin] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:37:20.340667 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG dbcounter [-] [87058] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:37:20.417099 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b75657-388d-4742-9775-d00381011264 None admin] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:37:20.453449 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5036b152-bf32-4533-bdb8-dbb092bbf8ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:37:20.454096 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5036b152-bf32-4533-bdb8-dbb092bbf8ac None admin] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:37:20.454379 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5036b152-bf32-4533-bdb8-dbb092bbf8ac None admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:37:20.454653 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5036b152-bf32-4533-bdb8-dbb092bbf8ac None admin] PATH_INFO: `/v3/domains` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:37:20.455348 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5036b152-bf32-4533-bdb8-dbb092bbf8ac None admin] RBAC: Adding query filter params (name=Default) {{(pid=87058) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:37:20.455649 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5036b152-bf32-4533-bdb8-dbb092bbf8ac None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:37:20.457076 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5036b152-bf32-4533-bdb8-dbb092bbf8ac None admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:37:20.469127 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 445/873] 10.222.0.20 () {56 vars in 1109 bytes} [Mon Aug 1 16:37:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 16:37:20.474759 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d515bae-e435-416f-8578-e5cb665812b8 None admin] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:37:20.579299 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG dbcounter [-] [87057] Writing DB stats keystone:SELECT=100 {{(pid=87057) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:37:20.582982 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d515bae-e435-416f-8578-e5cb665812b8 None admin] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:37:20.619865 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98fac3b8-0bd1-4901-ab06-3bd4eb7eea5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:37:20.620428 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98fac3b8-0bd1-4901-ab06-3bd4eb7eea5f None admin] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:37:20.620710 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98fac3b8-0bd1-4901-ab06-3bd4eb7eea5f None admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:37:20.620987 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98fac3b8-0bd1-4901-ab06-3bd4eb7eea5f None admin] PATH_INFO: `/v3/projects/service` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:37:20.627984 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98fac3b8-0bd1-4901-ab06-3bd4eb7eea5f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:37:20.629439 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98fac3b8-0bd1-4901-ab06-3bd4eb7eea5f None admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:37:20.641827 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application [None req-98fac3b8-0bd1-4901-ab06-3bd4eb7eea5f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 16:37:20.641827 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 16:37:20.641827 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 16:37:20.641827 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 16:37:20.641827 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 16:37:20.641827 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 16:37:20.641827 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 16:37:20.641827 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 16:37:20.641827 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 16:37:20.641827 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 16:37:20.641827 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 16:37:20.641827 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 16:37:20.641827 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 01 16:37:20.641827 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 01 16:37:20.641827 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 01 16:37:20.642774 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 01 16:37:20.642774 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 16:37:20.642774 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 16:37:20.642774 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 16:37:20.642774 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 16:37:20.642774 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 16:37:20.642774 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 16:37:20.642774 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 16:37:20.642774 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application with Lock( Aug 01 16:37:20.642774 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 16:37:20.642774 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self._enter() Aug 01 16:37:20.642774 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 16:37:20.642774 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 16:37:20.642774 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 16:37:20.642774 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self.creator() Aug 01 16:37:20.642774 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 16:37:20.642774 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application created_value = creator( Aug 01 16:37:20.643826 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 01 16:37:20.643826 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 01 16:37:20.643826 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 01 16:37:20.643826 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 01 16:37:20.643826 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 01 16:37:20.643826 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 01 16:37:20.643826 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 16:37:20.643826 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application Aug 01 16:37:20.643826 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 429/874] 10.222.0.20 () {56 vars in 1111 bytes} [Mon Aug 1 16:37:20 2022] GET /identity/v3/projects/service => generated 88 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 16:37:20.648794 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5036b152-bf32-4533-bdb8-dbb092bbf8ac None admin] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:37:20.752263 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG dbcounter [-] [87058] Writing DB stats keystone:SELECT=100 {{(pid=87058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:37:20.762872 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5036b152-bf32-4533-bdb8-dbb092bbf8ac None admin] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:37:20.804248 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2857f36-e295-45be-83fa-53b6de8011e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:37:20.804874 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2857f36-e295-45be-83fa-53b6de8011e7 None admin] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:37:20.805154 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2857f36-e295-45be-83fa-53b6de8011e7 None admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:37:20.805425 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2857f36-e295-45be-83fa-53b6de8011e7 None admin] PATH_INFO: `/v3/projects` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:37:20.806205 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2857f36-e295-45be-83fa-53b6de8011e7 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87058) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:37:20.806511 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2857f36-e295-45be-83fa-53b6de8011e7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:37:20.807937 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2857f36-e295-45be-83fa-53b6de8011e7 None admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:37:20.820403 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 446/875] 10.222.0.20 () {56 vars in 1148 bytes} [Mon Aug 1 16:37:20 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:37:20.826393 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98fac3b8-0bd1-4901-ab06-3bd4eb7eea5f None admin] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:37:20.948159 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98fac3b8-0bd1-4901-ab06-3bd4eb7eea5f None admin] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:37:20.985629 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b30b6176-fe18-46af-be8c-456badced225 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:37:20.986328 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30b6176-fe18-46af-be8c-456badced225 None admin] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:37:20.986611 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30b6176-fe18-46af-be8c-456badced225 None admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:37:20.986936 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30b6176-fe18-46af-be8c-456badced225 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:37:20.987747 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b30b6176-fe18-46af-be8c-456badced225 None admin] RBAC: Adding query filter params (role.id=440bb943dd5b46fea038b903b6eaaf81, scope.project.id=7b992d6a65df4bf9b27c13192d57c5f0, user.id=9ab4add1faa04faa811ef7a9341cc35c) {{(pid=87057) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:37:20.988058 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b30b6176-fe18-46af-be8c-456badced225 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:37:20.989499 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b30b6176-fe18-46af-be8c-456badced225 None admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:37:20.996964 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 430/876] 10.222.0.20 () {56 vars in 1374 bytes} [Mon Aug 1 16:37:20 2022] GET /identity/v3/role_assignments?user.id=9ab4add1faa04faa811ef7a9341cc35c&scope.project.id=7b992d6a65df4bf9b27c13192d57c5f0&role.id=440bb943dd5b46fea038b903b6eaaf81 => generated 608 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:37:21.644700 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e153acb7-8b56-4668-9b5d-2f6f89042d2c None None] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:37:21.645224 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e153acb7-8b56-4668-9b5d-2f6f89042d2c None None] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:37:21.645224 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e153acb7-8b56-4668-9b5d-2f6f89042d2c None None] PATH_INFO: `/` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:37:21.647019 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 447/877] 10.222.0.20 () {54 vars in 926 bytes} [Mon Aug 1 16:37:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 16:37:21.652233 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ba6938-215b-406f-9841-e14dc4b8af60 None None] REQUEST_METHOD: `POST` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:37:21.652640 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ba6938-215b-406f-9841-e14dc4b8af60 None None] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:37:21.652806 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ba6938-215b-406f-9841-e14dc4b8af60 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:37:21.671796 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.auth.core [None req-90ba6938-215b-406f-9841-e14dc4b8af60 None None] MFA Rules not processed for user `1007f8e379b74e169b98fb7c3c0ba5b4`. Rule list: `[]` (Enabled: `True`). {{(pid=87057) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:37:21.672989 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.fernet_utils [None req-90ba6938-215b-406f-9841-e14dc4b8af60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87057) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:37:21.691675 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG dbcounter [-] [87057] Writing DB stats keystone:SELECT=100 {{(pid=87057) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:37:21.743512 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 431/878] 10.222.0.20 () {58 vars in 1023 bytes} [Mon Aug 1 16:37:21 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 01 16:37:21.765837 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e153acb7-8b56-4668-9b5d-2f6f89042d2c None None] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:37:21.869753 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e153acb7-8b56-4668-9b5d-2f6f89042d2c None None] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:37:21.910898 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6a7b60b-8547-4320-81a4-4560668145a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:37:21.912576 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a7b60b-8547-4320-81a4-4560668145a3 None admin] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:37:21.913061 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a7b60b-8547-4320-81a4-4560668145a3 None admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:37:21.913536 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a7b60b-8547-4320-81a4-4560668145a3 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:37:21.914413 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6a7b60b-8547-4320-81a4-4560668145a3 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:37:21.915970 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6a7b60b-8547-4320-81a4-4560668145a3 None admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:37:21.925501 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application [None req-e6a7b60b-8547-4320-81a4-4560668145a3 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 01 16:37:21.925501 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 16:37:21.925501 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 01 16:37:21.925501 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 01 16:37:21.925501 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 01 16:37:21.925501 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 01 16:37:21.925501 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 01 16:37:21.925501 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 01 16:37:21.925501 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 01 16:37:21.925501 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application Aug 01 16:37:21.925501 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 01 16:37:21.925501 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application Aug 01 16:37:21.925501 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 16:37:21.925501 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 16:37:21.925501 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 16:37:21.925501 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 16:37:21.926668 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 16:37:21.926668 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 16:37:21.926668 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 16:37:21.926668 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 16:37:21.926668 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 16:37:21.926668 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 16:37:21.926668 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 16:37:21.926668 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 01 16:37:21.926668 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 01 16:37:21.926668 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 01 16:37:21.926668 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 01 16:37:21.926668 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 16:37:21.926668 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 16:37:21.926668 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 16:37:21.926668 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 16:37:21.926668 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 16:37:21.928112 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 16:37:21.928112 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 16:37:21.928112 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application with Lock( Aug 01 16:37:21.928112 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 16:37:21.928112 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self._enter() Aug 01 16:37:21.928112 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 16:37:21.928112 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 16:37:21.928112 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 16:37:21.928112 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self.creator() Aug 01 16:37:21.928112 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 16:37:21.928112 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application created_value = creator( Aug 01 16:37:21.928112 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 01 16:37:21.928112 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 01 16:37:21.928112 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 01 16:37:21.928112 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application Aug 01 16:37:21.930965 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 448/879] 10.222.0.20 () {56 vars in 1132 bytes} [Mon Aug 1 16:37:21 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 16:37:21.938338 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ba6938-215b-406f-9841-e14dc4b8af60 None None] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:37:22.039189 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ba6938-215b-406f-9841-e14dc4b8af60 None None] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:37:22.078570 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af59b3d5-5eae-4468-bbff-fb823d4c7e19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:37:22.079201 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af59b3d5-5eae-4468-bbff-fb823d4c7e19 None admin] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:37:22.079484 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af59b3d5-5eae-4468-bbff-fb823d4c7e19 None admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:37:22.079757 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af59b3d5-5eae-4468-bbff-fb823d4c7e19 None admin] PATH_INFO: `/v3/services` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:37:22.080442 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af59b3d5-5eae-4468-bbff-fb823d4c7e19 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=87057) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:37:22.080746 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af59b3d5-5eae-4468-bbff-fb823d4c7e19 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:37:22.082141 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af59b3d5-5eae-4468-bbff-fb823d4c7e19 None admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:37:22.087371 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 432/880] 10.222.0.20 () {56 vars in 1126 bytes} [Mon Aug 1 16:37:21 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:37:22.092648 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6a7b60b-8547-4320-81a4-4560668145a3 None admin] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:37:22.110346 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG dbcounter [-] [87058] Writing DB stats keystone:SELECT=100 {{(pid=87058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:37:22.199864 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6a7b60b-8547-4320-81a4-4560668145a3 None admin] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:37:22.239384 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4dc5c39-84f5-40da-84e5-fc314d1d083a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:37:22.240574 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4dc5c39-84f5-40da-84e5-fc314d1d083a None admin] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:37:22.240856 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4dc5c39-84f5-40da-84e5-fc314d1d083a None admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:37:22.241128 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4dc5c39-84f5-40da-84e5-fc314d1d083a None admin] PATH_INFO: `/v3/services` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:37:22.241813 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4dc5c39-84f5-40da-84e5-fc314d1d083a None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=87058) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:37:22.242144 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4dc5c39-84f5-40da-84e5-fc314d1d083a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:37:22.243517 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4dc5c39-84f5-40da-84e5-fc314d1d083a None admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:37:22.248809 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 449/881] 10.222.0.20 () {56 vars in 1126 bytes} [Mon Aug 1 16:37:22 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:37:22.254492 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af59b3d5-5eae-4468-bbff-fb823d4c7e19 None admin] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:37:22.282012 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG dbcounter [-] [87057] Writing DB stats keystone:SELECT=100 {{(pid=87057) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:37:22.355670 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5368e4d-5e7e-4222-b5a2-9c0ad0f98aa2 None None] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:37:22.356127 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5368e4d-5e7e-4222-b5a2-9c0ad0f98aa2 None None] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:37:22.356503 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5368e4d-5e7e-4222-b5a2-9c0ad0f98aa2 None None] PATH_INFO: `/` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:37:22.358253 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 450/882] 10.222.0.20 () {54 vars in 926 bytes} [Mon Aug 1 16:37:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 01 16:37:22.363008 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3e1c3b-bbeb-46ce-a1d5-a49e99aa9411 None None] REQUEST_METHOD: `POST` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:37:22.363347 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3e1c3b-bbeb-46ce-a1d5-a49e99aa9411 None None] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:37:22.363683 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3e1c3b-bbeb-46ce-a1d5-a49e99aa9411 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:37:22.371588 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af59b3d5-5eae-4468-bbff-fb823d4c7e19 None admin] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:37:22.382950 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.auth.core [None req-4a3e1c3b-bbeb-46ce-a1d5-a49e99aa9411 None None] MFA Rules not processed for user `1007f8e379b74e169b98fb7c3c0ba5b4`. Rule list: `[]` (Enabled: `True`). {{(pid=87058) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:37:22.384399 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.fernet_utils [None req-4a3e1c3b-bbeb-46ce-a1d5-a49e99aa9411 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:37:22.409842 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd03cdb-a0e3-4258-9b7c-54b150b7270c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:37:22.410567 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd03cdb-a0e3-4258-9b7c-54b150b7270c None admin] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:37:22.410850 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd03cdb-a0e3-4258-9b7c-54b150b7270c None admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:37:22.411122 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd03cdb-a0e3-4258-9b7c-54b150b7270c None admin] PATH_INFO: `/v3/endpoints` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:37:22.412032 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bd03cdb-a0e3-4258-9b7c-54b150b7270c None admin] RBAC: Adding query filter params (interface=public, service_id=38ed9a1b14c9490b9ba289f64b642727, region_id=RegionOne) {{(pid=87057) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:37:22.412262 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bd03cdb-a0e3-4258-9b7c-54b150b7270c None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:37:22.413631 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bd03cdb-a0e3-4258-9b7c-54b150b7270c None admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:37:22.422480 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 433/883] 10.222.0.20 () {56 vars in 1251 bytes} [Mon Aug 1 16:37:22 2022] GET /identity/v3/endpoints?service_id=38ed9a1b14c9490b9ba289f64b642727&interface=public®ion_id=RegionOne => generated 194 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:37:22.422480 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7023f8-5833-4dd9-a04e-521f252fb960 None None] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:37:22.422480 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7023f8-5833-4dd9-a04e-521f252fb960 None None] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:37:22.422480 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7023f8-5833-4dd9-a04e-521f252fb960 None None] PATH_INFO: `/` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:37:22.422480 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 434/884] 10.222.0.20 () {54 vars in 926 bytes} [Mon Aug 1 16:37:22 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 16:37:22.426921 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed7023f8-5833-4dd9-a04e-521f252fb960 None None] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:37:22.466156 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 451/885] 10.222.0.20 () {58 vars in 1023 bytes} [Mon Aug 1 16:37:22 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 01 16:37:22.466156 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9239ef-79a3-4e01-82cf-96b905bdefb0 None None] REQUEST_METHOD: `POST` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:37:22.466156 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9239ef-79a3-4e01-82cf-96b905bdefb0 None None] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:37:22.466156 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9239ef-79a3-4e01-82cf-96b905bdefb0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:37:22.490446 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.auth.core [None req-df9239ef-79a3-4e01-82cf-96b905bdefb0 None None] MFA Rules not processed for user `1007f8e379b74e169b98fb7c3c0ba5b4`. Rule list: `[]` (Enabled: `True`). {{(pid=87058) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:37:22.491634 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.fernet_utils [None req-df9239ef-79a3-4e01-82cf-96b905bdefb0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:37:22.493705 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG dbcounter [-] [87058] Writing DB stats keystone:SELECT=100 {{(pid=87058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:37:22.526004 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG dbcounter [-] [87057] Writing DB stats keystone:SELECT=100 {{(pid=87057) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:37:22.545944 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed7023f8-5833-4dd9-a04e-521f252fb960 None None] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:37:22.564541 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 452/886] 10.222.0.20 () {58 vars in 1023 bytes} [Mon Aug 1 16:37:22 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 01 16:37:22.565416 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df9239ef-79a3-4e01-82cf-96b905bdefb0 None None] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:37:22.594947 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e141ddb-a0a6-4192-b00b-df7901f68d0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:37:22.595579 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e141ddb-a0a6-4192-b00b-df7901f68d0b None admin] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:37:22.595860 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e141ddb-a0a6-4192-b00b-df7901f68d0b None admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:37:22.596132 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e141ddb-a0a6-4192-b00b-df7901f68d0b None admin] PATH_INFO: `/v3/services` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:37:22.596800 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e141ddb-a0a6-4192-b00b-df7901f68d0b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:37:22.598196 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e141ddb-a0a6-4192-b00b-df7901f68d0b None admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:37:22.603914 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 435/887] 10.222.0.20 () {56 vars in 1087 bytes} [Mon Aug 1 16:37:22 2022] GET /identity/v3/services => generated 1268 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 16:37:22.604705 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e141ddb-a0a6-4192-b00b-df7901f68d0b None admin] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:37:22.713143 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df9239ef-79a3-4e01-82cf-96b905bdefb0 None None] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:37:22.756182 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e141ddb-a0a6-4192-b00b-df7901f68d0b None admin] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:37:22.758137 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6772974e-2cd4-42ca-97a6-9ecf996907e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:37:22.758877 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6772974e-2cd4-42ca-97a6-9ecf996907e0 None admin] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:37:22.759216 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6772974e-2cd4-42ca-97a6-9ecf996907e0 None admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:37:22.759567 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6772974e-2cd4-42ca-97a6-9ecf996907e0 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:37:22.760748 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6772974e-2cd4-42ca-97a6-9ecf996907e0 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:37:22.761758 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6772974e-2cd4-42ca-97a6-9ecf996907e0 None admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:37:22.773985 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application [None req-6772974e-2cd4-42ca-97a6-9ecf996907e0 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 01 16:37:22.773985 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 16:37:22.773985 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 01 16:37:22.773985 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 01 16:37:22.773985 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 01 16:37:22.773985 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 01 16:37:22.773985 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 01 16:37:22.773985 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 01 16:37:22.773985 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 01 16:37:22.773985 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application Aug 01 16:37:22.773985 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 01 16:37:22.773985 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application Aug 01 16:37:22.773985 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 16:37:22.773985 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 16:37:22.773985 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 16:37:22.773985 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 16:37:22.773985 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 16:37:22.775073 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 16:37:22.775073 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 16:37:22.775073 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 16:37:22.775073 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 16:37:22.775073 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 16:37:22.775073 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 16:37:22.775073 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 01 16:37:22.775073 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 01 16:37:22.775073 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 01 16:37:22.775073 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 01 16:37:22.775073 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 16:37:22.775073 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 16:37:22.775073 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 16:37:22.775073 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 16:37:22.775073 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 16:37:22.775073 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 16:37:22.775919 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 16:37:22.775919 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application with Lock( Aug 01 16:37:22.775919 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 16:37:22.775919 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self._enter() Aug 01 16:37:22.775919 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 16:37:22.775919 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 16:37:22.775919 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 16:37:22.775919 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self.creator() Aug 01 16:37:22.775919 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 16:37:22.775919 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application created_value = creator( Aug 01 16:37:22.775919 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 01 16:37:22.775919 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 01 16:37:22.775919 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 01 16:37:22.775919 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application Aug 01 16:37:22.779843 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 453/888] 10.222.0.20 () {56 vars in 1114 bytes} [Mon Aug 1 16:37:22 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 16:37:22.780883 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6772974e-2cd4-42ca-97a6-9ecf996907e0 None admin] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:37:22.783820 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG dbcounter [-] [87058] Writing DB stats keystone:SELECT=100 {{(pid=87058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:37:22.805760 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df62da5-89e6-46bd-b450-f2cf0f91b1a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:37:22.806467 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df62da5-89e6-46bd-b450-f2cf0f91b1a3 None admin] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:37:22.806743 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df62da5-89e6-46bd-b450-f2cf0f91b1a3 None admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:37:22.807011 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df62da5-89e6-46bd-b450-f2cf0f91b1a3 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:37:22.819500 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7df62da5-89e6-46bd-b450-f2cf0f91b1a3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 16:37:22.824882 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application [None req-7df62da5-89e6-46bd-b450-f2cf0f91b1a3 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 01 16:37:22.824882 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 16:37:22.824882 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 16:37:22.824882 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 16:37:22.824882 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 16:37:22.824882 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 16:37:22.824882 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 16:37:22.824882 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 16:37:22.824882 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 16:37:22.824882 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 16:37:22.824882 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 16:37:22.824882 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 16:37:22.824882 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 16:37:22.824882 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 16:37:22.824882 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 16:37:22.824882 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 16:37:22.825836 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 16:37:22.825836 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 16:37:22.825836 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 16:37:22.825836 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 16:37:22.825836 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 16:37:22.825836 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 16:37:22.825836 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 16:37:22.825836 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 16:37:22.825836 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 16:37:22.825836 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 16:37:22.825836 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 16:37:22.825836 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application with Lock( Aug 01 16:37:22.825836 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 16:37:22.825836 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self._enter() Aug 01 16:37:22.825836 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 16:37:22.825836 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 16:37:22.826753 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 16:37:22.826753 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self.creator() Aug 01 16:37:22.826753 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 16:37:22.826753 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application created_value = creator( Aug 01 16:37:22.826753 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 16:37:22.826753 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 16:37:22.826753 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 16:37:22.826753 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 16:37:22.826753 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 16:37:22.826753 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 16:37:22.826753 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 01 16:37:22.826753 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application Aug 01 16:37:22.826753 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 436/889] 10.222.0.20 () {56 vars in 1120 bytes} [Mon Aug 1 16:37:22 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 16:37:22.832087 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df62da5-89e6-46bd-b450-f2cf0f91b1a3 None admin] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:37:22.841569 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG dbcounter [-] [87057] Writing DB stats keystone:SELECT=100 {{(pid=87057) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:37:22.915590 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6772974e-2cd4-42ca-97a6-9ecf996907e0 None admin] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:37:22.938065 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df62da5-89e6-46bd-b450-f2cf0f91b1a3 None admin] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:37:22.952965 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-344559e9-b4b2-418b-9466-89a42e1fcbcc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:37:22.953721 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344559e9-b4b2-418b-9466-89a42e1fcbcc None admin] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:37:22.953999 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344559e9-b4b2-418b-9466-89a42e1fcbcc None admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:37:22.954295 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344559e9-b4b2-418b-9466-89a42e1fcbcc None admin] PATH_INFO: `/v3/services` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:37:22.955011 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-344559e9-b4b2-418b-9466-89a42e1fcbcc None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=87058) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:37:22.955315 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-344559e9-b4b2-418b-9466-89a42e1fcbcc None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:37:22.956871 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-344559e9-b4b2-418b-9466-89a42e1fcbcc None admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:37:22.962804 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 454/890] 10.222.0.20 () {56 vars in 1114 bytes} [Mon Aug 1 16:37:22 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 16:37:22.968052 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-344559e9-b4b2-418b-9466-89a42e1fcbcc None admin] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:37:22.976324 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-044f055a-5ed0-4c9a-9215-9f46c86562b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:37:22.976934 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044f055a-5ed0-4c9a-9215-9f46c86562b8 None admin] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:37:22.977212 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044f055a-5ed0-4c9a-9215-9f46c86562b8 None admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:37:22.977483 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044f055a-5ed0-4c9a-9215-9f46c86562b8 None admin] PATH_INFO: `/v3/roles` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:37:22.982072 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-044f055a-5ed0-4c9a-9215-9f46c86562b8 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=87057) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:37:22.982072 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-044f055a-5ed0-4c9a-9215-9f46c86562b8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:37:22.982072 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-044f055a-5ed0-4c9a-9215-9f46c86562b8 None admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:37:22.994447 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 437/891] 10.222.0.20 () {56 vars in 1115 bytes} [Mon Aug 1 16:37:22 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 16:37:22.999464 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-044f055a-5ed0-4c9a-9215-9f46c86562b8 None admin] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:37:23.043887 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG dbcounter [-] [87058] Writing DB stats keystone:SELECT=100 {{(pid=87058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:37:23.070381 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG dbcounter [-] [87057] Writing DB stats keystone:SELECT=100 {{(pid=87057) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:37:23.081469 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-344559e9-b4b2-418b-9466-89a42e1fcbcc None admin] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:37:23.105037 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-044f055a-5ed0-4c9a-9215-9f46c86562b8 None admin] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:37:23.122460 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf02d4b5-5c03-433c-9aae-03315c1927b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:37:23.123089 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf02d4b5-5c03-433c-9aae-03315c1927b1 None admin] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:37:23.123381 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf02d4b5-5c03-433c-9aae-03315c1927b1 None admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:37:23.123653 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf02d4b5-5c03-433c-9aae-03315c1927b1 None admin] PATH_INFO: `/v3/services` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:37:23.124330 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf02d4b5-5c03-433c-9aae-03315c1927b1 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=87058) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:37:23.124638 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf02d4b5-5c03-433c-9aae-03315c1927b1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:37:23.126005 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf02d4b5-5c03-433c-9aae-03315c1927b1 None admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:37:23.131498 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 455/892] 10.222.0.20 () {56 vars in 1114 bytes} [Mon Aug 1 16:37:22 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 16:37:23.137520 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8800029d-e989-435d-beaa-258ecc039ad7 None None] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:37:23.138066 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8800029d-e989-435d-beaa-258ecc039ad7 None None] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:37:23.138066 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8800029d-e989-435d-beaa-258ecc039ad7 None None] PATH_INFO: `/` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:37:23.139367 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 456/893] 10.222.0.20 () {54 vars in 926 bytes} [Mon Aug 1 16:37:23 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 16:37:23.142179 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8800029d-e989-435d-beaa-258ecc039ad7 None None] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:37:23.150675 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc59a9eb-87da-43f5-b208-3693365ae73e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:37:23.151589 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc59a9eb-87da-43f5-b208-3693365ae73e None admin] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:37:23.151995 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc59a9eb-87da-43f5-b208-3693365ae73e None admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:37:23.152409 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc59a9eb-87da-43f5-b208-3693365ae73e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:37:23.165733 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc59a9eb-87da-43f5-b208-3693365ae73e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:37:23.167255 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc59a9eb-87da-43f5-b208-3693365ae73e None admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:37:23.181988 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application [None req-dc59a9eb-87da-43f5-b208-3693365ae73e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 16:37:23.181988 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 16:37:23.181988 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 16:37:23.181988 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 16:37:23.181988 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 16:37:23.181988 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 16:37:23.181988 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 16:37:23.181988 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 16:37:23.181988 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 16:37:23.181988 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 16:37:23.181988 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 16:37:23.181988 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 16:37:23.181988 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 16:37:23.181988 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 16:37:23.181988 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 16:37:23.182914 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 16:37:23.182914 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 16:37:23.182914 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 16:37:23.182914 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 16:37:23.182914 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 16:37:23.182914 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 16:37:23.182914 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 16:37:23.182914 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 16:37:23.182914 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application with Lock( Aug 01 16:37:23.182914 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 16:37:23.182914 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self._enter() Aug 01 16:37:23.182914 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 16:37:23.182914 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 16:37:23.182914 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 16:37:23.182914 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self.creator() Aug 01 16:37:23.182914 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 16:37:23.182914 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application created_value = creator( Aug 01 16:37:23.183820 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 16:37:23.183820 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 16:37:23.183820 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 16:37:23.183820 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application Aug 01 16:37:23.183820 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 438/894] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Aug 1 16:37:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 01 16:37:23.184502 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a3e398-b39e-4072-80f7-e392e7bb93c4 None None] REQUEST_METHOD: `POST` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:37:23.184610 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a3e398-b39e-4072-80f7-e392e7bb93c4 None None] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:37:23.184902 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a3e398-b39e-4072-80f7-e392e7bb93c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:37:23.202429 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.auth.core [None req-67a3e398-b39e-4072-80f7-e392e7bb93c4 None None] MFA Rules not processed for user `1007f8e379b74e169b98fb7c3c0ba5b4`. Rule list: `[]` (Enabled: `True`). {{(pid=87057) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:37:23.203623 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.fernet_utils [None req-67a3e398-b39e-4072-80f7-e392e7bb93c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87057) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:37:23.258467 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8800029d-e989-435d-beaa-258ecc039ad7 None None] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:37:23.277373 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 439/895] 10.222.0.20 () {58 vars in 1023 bytes} [Mon Aug 1 16:37:23 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 01 16:37:23.278325 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67a3e398-b39e-4072-80f7-e392e7bb93c4 None None] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:37:23.304174 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG dbcounter [-] [87058] Writing DB stats keystone:SELECT=100 {{(pid=87058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:37:23.310734 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd1ad27-46d5-47f2-b624-3b669b26b535 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:37:23.311482 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd1ad27-46d5-47f2-b624-3b669b26b535 None admin] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:37:23.316465 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd1ad27-46d5-47f2-b624-3b669b26b535 None admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:37:23.316465 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd1ad27-46d5-47f2-b624-3b669b26b535 None admin] PATH_INFO: `/v3/endpoints` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:37:23.316465 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bd1ad27-46d5-47f2-b624-3b669b26b535 None admin] RBAC: Adding query filter params (interface=public, service_id=5901105c2551488a948779cb77f3bd48, region_id=RegionOne) {{(pid=87058) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:37:23.316465 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bd1ad27-46d5-47f2-b624-3b669b26b535 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:37:23.316465 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bd1ad27-46d5-47f2-b624-3b669b26b535 None admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:37:23.325311 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 457/896] 10.222.0.20 () {56 vars in 1251 bytes} [Mon Aug 1 16:37:23 2022] GET /identity/v3/endpoints?service_id=5901105c2551488a948779cb77f3bd48&interface=public®ion_id=RegionOne => generated 194 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:37:23.325311 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd1ad27-46d5-47f2-b624-3b669b26b535 None admin] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:37:23.346303 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG dbcounter [-] [87057] Writing DB stats keystone:SELECT=100 {{(pid=87057) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:37:23.399746 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67a3e398-b39e-4072-80f7-e392e7bb93c4 None None] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:37:23.435771 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd1ad27-46d5-47f2-b624-3b669b26b535 None admin] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:37:23.436791 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c298cea-7160-49cb-8e10-5c098ea14029 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:37:23.437443 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c298cea-7160-49cb-8e10-5c098ea14029 None admin] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:37:23.437723 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c298cea-7160-49cb-8e10-5c098ea14029 None admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:37:23.438001 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c298cea-7160-49cb-8e10-5c098ea14029 None admin] PATH_INFO: `/v3/domains` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:37:23.438719 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c298cea-7160-49cb-8e10-5c098ea14029 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87057) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:37:23.439026 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c298cea-7160-49cb-8e10-5c098ea14029 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:37:23.440399 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c298cea-7160-49cb-8e10-5c098ea14029 None admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:37:23.452590 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 440/897] 10.222.0.20 () {56 vars in 1109 bytes} [Mon Aug 1 16:37:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:37:23.453421 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c298cea-7160-49cb-8e10-5c098ea14029 None admin] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:37:23.478938 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc4c10e3-fa3d-43e1-8aca-efaf7b5723ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:37:23.479889 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4c10e3-fa3d-43e1-8aca-efaf7b5723ed None admin] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:37:23.480351 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4c10e3-fa3d-43e1-8aca-efaf7b5723ed None admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:37:23.480818 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4c10e3-fa3d-43e1-8aca-efaf7b5723ed None admin] PATH_INFO: `/v3/services/object-store` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:37:23.481519 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc4c10e3-fa3d-43e1-8aca-efaf7b5723ed None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:37:23.483027 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc4c10e3-fa3d-43e1-8aca-efaf7b5723ed None admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:37:23.492130 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application [None req-fc4c10e3-fa3d-43e1-8aca-efaf7b5723ed None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 01 16:37:23.492130 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 16:37:23.492130 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 01 16:37:23.492130 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 01 16:37:23.492130 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 01 16:37:23.492130 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 01 16:37:23.492130 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 01 16:37:23.492130 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 01 16:37:23.492130 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 01 16:37:23.492130 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application Aug 01 16:37:23.492130 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 01 16:37:23.492130 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application Aug 01 16:37:23.492130 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 16:37:23.492130 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 16:37:23.492130 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 16:37:23.492130 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 16:37:23.492130 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 16:37:23.493133 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 16:37:23.493133 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 16:37:23.493133 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 16:37:23.493133 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 16:37:23.493133 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 16:37:23.493133 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 16:37:23.493133 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 01 16:37:23.493133 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 01 16:37:23.493133 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 01 16:37:23.493133 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 01 16:37:23.493133 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 16:37:23.493133 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 16:37:23.493133 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 16:37:23.493133 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 16:37:23.493133 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 16:37:23.493133 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 16:37:23.493987 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 16:37:23.493987 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application with Lock( Aug 01 16:37:23.493987 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 16:37:23.493987 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self._enter() Aug 01 16:37:23.493987 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 16:37:23.493987 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 16:37:23.493987 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 16:37:23.493987 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self.creator() Aug 01 16:37:23.493987 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 16:37:23.493987 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application created_value = creator( Aug 01 16:37:23.493987 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 01 16:37:23.493987 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 01 16:37:23.493987 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 01 16:37:23.493987 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application Aug 01 16:37:23.498345 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 458/898] 10.222.0.20 () {56 vars in 1126 bytes} [Mon Aug 1 16:37:23 2022] GET /identity/v3/services/object-store => generated 93 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 16:37:23.499166 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc4c10e3-fa3d-43e1-8aca-efaf7b5723ed None admin] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:37:23.561120 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG dbcounter [-] [87058] Writing DB stats keystone:SELECT=100 {{(pid=87058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:37:23.563189 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c298cea-7160-49cb-8e10-5c098ea14029 None admin] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:37:23.574400 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG dbcounter [-] [87057] Writing DB stats keystone:SELECT=100 {{(pid=87057) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:37:23.600346 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5916e641-6e1e-4070-91a6-68574085730a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:37:23.601024 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5916e641-6e1e-4070-91a6-68574085730a None admin] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:37:23.601305 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5916e641-6e1e-4070-91a6-68574085730a None admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:37:23.601579 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5916e641-6e1e-4070-91a6-68574085730a None admin] PATH_INFO: `/v3/services` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:37:23.602258 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5916e641-6e1e-4070-91a6-68574085730a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:37:23.603624 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5916e641-6e1e-4070-91a6-68574085730a None admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:37:23.607415 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc4c10e3-fa3d-43e1-8aca-efaf7b5723ed None admin] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:37:23.618122 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 441/899] 10.222.0.20 () {56 vars in 1087 bytes} [Mon Aug 1 16:37:23 2022] GET /identity/v3/services => generated 1268 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 16:37:23.618122 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5916e641-6e1e-4070-91a6-68574085730a None admin] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:37:23.672892 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bffd6b81-37ba-4461-bced-15a07c2c40fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:37:23.673716 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bffd6b81-37ba-4461-bced-15a07c2c40fa None admin] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:37:23.674095 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bffd6b81-37ba-4461-bced-15a07c2c40fa None admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:37:23.674440 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bffd6b81-37ba-4461-bced-15a07c2c40fa None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:37:23.688510 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bffd6b81-37ba-4461-bced-15a07c2c40fa None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:37:23.690393 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bffd6b81-37ba-4461-bced-15a07c2c40fa None admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:37:23.710081 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application [None req-bffd6b81-37ba-4461-bced-15a07c2c40fa None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 01 16:37:23.710081 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 16:37:23.710081 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 16:37:23.710081 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 16:37:23.710081 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 16:37:23.710081 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 16:37:23.710081 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 16:37:23.710081 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 16:37:23.710081 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 16:37:23.710081 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 16:37:23.710081 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 16:37:23.710081 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 16:37:23.710081 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 01 16:37:23.710081 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 01 16:37:23.710081 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 01 16:37:23.710081 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 01 16:37:23.711105 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 16:37:23.711105 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 16:37:23.711105 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 01 16:37:23.711105 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 16:37:23.711105 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 01 16:37:23.711105 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 16:37:23.711105 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 16:37:23.711105 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 16:37:23.711105 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 16:37:23.711105 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 16:37:23.711105 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 16:37:23.711105 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application with Lock( Aug 01 16:37:23.711105 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 16:37:23.711105 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self._enter() Aug 01 16:37:23.711105 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 16:37:23.711105 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 16:37:23.711105 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 16:37:23.711105 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self.creator() Aug 01 16:37:23.712062 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 16:37:23.712062 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application created_value = creator( Aug 01 16:37:23.712062 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 01 16:37:23.712062 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 01 16:37:23.712062 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 01 16:37:23.712062 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 01 16:37:23.712062 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 01 16:37:23.712062 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 01 16:37:23.712062 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 01 16:37:23.712062 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application Aug 01 16:37:23.714197 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 459/900] 10.222.0.20 () {56 vars in 1117 bytes} [Mon Aug 1 16:37:23 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 16:37:23.718082 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bffd6b81-37ba-4461-bced-15a07c2c40fa None admin] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:37:23.742649 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5916e641-6e1e-4070-91a6-68574085730a None admin] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:37:23.794833 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ae4d68-2195-483b-9d4a-21d321d8bb96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:37:23.796145 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ae4d68-2195-483b-9d4a-21d321d8bb96 None admin] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:37:23.796464 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ae4d68-2195-483b-9d4a-21d321d8bb96 None admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:37:23.796699 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ae4d68-2195-483b-9d4a-21d321d8bb96 None admin] PATH_INFO: `/v3/services` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:37:23.797394 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6ae4d68-2195-483b-9d4a-21d321d8bb96 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=87057) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:37:23.797700 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6ae4d68-2195-483b-9d4a-21d321d8bb96 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:37:23.799330 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6ae4d68-2195-483b-9d4a-21d321d8bb96 None admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:37:23.806980 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 442/901] 10.222.0.20 () {56 vars in 1122 bytes} [Mon Aug 1 16:37:23 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:37:23.813390 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ae4d68-2195-483b-9d4a-21d321d8bb96 None admin] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:37:23.849210 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bffd6b81-37ba-4461-bced-15a07c2c40fa None admin] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:37:23.862855 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG dbcounter [-] [87057] Writing DB stats keystone:SELECT=100 {{(pid=87057) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:37:23.869729 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG dbcounter [-] [87058] Writing DB stats keystone:SELECT=100 {{(pid=87058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:37:23.890010 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4abe96ea-e50d-4250-a808-fd52f48d9fd7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:37:23.890922 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4abe96ea-e50d-4250-a808-fd52f48d9fd7 None admin] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:37:23.892305 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4abe96ea-e50d-4250-a808-fd52f48d9fd7 None admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:37:23.892585 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4abe96ea-e50d-4250-a808-fd52f48d9fd7 None admin] PATH_INFO: `/v3/users` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:37:23.903646 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4abe96ea-e50d-4250-a808-fd52f48d9fd7 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=87058) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:37:23.904301 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4abe96ea-e50d-4250-a808-fd52f48d9fd7 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:37:23.905837 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4abe96ea-e50d-4250-a808-fd52f48d9fd7 None admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:37:23.927166 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 460/902] 10.222.0.20 () {56 vars in 1149 bytes} [Mon Aug 1 16:37:23 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:37:23.936488 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4abe96ea-e50d-4250-a808-fd52f48d9fd7 None admin] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:37:23.960526 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ae4d68-2195-483b-9d4a-21d321d8bb96 None admin] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:37:24.004718 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d289f6a7-c5c2-425c-9ca2-c8162ce681a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:37:24.006284 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d289f6a7-c5c2-425c-9ca2-c8162ce681a3 None admin] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:37:24.006768 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d289f6a7-c5c2-425c-9ca2-c8162ce681a3 None admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:37:24.007365 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d289f6a7-c5c2-425c-9ca2-c8162ce681a3 None admin] PATH_INFO: `/v3/services` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:37:24.008351 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d289f6a7-c5c2-425c-9ca2-c8162ce681a3 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=87057) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:37:24.008867 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d289f6a7-c5c2-425c-9ca2-c8162ce681a3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:37:24.010556 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d289f6a7-c5c2-425c-9ca2-c8162ce681a3 None admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:37:24.016442 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 443/903] 10.222.0.20 () {56 vars in 1122 bytes} [Mon Aug 1 16:37:23 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:37:24.056617 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4abe96ea-e50d-4250-a808-fd52f48d9fd7 None admin] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:37:24.096003 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b8817d3-6d2e-4781-b1ff-8767334db8f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:37:24.096554 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8817d3-6d2e-4781-b1ff-8767334db8f4 None admin] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:37:24.096828 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8817d3-6d2e-4781-b1ff-8767334db8f4 None admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:37:24.097098 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8817d3-6d2e-4781-b1ff-8767334db8f4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:37:24.148590 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG dbcounter [-] [87058] Writing DB stats keystone:SELECT=100 {{(pid=87058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:37:24.173405 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b8817d3-6d2e-4781-b1ff-8767334db8f4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:37:24.174880 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b8817d3-6d2e-4781-b1ff-8767334db8f4 None admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:37:24.278389 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 461/904] 10.222.0.20 () {58 vars in 1282 bytes} [Mon Aug 1 16:37:23 2022] GET /identity/v3/auth/tokens => generated 1362 bytes in 342 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 01 16:37:24.283977 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d289f6a7-c5c2-425c-9ca2-c8162ce681a3 None admin] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:37:24.391950 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d289f6a7-c5c2-425c-9ca2-c8162ce681a3 None admin] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:37:24.393848 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG dbcounter [-] [87057] Writing DB stats keystone:SELECT=100 {{(pid=87057) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:37:24.429388 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1052812-5f47-4626-a69d-4cc886cda989 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:37:24.430703 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1052812-5f47-4626-a69d-4cc886cda989 None admin] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:37:24.431640 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1052812-5f47-4626-a69d-4cc886cda989 None admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:37:24.432126 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1052812-5f47-4626-a69d-4cc886cda989 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:37:24.444042 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1052812-5f47-4626-a69d-4cc886cda989 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:37:24.445816 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1052812-5f47-4626-a69d-4cc886cda989 None admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:37:24.462278 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application [None req-e1052812-5f47-4626-a69d-4cc886cda989 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 16:37:24.462278 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 16:37:24.462278 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 16:37:24.462278 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 16:37:24.462278 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 16:37:24.462278 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 16:37:24.462278 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 16:37:24.462278 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 16:37:24.462278 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 16:37:24.462278 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 16:37:24.462278 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 16:37:24.462278 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 16:37:24.462278 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 16:37:24.462278 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 16:37:24.462278 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 16:37:24.463239 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 16:37:24.463239 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 16:37:24.463239 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 16:37:24.463239 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 16:37:24.463239 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 16:37:24.463239 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 16:37:24.463239 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 16:37:24.463239 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 16:37:24.463239 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application with Lock( Aug 01 16:37:24.463239 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 16:37:24.463239 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self._enter() Aug 01 16:37:24.463239 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 16:37:24.463239 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 16:37:24.463239 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 16:37:24.463239 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self.creator() Aug 01 16:37:24.463239 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 16:37:24.463239 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application created_value = creator( Aug 01 16:37:24.464160 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 16:37:24.464160 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 16:37:24.464160 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 16:37:24.464160 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application Aug 01 16:37:24.466391 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 444/905] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Aug 1 16:37:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 16:37:24.472169 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b8817d3-6d2e-4781-b1ff-8767334db8f4 None admin] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:37:24.550667 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG dbcounter [-] [87058] Writing DB stats keystone:SELECT=100 {{(pid=87058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:37:24.574847 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b8817d3-6d2e-4781-b1ff-8767334db8f4 None admin] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:37:24.610881 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f9cc803-4ddf-4118-a75e-3da206bebb72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:37:24.611906 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9cc803-4ddf-4118-a75e-3da206bebb72 None admin] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:37:24.612394 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9cc803-4ddf-4118-a75e-3da206bebb72 None admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:37:24.612894 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9cc803-4ddf-4118-a75e-3da206bebb72 None admin] PATH_INFO: `/v3/domains` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:37:24.613800 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f9cc803-4ddf-4118-a75e-3da206bebb72 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87058) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:37:24.614345 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f9cc803-4ddf-4118-a75e-3da206bebb72 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:37:24.616009 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f9cc803-4ddf-4118-a75e-3da206bebb72 None admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:37:24.627997 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 462/906] 10.222.0.20 () {56 vars in 1109 bytes} [Mon Aug 1 16:37:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:37:24.634442 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1052812-5f47-4626-a69d-4cc886cda989 None admin] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:37:24.744767 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104f3677-ed67-44e1-9681-ffa045cfa95d None None] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:37:24.745700 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1052812-5f47-4626-a69d-4cc886cda989 None admin] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:37:24.745904 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104f3677-ed67-44e1-9681-ffa045cfa95d None None] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:37:24.746402 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104f3677-ed67-44e1-9681-ffa045cfa95d None None] PATH_INFO: `/` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:37:24.749268 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 463/907] 10.222.0.20 () {54 vars in 926 bytes} [Mon Aug 1 16:37:24 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 01 16:37:24.753424 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9169cc60-7435-4dcd-a9ad-aa5927208dfa None None] REQUEST_METHOD: `POST` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:37:24.753735 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9169cc60-7435-4dcd-a9ad-aa5927208dfa None None] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:37:24.754013 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9169cc60-7435-4dcd-a9ad-aa5927208dfa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:37:24.773914 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.auth.core [None req-9169cc60-7435-4dcd-a9ad-aa5927208dfa None None] MFA Rules not processed for user `1007f8e379b74e169b98fb7c3c0ba5b4`. Rule list: `[]` (Enabled: `True`). {{(pid=87058) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:37:24.775146 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.fernet_utils [None req-9169cc60-7435-4dcd-a9ad-aa5927208dfa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:37:24.785109 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7bdf865-ce31-49f3-98aa-4b98e7fb5166 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:37:24.786067 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7bdf865-ce31-49f3-98aa-4b98e7fb5166 None admin] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:37:24.786601 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7bdf865-ce31-49f3-98aa-4b98e7fb5166 None admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:37:24.787099 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7bdf865-ce31-49f3-98aa-4b98e7fb5166 None admin] PATH_INFO: `/v3/projects/service` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:37:24.794597 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7bdf865-ce31-49f3-98aa-4b98e7fb5166 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:37:24.796368 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7bdf865-ce31-49f3-98aa-4b98e7fb5166 None admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:37:24.809599 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application [None req-a7bdf865-ce31-49f3-98aa-4b98e7fb5166 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 16:37:24.809599 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 16:37:24.809599 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 16:37:24.809599 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 16:37:24.809599 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 16:37:24.809599 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 16:37:24.809599 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 16:37:24.809599 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 16:37:24.809599 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 16:37:24.809599 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 16:37:24.809599 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 16:37:24.809599 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 16:37:24.809599 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 01 16:37:24.809599 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 01 16:37:24.809599 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 01 16:37:24.810538 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 01 16:37:24.810538 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 16:37:24.810538 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 16:37:24.810538 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 16:37:24.810538 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 16:37:24.810538 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 16:37:24.810538 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 16:37:24.810538 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 16:37:24.810538 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application with Lock( Aug 01 16:37:24.810538 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 16:37:24.810538 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self._enter() Aug 01 16:37:24.810538 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 16:37:24.810538 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 16:37:24.810538 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 16:37:24.810538 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self.creator() Aug 01 16:37:24.810538 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 16:37:24.810538 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application created_value = creator( Aug 01 16:37:24.811631 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 01 16:37:24.811631 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 01 16:37:24.811631 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 01 16:37:24.811631 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 01 16:37:24.811631 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 01 16:37:24.811631 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 01 16:37:24.811631 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 16:37:24.811631 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application Aug 01 16:37:24.811631 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 445/908] 10.222.0.20 () {56 vars in 1111 bytes} [Mon Aug 1 16:37:24 2022] GET /identity/v3/projects/service => generated 88 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 16:37:24.822055 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7bdf865-ce31-49f3-98aa-4b98e7fb5166 None admin] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:37:24.842437 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG dbcounter [-] [87057] Writing DB stats keystone:SELECT=100 {{(pid=87057) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:37:24.853431 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 464/909] 10.222.0.20 () {58 vars in 1023 bytes} [Mon Aug 1 16:37:24 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 01 16:37:24.875630 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9169cc60-7435-4dcd-a9ad-aa5927208dfa None None] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:37:24.947438 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7bdf865-ce31-49f3-98aa-4b98e7fb5166 None admin] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:37:24.967114 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG dbcounter [-] [87058] Writing DB stats keystone:SELECT=100 {{(pid=87058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:37:24.995895 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ecc3192-7942-4eae-ad25-2832573470f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:37:24.997148 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ecc3192-7942-4eae-ad25-2832573470f7 None admin] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:37:24.997723 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ecc3192-7942-4eae-ad25-2832573470f7 None admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:37:24.998466 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ecc3192-7942-4eae-ad25-2832573470f7 None admin] PATH_INFO: `/v3/projects` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:37:25.000115 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ecc3192-7942-4eae-ad25-2832573470f7 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87057) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:37:25.000646 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ecc3192-7942-4eae-ad25-2832573470f7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:37:25.003209 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ecc3192-7942-4eae-ad25-2832573470f7 None admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:37:25.008105 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9169cc60-7435-4dcd-a9ad-aa5927208dfa None None] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:37:25.018827 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 446/910] 10.222.0.20 () {56 vars in 1148 bytes} [Mon Aug 1 16:37:24 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 16:37:25.024969 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ecc3192-7942-4eae-ad25-2832573470f7 None admin] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:37:25.050937 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e4f3ff3-a3e9-490c-aa9d-b55d2bc1c7a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:37:25.051912 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4f3ff3-a3e9-490c-aa9d-b55d2bc1c7a3 None admin] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:37:25.054826 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4f3ff3-a3e9-490c-aa9d-b55d2bc1c7a3 None admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:37:25.055122 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4f3ff3-a3e9-490c-aa9d-b55d2bc1c7a3 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:37:25.055775 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e4f3ff3-a3e9-490c-aa9d-b55d2bc1c7a3 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:37:25.057398 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e4f3ff3-a3e9-490c-aa9d-b55d2bc1c7a3 None admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:37:25.068942 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application [None req-7e4f3ff3-a3e9-490c-aa9d-b55d2bc1c7a3 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 01 16:37:25.068942 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 16:37:25.068942 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 01 16:37:25.068942 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 01 16:37:25.068942 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 01 16:37:25.068942 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 01 16:37:25.068942 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 01 16:37:25.068942 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 01 16:37:25.068942 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 01 16:37:25.068942 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application Aug 01 16:37:25.068942 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 01 16:37:25.068942 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application Aug 01 16:37:25.068942 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 16:37:25.068942 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 16:37:25.068942 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 16:37:25.068942 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 16:37:25.069917 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 16:37:25.069917 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 16:37:25.069917 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 16:37:25.069917 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 16:37:25.069917 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 16:37:25.069917 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 16:37:25.069917 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 16:37:25.069917 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 01 16:37:25.069917 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 01 16:37:25.069917 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 01 16:37:25.069917 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 01 16:37:25.069917 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 16:37:25.069917 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 16:37:25.069917 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 16:37:25.069917 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 16:37:25.069917 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 16:37:25.070818 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 16:37:25.070818 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 16:37:25.070818 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application with Lock( Aug 01 16:37:25.070818 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 16:37:25.070818 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self._enter() Aug 01 16:37:25.070818 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 16:37:25.070818 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 16:37:25.070818 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 16:37:25.070818 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self.creator() Aug 01 16:37:25.070818 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 16:37:25.070818 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application created_value = creator( Aug 01 16:37:25.070818 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 01 16:37:25.070818 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 01 16:37:25.070818 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 01 16:37:25.070818 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application Aug 01 16:37:25.074113 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 465/911] 10.222.0.20 () {56 vars in 1132 bytes} [Mon Aug 1 16:37:24 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 01 16:37:25.078982 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e4f3ff3-a3e9-490c-aa9d-b55d2bc1c7a3 None admin] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:37:25.123646 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG dbcounter [-] [87057] Writing DB stats keystone:SELECT=100 {{(pid=87057) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:37:25.160740 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ecc3192-7942-4eae-ad25-2832573470f7 None admin] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:37:25.199048 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e4f3ff3-a3e9-490c-aa9d-b55d2bc1c7a3 None admin] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:37:25.213555 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e3f17ae-8838-436f-a983-a61adbb9b9d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:37:25.214723 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3f17ae-8838-436f-a983-a61adbb9b9d2 None admin] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:37:25.215638 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3f17ae-8838-436f-a983-a61adbb9b9d2 None admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:37:25.216367 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3f17ae-8838-436f-a983-a61adbb9b9d2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:37:25.217352 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e3f17ae-8838-436f-a983-a61adbb9b9d2 None admin] RBAC: Adding query filter params (role.id=653d2ccc68e8415191e22643427e28bc, scope.project.id=7b992d6a65df4bf9b27c13192d57c5f0, user.id=b6a93d1c6c82489099adc8b413280fc0) {{(pid=87057) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:37:25.218052 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e3f17ae-8838-436f-a983-a61adbb9b9d2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:37:25.220071 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e3f17ae-8838-436f-a983-a61adbb9b9d2 None admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:37:25.229380 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 447/912] 10.222.0.20 () {56 vars in 1374 bytes} [Mon Aug 1 16:37:25 2022] GET /identity/v3/role_assignments?user.id=b6a93d1c6c82489099adc8b413280fc0&scope.project.id=7b992d6a65df4bf9b27c13192d57c5f0&role.id=653d2ccc68e8415191e22643427e28bc => generated 259 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 16:37:25.242061 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG dbcounter [-] [87058] Writing DB stats keystone:SELECT=100 {{(pid=87058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:37:25.242381 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ffd6af9-da5d-45a9-8166-9367536f04f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:37:25.243193 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ffd6af9-da5d-45a9-8166-9367536f04f4 None admin] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:37:25.243568 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ffd6af9-da5d-45a9-8166-9367536f04f4 None admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:37:25.243933 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ffd6af9-da5d-45a9-8166-9367536f04f4 None admin] PATH_INFO: `/v3/services` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:37:25.244904 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ffd6af9-da5d-45a9-8166-9367536f04f4 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=87058) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:37:25.245303 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ffd6af9-da5d-45a9-8166-9367536f04f4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:37:25.252139 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ffd6af9-da5d-45a9-8166-9367536f04f4 None admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:37:25.258231 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 466/913] 10.222.0.20 () {56 vars in 1126 bytes} [Mon Aug 1 16:37:25 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:37:25.260654 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e3f17ae-8838-436f-a983-a61adbb9b9d2 None admin] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:37:25.378686 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e3f17ae-8838-436f-a983-a61adbb9b9d2 None admin] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:37:25.417278 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ec29869-a5e2-4494-8d15-ad2952e992ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:37:25.418301 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec29869-a5e2-4494-8d15-ad2952e992ee None admin] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:37:25.418789 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec29869-a5e2-4494-8d15-ad2952e992ee None admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:37:25.419263 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec29869-a5e2-4494-8d15-ad2952e992ee None admin] PATH_INFO: `/v3/services` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:37:25.420167 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ec29869-a5e2-4494-8d15-ad2952e992ee None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=87057) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:37:25.420683 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ec29869-a5e2-4494-8d15-ad2952e992ee None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:37:25.422279 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ec29869-a5e2-4494-8d15-ad2952e992ee None admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:37:25.428091 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 448/914] 10.222.0.20 () {56 vars in 1126 bytes} [Mon Aug 1 16:37:25 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:37:25.434256 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ffd6af9-da5d-45a9-8166-9367536f04f4 None admin] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:37:25.540712 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ffd6af9-da5d-45a9-8166-9367536f04f4 None admin] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:37:25.576803 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f9b60a6-7aca-4463-9021-f3b951fa712f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:37:25.577424 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9b60a6-7aca-4463-9021-f3b951fa712f None admin] REQUEST_METHOD: `POST` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:37:25.577698 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9b60a6-7aca-4463-9021-f3b951fa712f None admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:37:25.577965 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9b60a6-7aca-4463-9021-f3b951fa712f None admin] PATH_INFO: `/v3/endpoints` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:37:25.578713 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f9b60a6-7aca-4463-9021-f3b951fa712f None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:37:25.580068 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f9b60a6-7aca-4463-9021-f3b951fa712f None admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:37:25.693613 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 467/915] 10.222.0.20 () {60 vars in 1144 bytes} [Mon Aug 1 16:37:25 2022] POST /identity/v3/endpoints => generated 347 bytes in 255 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 16:37:25.829428 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c724b101-a0be-4f4e-87cd-0fe0be917425 None None] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:37:25.829932 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c724b101-a0be-4f4e-87cd-0fe0be917425 None None] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:37:25.829932 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c724b101-a0be-4f4e-87cd-0fe0be917425 None None] PATH_INFO: `/` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:37:25.831590 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 449/916] 10.222.0.20 () {54 vars in 926 bytes} [Mon Aug 1 16:37:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 16:37:25.838709 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19f3d41-76df-4295-b4e0-1e8f567c974f None None] REQUEST_METHOD: `POST` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:37:25.839195 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19f3d41-76df-4295-b4e0-1e8f567c974f None None] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:37:25.839271 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19f3d41-76df-4295-b4e0-1e8f567c974f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:37:25.858723 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.auth.core [None req-f19f3d41-76df-4295-b4e0-1e8f567c974f None None] MFA Rules not processed for user `1007f8e379b74e169b98fb7c3c0ba5b4`. Rule list: `[]` (Enabled: `True`). {{(pid=87058) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:37:25.859929 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.fernet_utils [None req-f19f3d41-76df-4295-b4e0-1e8f567c974f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:37:25.918236 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG dbcounter [-] [87058] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:37:25.945776 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 468/917] 10.222.0.20 () {58 vars in 1023 bytes} [Mon Aug 1 16:37:25 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 01 16:37:25.967886 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c724b101-a0be-4f4e-87cd-0fe0be917425 None None] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:37:25.971531 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG dbcounter [-] [87057] Writing DB stats keystone:SELECT=100 {{(pid=87057) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:37:26.076391 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c724b101-a0be-4f4e-87cd-0fe0be917425 None None] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:37:26.112014 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd54f99-5ad2-4fe2-8d8f-321b6ab0a6cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:37:26.112979 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd54f99-5ad2-4fe2-8d8f-321b6ab0a6cc None admin] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:37:26.113477 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd54f99-5ad2-4fe2-8d8f-321b6ab0a6cc None admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:37:26.113966 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd54f99-5ad2-4fe2-8d8f-321b6ab0a6cc None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:37:26.115741 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bd54f99-5ad2-4fe2-8d8f-321b6ab0a6cc None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:37:26.117317 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bd54f99-5ad2-4fe2-8d8f-321b6ab0a6cc None admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:37:26.126789 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application [None req-8bd54f99-5ad2-4fe2-8d8f-321b6ab0a6cc None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 01 16:37:26.126789 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 16:37:26.126789 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 01 16:37:26.126789 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 01 16:37:26.126789 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 01 16:37:26.126789 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 01 16:37:26.126789 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 01 16:37:26.126789 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 01 16:37:26.126789 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 01 16:37:26.126789 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application Aug 01 16:37:26.126789 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 01 16:37:26.126789 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application Aug 01 16:37:26.126789 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 16:37:26.126789 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 16:37:26.126789 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 16:37:26.126789 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 16:37:26.126789 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 16:37:26.127793 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 16:37:26.127793 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 16:37:26.127793 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 16:37:26.127793 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 16:37:26.127793 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 16:37:26.127793 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 16:37:26.127793 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 01 16:37:26.127793 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 01 16:37:26.127793 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 01 16:37:26.127793 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 01 16:37:26.127793 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 16:37:26.127793 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 16:37:26.127793 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 16:37:26.127793 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 16:37:26.127793 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 16:37:26.127793 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 16:37:26.128641 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 16:37:26.128641 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application with Lock( Aug 01 16:37:26.128641 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 16:37:26.128641 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self._enter() Aug 01 16:37:26.128641 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 16:37:26.128641 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 16:37:26.128641 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 16:37:26.128641 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self.creator() Aug 01 16:37:26.128641 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 16:37:26.128641 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application created_value = creator( Aug 01 16:37:26.128641 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 01 16:37:26.128641 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 01 16:37:26.128641 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 01 16:37:26.128641 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application Aug 01 16:37:26.131860 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 450/918] 10.222.0.20 () {56 vars in 1114 bytes} [Mon Aug 1 16:37:25 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 16:37:26.138144 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f19f3d41-76df-4295-b4e0-1e8f567c974f None None] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:37:26.151522 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1dc47f-4963-41f8-847f-659633b6266b None None] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:37:26.151809 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1dc47f-4963-41f8-847f-659633b6266b None None] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:37:26.152080 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1dc47f-4963-41f8-847f-659633b6266b None None] PATH_INFO: `/` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:37:26.153670 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 451/919] 10.222.0.20 () {54 vars in 926 bytes} [Mon Aug 1 16:37:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 16:37:26.158944 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291d80f0-1339-44cb-b82f-f2726ffe9437 None None] REQUEST_METHOD: `POST` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:37:26.159306 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291d80f0-1339-44cb-b82f-f2726ffe9437 None None] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:37:26.159678 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291d80f0-1339-44cb-b82f-f2726ffe9437 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:37:26.179066 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.auth.core [None req-291d80f0-1339-44cb-b82f-f2726ffe9437 None None] MFA Rules not processed for user `1007f8e379b74e169b98fb7c3c0ba5b4`. Rule list: `[]` (Enabled: `True`). {{(pid=87057) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:37:26.180363 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.fernet_utils [None req-291d80f0-1339-44cb-b82f-f2726ffe9437 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87057) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:37:26.245623 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG dbcounter [-] [87057] Writing DB stats keystone:SELECT=100 {{(pid=87057) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:37:26.258395 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 452/920] 10.222.0.20 () {58 vars in 1023 bytes} [Mon Aug 1 16:37:26 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 01 16:37:26.263893 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f19f3d41-76df-4295-b4e0-1e8f567c974f None None] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:37:26.278503 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-291d80f0-1339-44cb-b82f-f2726ffe9437 None None] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:37:26.307527 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7298ec2-3391-47d1-a6cb-b9083f77d3dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:37:26.308682 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7298ec2-3391-47d1-a6cb-b9083f77d3dc None admin] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:37:26.309465 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7298ec2-3391-47d1-a6cb-b9083f77d3dc None admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:37:26.309924 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7298ec2-3391-47d1-a6cb-b9083f77d3dc None admin] PATH_INFO: `/v3/services` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:37:26.310843 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7298ec2-3391-47d1-a6cb-b9083f77d3dc None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=87058) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:37:26.311328 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7298ec2-3391-47d1-a6cb-b9083f77d3dc None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:37:26.312877 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7298ec2-3391-47d1-a6cb-b9083f77d3dc None admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:37:26.318659 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 469/921] 10.222.0.20 () {56 vars in 1114 bytes} [Mon Aug 1 16:37:26 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:37:26.323870 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7298ec2-3391-47d1-a6cb-b9083f77d3dc None admin] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:37:26.377240 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG dbcounter [-] [87058] Writing DB stats keystone:SELECT=100 {{(pid=87058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:37:26.406315 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-291d80f0-1339-44cb-b82f-f2726ffe9437 None None] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:37:26.445070 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da2dee55-0a4f-4c4f-ac9d-3f529b494c45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:37:26.445697 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2dee55-0a4f-4c4f-ac9d-3f529b494c45 None admin] REQUEST_METHOD: `POST` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:37:26.445977 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2dee55-0a4f-4c4f-ac9d-3f529b494c45 None admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:37:26.446281 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2dee55-0a4f-4c4f-ac9d-3f529b494c45 None admin] PATH_INFO: `/v3/services` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:37:26.447078 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da2dee55-0a4f-4c4f-ac9d-3f529b494c45 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:37:26.448622 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da2dee55-0a4f-4c4f-ac9d-3f529b494c45 None admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:37:26.454055 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7298ec2-3391-47d1-a6cb-b9083f77d3dc None admin] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:37:26.490896 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35cd2ca0-c6c1-415b-8c84-03f5b1b7c277 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:37:26.491518 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35cd2ca0-c6c1-415b-8c84-03f5b1b7c277 None admin] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:37:26.491793 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35cd2ca0-c6c1-415b-8c84-03f5b1b7c277 None admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:37:26.492066 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35cd2ca0-c6c1-415b-8c84-03f5b1b7c277 None admin] PATH_INFO: `/v3/services` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:37:26.492739 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35cd2ca0-c6c1-415b-8c84-03f5b1b7c277 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=87058) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:37:26.493041 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35cd2ca0-c6c1-415b-8c84-03f5b1b7c277 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:37:26.494441 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35cd2ca0-c6c1-415b-8c84-03f5b1b7c277 None admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:37:26.499789 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 470/922] 10.222.0.20 () {56 vars in 1114 bytes} [Mon Aug 1 16:37:26 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:37:26.505080 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35cd2ca0-c6c1-415b-8c84-03f5b1b7c277 None admin] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:37:26.530087 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 453/923] 10.222.0.20 () {60 vars in 1141 bytes} [Mon Aug 1 16:37:26 2022] POST /identity/v3/services => generated 241 bytes in 250 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 16:37:26.631879 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35cd2ca0-c6c1-415b-8c84-03f5b1b7c277 None admin] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:37:26.634715 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG dbcounter [-] [87058] Writing DB stats keystone:SELECT=100 {{(pid=87058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:37:26.679150 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b01126c2-495b-43e8-8312-0da2dac5276d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:37:26.679835 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01126c2-495b-43e8-8312-0da2dac5276d None admin] REQUEST_METHOD: `POST` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:37:26.680117 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01126c2-495b-43e8-8312-0da2dac5276d None admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:37:26.680385 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01126c2-495b-43e8-8312-0da2dac5276d None admin] PATH_INFO: `/v3/endpoints` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:37:26.681101 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b01126c2-495b-43e8-8312-0da2dac5276d None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:37:26.682515 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b01126c2-495b-43e8-8312-0da2dac5276d None admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:37:26.776445 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 471/924] 10.222.0.20 () {60 vars in 1144 bytes} [Mon Aug 1 16:37:26 2022] POST /identity/v3/endpoints => generated 346 bytes in 272 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 16:37:27.277674 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71ece25-8b02-4810-bcd9-dd243373e96d None None] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:37:27.278220 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71ece25-8b02-4810-bcd9-dd243373e96d None None] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:37:27.278327 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71ece25-8b02-4810-bcd9-dd243373e96d None None] PATH_INFO: `/` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:37:27.279881 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 454/925] 10.222.0.20 () {54 vars in 926 bytes} [Mon Aug 1 16:37:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 16:37:27.284923 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47f48d7-705e-404a-90d0-947c8576afd1 None None] REQUEST_METHOD: `POST` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:37:27.285326 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47f48d7-705e-404a-90d0-947c8576afd1 None None] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:37:27.285495 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47f48d7-705e-404a-90d0-947c8576afd1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:37:27.304447 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.auth.core [None req-d47f48d7-705e-404a-90d0-947c8576afd1 None None] MFA Rules not processed for user `1007f8e379b74e169b98fb7c3c0ba5b4`. Rule list: `[]` (Enabled: `True`). {{(pid=87058) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:37:27.305630 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.fernet_utils [None req-d47f48d7-705e-404a-90d0-947c8576afd1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:37:27.382199 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 472/926] 10.222.0.20 () {58 vars in 1023 bytes} [Mon Aug 1 16:37:27 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 01 16:37:27.404065 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d71ece25-8b02-4810-bcd9-dd243373e96d None None] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:37:27.466764 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG dbcounter [-] [87057] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87057) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:37:27.513106 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d71ece25-8b02-4810-bcd9-dd243373e96d None None] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:37:27.554660 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b9b240-b677-4f32-8707-d363bbdedd1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:37:27.555227 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b9b240-b677-4f32-8707-d363bbdedd1a None admin] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:37:27.555511 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b9b240-b677-4f32-8707-d363bbdedd1a None admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:37:27.555846 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b9b240-b677-4f32-8707-d363bbdedd1a None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:37:27.566807 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73b9b240-b677-4f32-8707-d363bbdedd1a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 16:37:27.572451 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application [None req-73b9b240-b677-4f32-8707-d363bbdedd1a None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 01 16:37:27.572451 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 16:37:27.572451 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 16:37:27.572451 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 16:37:27.572451 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 16:37:27.572451 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 16:37:27.572451 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 16:37:27.572451 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 16:37:27.572451 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 16:37:27.572451 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 16:37:27.572451 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 16:37:27.572451 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 16:37:27.572451 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 16:37:27.572451 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 16:37:27.572451 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 16:37:27.572451 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 16:37:27.573375 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 16:37:27.573375 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 16:37:27.573375 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 16:37:27.573375 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 16:37:27.573375 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 16:37:27.573375 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 16:37:27.573375 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 16:37:27.573375 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 16:37:27.573375 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 16:37:27.573375 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 16:37:27.573375 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 16:37:27.573375 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application with Lock( Aug 01 16:37:27.573375 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 16:37:27.573375 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self._enter() Aug 01 16:37:27.573375 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 16:37:27.573375 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 16:37:27.574263 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 16:37:27.574263 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self.creator() Aug 01 16:37:27.574263 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 16:37:27.574263 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application created_value = creator( Aug 01 16:37:27.574263 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 16:37:27.574263 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 16:37:27.574263 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 16:37:27.574263 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 16:37:27.574263 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 16:37:27.574263 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 16:37:27.574263 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 01 16:37:27.574263 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application Aug 01 16:37:27.574263 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 455/927] 10.222.0.20 () {56 vars in 1120 bytes} [Mon Aug 1 16:37:27 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 16:37:27.581468 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d47f48d7-705e-404a-90d0-947c8576afd1 None None] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:37:27.665708 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG dbcounter [-] [87058] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:37:27.683963 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d47f48d7-705e-404a-90d0-947c8576afd1 None None] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:37:27.720417 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad62f88-0f7d-4ca0-8f8a-b3a76eb3d273 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:37:27.721046 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad62f88-0f7d-4ca0-8f8a-b3a76eb3d273 None admin] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:37:27.722959 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad62f88-0f7d-4ca0-8f8a-b3a76eb3d273 None admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:37:27.722959 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad62f88-0f7d-4ca0-8f8a-b3a76eb3d273 None admin] PATH_INFO: `/v3/roles` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:37:27.722959 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ad62f88-0f7d-4ca0-8f8a-b3a76eb3d273 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=87058) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:37:27.722959 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ad62f88-0f7d-4ca0-8f8a-b3a76eb3d273 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:37:27.723980 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ad62f88-0f7d-4ca0-8f8a-b3a76eb3d273 None admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:37:27.732614 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 473/928] 10.222.0.20 () {56 vars in 1115 bytes} [Mon Aug 1 16:37:27 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:37:27.738300 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b9b240-b677-4f32-8707-d363bbdedd1a None admin] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:37:27.850732 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b9b240-b677-4f32-8707-d363bbdedd1a None admin] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:37:27.854510 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f53f472-07f1-43b6-b794-aed88e5a4016 None None] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:37:27.854795 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f53f472-07f1-43b6-b794-aed88e5a4016 None None] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:37:27.855068 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f53f472-07f1-43b6-b794-aed88e5a4016 None None] PATH_INFO: `/` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:37:27.857586 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 474/929] 10.222.0.20 () {54 vars in 926 bytes} [Mon Aug 1 16:37:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 01 16:37:27.861433 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e36d38-70bd-4d49-90fc-0f67702c264d None None] REQUEST_METHOD: `POST` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:37:27.861824 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e36d38-70bd-4d49-90fc-0f67702c264d None None] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:37:27.862128 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e36d38-70bd-4d49-90fc-0f67702c264d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:37:27.868500 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG dbcounter [-] [87057] Writing DB stats keystone:SELECT=100 {{(pid=87057) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:37:27.883724 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.auth.core [None req-76e36d38-70bd-4d49-90fc-0f67702c264d None None] MFA Rules not processed for user `1007f8e379b74e169b98fb7c3c0ba5b4`. Rule list: `[]` (Enabled: `True`). {{(pid=87058) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:37:27.884931 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.fernet_utils [None req-76e36d38-70bd-4d49-90fc-0f67702c264d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:37:27.892333 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-263d92d4-7234-41c6-ba44-4f92e1695c99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:37:27.894194 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263d92d4-7234-41c6-ba44-4f92e1695c99 None admin] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:37:27.894798 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263d92d4-7234-41c6-ba44-4f92e1695c99 None admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:37:27.895275 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263d92d4-7234-41c6-ba44-4f92e1695c99 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:37:27.914447 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-263d92d4-7234-41c6-ba44-4f92e1695c99 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:37:27.915916 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-263d92d4-7234-41c6-ba44-4f92e1695c99 None admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:37:27.931019 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application [None req-263d92d4-7234-41c6-ba44-4f92e1695c99 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 16:37:27.931019 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 16:37:27.931019 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 16:37:27.931019 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 16:37:27.931019 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 16:37:27.931019 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 16:37:27.931019 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 16:37:27.931019 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 16:37:27.931019 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 16:37:27.931019 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 16:37:27.931019 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 16:37:27.931019 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 16:37:27.931019 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 16:37:27.931019 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 16:37:27.931019 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 16:37:27.931962 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 16:37:27.931962 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 16:37:27.931962 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 16:37:27.931962 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 16:37:27.931962 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 16:37:27.931962 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 16:37:27.931962 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 16:37:27.931962 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 16:37:27.931962 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application with Lock( Aug 01 16:37:27.931962 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 16:37:27.931962 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self._enter() Aug 01 16:37:27.931962 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 16:37:27.931962 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 16:37:27.931962 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 16:37:27.931962 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self.creator() Aug 01 16:37:27.931962 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 16:37:27.931962 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application created_value = creator( Aug 01 16:37:27.932876 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 16:37:27.932876 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 16:37:27.932876 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 16:37:27.932876 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application Aug 01 16:37:27.932876 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 456/930] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Aug 1 16:37:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 16:37:27.938987 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-263d92d4-7234-41c6-ba44-4f92e1695c99 None admin] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:37:27.982597 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 475/931] 10.222.0.20 () {58 vars in 1023 bytes} [Mon Aug 1 16:37:27 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 01 16:37:28.003943 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e36d38-70bd-4d49-90fc-0f67702c264d None None] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:37:28.047301 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-263d92d4-7234-41c6-ba44-4f92e1695c99 None admin] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:37:28.084310 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae7bfb2-cbf5-4050-9553-4904e311c8b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:37:28.084935 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae7bfb2-cbf5-4050-9553-4904e311c8b4 None admin] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:37:28.085219 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae7bfb2-cbf5-4050-9553-4904e311c8b4 None admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:37:28.085496 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae7bfb2-cbf5-4050-9553-4904e311c8b4 None admin] PATH_INFO: `/v3/domains` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:37:28.086365 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ae7bfb2-cbf5-4050-9553-4904e311c8b4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87057) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:37:28.086669 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ae7bfb2-cbf5-4050-9553-4904e311c8b4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:37:28.088050 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ae7bfb2-cbf5-4050-9553-4904e311c8b4 None admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:37:28.097135 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG dbcounter [-] [87058] Writing DB stats keystone:SELECT=100 {{(pid=87058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:37:28.105593 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 457/932] 10.222.0.20 () {56 vars in 1109 bytes} [Mon Aug 1 16:37:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:37:28.106652 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae7bfb2-cbf5-4050-9553-4904e311c8b4 None admin] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:37:28.126331 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e36d38-70bd-4d49-90fc-0f67702c264d None None] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:37:28.135484 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG dbcounter [-] [87057] Writing DB stats keystone:SELECT=100 {{(pid=87057) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:37:28.167347 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a0fba14-b12e-40d5-878e-541a99ab309a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:37:28.168376 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0fba14-b12e-40d5-878e-541a99ab309a None admin] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:37:28.168841 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0fba14-b12e-40d5-878e-541a99ab309a None admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:37:28.169287 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0fba14-b12e-40d5-878e-541a99ab309a None admin] PATH_INFO: `/v3/services/compute` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:37:28.170262 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a0fba14-b12e-40d5-878e-541a99ab309a None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:37:28.172041 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a0fba14-b12e-40d5-878e-541a99ab309a None admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:37:28.182222 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application [None req-0a0fba14-b12e-40d5-878e-541a99ab309a None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Aug 01 16:37:28.182222 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 16:37:28.182222 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 01 16:37:28.182222 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 01 16:37:28.182222 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 01 16:37:28.182222 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 01 16:37:28.182222 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 01 16:37:28.182222 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 01 16:37:28.182222 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 01 16:37:28.182222 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application Aug 01 16:37:28.182222 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 01 16:37:28.182222 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application Aug 01 16:37:28.182222 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 16:37:28.182222 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 16:37:28.182222 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 16:37:28.182222 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 16:37:28.182222 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 16:37:28.183281 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 16:37:28.183281 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 16:37:28.183281 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 16:37:28.183281 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 16:37:28.183281 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 16:37:28.183281 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 16:37:28.183281 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 01 16:37:28.183281 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 01 16:37:28.183281 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 01 16:37:28.183281 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 01 16:37:28.183281 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 16:37:28.183281 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 16:37:28.183281 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 16:37:28.183281 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 16:37:28.183281 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 16:37:28.183281 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 16:37:28.184142 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 16:37:28.184142 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application with Lock( Aug 01 16:37:28.184142 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 16:37:28.184142 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self._enter() Aug 01 16:37:28.184142 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 16:37:28.184142 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 16:37:28.184142 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 16:37:28.184142 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self.creator() Aug 01 16:37:28.184142 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 16:37:28.184142 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application created_value = creator( Aug 01 16:37:28.184142 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 01 16:37:28.184142 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 01 16:37:28.184142 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 01 16:37:28.184142 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application Aug 01 16:37:28.187215 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 476/933] 10.222.0.20 () {56 vars in 1111 bytes} [Mon Aug 1 16:37:28 2022] GET /identity/v3/services/compute => generated 88 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 01 16:37:28.193667 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a0fba14-b12e-40d5-878e-541a99ab309a None admin] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:37:28.214072 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae7bfb2-cbf5-4050-9553-4904e311c8b4 None admin] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:37:28.253335 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a30c798-c4f1-4682-b9af-bfd14aa5db8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:37:28.254183 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a30c798-c4f1-4682-b9af-bfd14aa5db8f None admin] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:37:28.254573 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a30c798-c4f1-4682-b9af-bfd14aa5db8f None admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:37:28.254916 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a30c798-c4f1-4682-b9af-bfd14aa5db8f None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:37:28.263678 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a30c798-c4f1-4682-b9af-bfd14aa5db8f None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:37:28.265347 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a30c798-c4f1-4682-b9af-bfd14aa5db8f None admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:37:28.279900 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application [None req-8a30c798-c4f1-4682-b9af-bfd14aa5db8f None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 01 16:37:28.279900 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 16:37:28.279900 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 16:37:28.279900 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 16:37:28.279900 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 16:37:28.279900 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 16:37:28.279900 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 16:37:28.279900 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 16:37:28.279900 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 16:37:28.279900 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 16:37:28.279900 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 16:37:28.279900 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 16:37:28.279900 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 01 16:37:28.279900 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 01 16:37:28.279900 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 01 16:37:28.279900 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 01 16:37:28.281454 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 16:37:28.281454 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 16:37:28.281454 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 01 16:37:28.281454 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 16:37:28.281454 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 01 16:37:28.281454 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 16:37:28.281454 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 16:37:28.281454 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 16:37:28.281454 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 16:37:28.281454 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 16:37:28.281454 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 16:37:28.281454 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application with Lock( Aug 01 16:37:28.281454 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 16:37:28.281454 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self._enter() Aug 01 16:37:28.281454 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 16:37:28.281454 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 16:37:28.281454 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 16:37:28.281454 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self.creator() Aug 01 16:37:28.282459 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 16:37:28.282459 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application created_value = creator( Aug 01 16:37:28.282459 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 01 16:37:28.282459 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 01 16:37:28.282459 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 01 16:37:28.282459 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 01 16:37:28.282459 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 01 16:37:28.282459 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 01 16:37:28.282459 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 01 16:37:28.282459 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application Aug 01 16:37:28.285786 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 458/934] 10.222.0.20 () {56 vars in 1117 bytes} [Mon Aug 1 16:37:28 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 16:37:28.287978 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a30c798-c4f1-4682-b9af-bfd14aa5db8f None admin] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:37:28.325807 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a0fba14-b12e-40d5-878e-541a99ab309a None admin] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:37:28.372099 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c7723e-ef0b-4af5-9553-2f12bab6f785 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:37:28.373074 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c7723e-ef0b-4af5-9553-2f12bab6f785 None admin] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:37:28.373577 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c7723e-ef0b-4af5-9553-2f12bab6f785 None admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:37:28.374094 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c7723e-ef0b-4af5-9553-2f12bab6f785 None admin] PATH_INFO: `/v3/services` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:37:28.375030 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7c7723e-ef0b-4af5-9553-2f12bab6f785 None admin] RBAC: Adding query filter params (name=compute) {{(pid=87058) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:37:28.376082 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7c7723e-ef0b-4af5-9553-2f12bab6f785 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:37:28.378153 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7c7723e-ef0b-4af5-9553-2f12bab6f785 None admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:37:28.390119 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 477/935] 10.222.0.20 () {56 vars in 1112 bytes} [Mon Aug 1 16:37:28 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:37:28.396970 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG dbcounter [-] [87057] Writing DB stats keystone:SELECT=100 {{(pid=87057) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:37:28.398867 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c7723e-ef0b-4af5-9553-2f12bab6f785 None admin] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:37:28.402896 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG dbcounter [-] [87058] Writing DB stats keystone:SELECT=100 {{(pid=87058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:37:28.411140 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a30c798-c4f1-4682-b9af-bfd14aa5db8f None admin] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:37:28.449937 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e199b9-1f20-452c-ba30-5b34b795848b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:37:28.450587 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e199b9-1f20-452c-ba30-5b34b795848b None admin] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:37:28.450905 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e199b9-1f20-452c-ba30-5b34b795848b None admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:37:28.451237 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e199b9-1f20-452c-ba30-5b34b795848b None admin] PATH_INFO: `/v3/users` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:37:28.452042 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7e199b9-1f20-452c-ba30-5b34b795848b None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=87057) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:37:28.452394 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7e199b9-1f20-452c-ba30-5b34b795848b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:37:28.453927 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7e199b9-1f20-452c-ba30-5b34b795848b None admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:37:28.482207 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 459/936] 10.222.0.20 () {56 vars in 1149 bytes} [Mon Aug 1 16:37:28 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:37:28.483909 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e199b9-1f20-452c-ba30-5b34b795848b None admin] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:37:28.530362 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c7723e-ef0b-4af5-9553-2f12bab6f785 None admin] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:37:28.577762 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-370565ac-c2e2-475e-9733-17d6da18897b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:37:28.578431 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370565ac-c2e2-475e-9733-17d6da18897b None admin] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:37:28.578712 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370565ac-c2e2-475e-9733-17d6da18897b None admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:37:28.578987 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370565ac-c2e2-475e-9733-17d6da18897b None admin] PATH_INFO: `/v3/services` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:37:28.579676 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-370565ac-c2e2-475e-9733-17d6da18897b None admin] RBAC: Adding query filter params (type=compute) {{(pid=87058) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:37:28.579980 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-370565ac-c2e2-475e-9733-17d6da18897b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:37:28.581351 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-370565ac-c2e2-475e-9733-17d6da18897b None admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:37:28.590204 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 478/937] 10.222.0.20 () {56 vars in 1112 bytes} [Mon Aug 1 16:37:28 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:37:28.608617 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e199b9-1f20-452c-ba30-5b34b795848b None admin] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:37:28.650353 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a32046-7d12-4437-b714-49492e5d016d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:37:28.651367 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a32046-7d12-4437-b714-49492e5d016d None admin] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:37:28.651873 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a32046-7d12-4437-b714-49492e5d016d None admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:37:28.652364 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a32046-7d12-4437-b714-49492e5d016d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:37:28.664642 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38a32046-7d12-4437-b714-49492e5d016d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:37:28.666545 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38a32046-7d12-4437-b714-49492e5d016d None admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:37:28.674974 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG dbcounter [-] [87057] Writing DB stats keystone:SELECT=100 {{(pid=87057) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:37:28.686096 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application [None req-38a32046-7d12-4437-b714-49492e5d016d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 16:37:28.686096 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 16:37:28.686096 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 16:37:28.686096 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 16:37:28.686096 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 16:37:28.686096 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 16:37:28.686096 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 16:37:28.686096 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 16:37:28.686096 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 16:37:28.686096 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 16:37:28.686096 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 16:37:28.686096 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 16:37:28.686096 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 16:37:28.686096 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 16:37:28.686096 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 16:37:28.693302 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 16:37:28.693302 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 16:37:28.693302 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 16:37:28.693302 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 16:37:28.693302 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 16:37:28.693302 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 16:37:28.693302 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 16:37:28.693302 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 16:37:28.693302 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application with Lock( Aug 01 16:37:28.693302 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 16:37:28.693302 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self._enter() Aug 01 16:37:28.693302 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 16:37:28.693302 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 16:37:28.693302 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 16:37:28.693302 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self.creator() Aug 01 16:37:28.693302 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 16:37:28.693302 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application created_value = creator( Aug 01 16:37:28.694261 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 16:37:28.694261 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 16:37:28.694261 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 16:37:28.694261 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application Aug 01 16:37:28.694261 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 460/938] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Aug 1 16:37:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 16:37:28.697769 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-370565ac-c2e2-475e-9733-17d6da18897b None admin] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:37:28.717178 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d25722b-e776-4765-b270-f8212d8c27b7 None None] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:37:28.717552 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d25722b-e776-4765-b270-f8212d8c27b7 None None] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:37:28.717720 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d25722b-e776-4765-b270-f8212d8c27b7 None None] PATH_INFO: `/` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:37:28.719700 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 461/939] 10.222.0.20 () {54 vars in 926 bytes} [Mon Aug 1 16:37:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 01 16:37:28.726073 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ae5953-c47f-47d4-aac7-b7ee4a9dc98f None None] REQUEST_METHOD: `POST` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:37:28.726374 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ae5953-c47f-47d4-aac7-b7ee4a9dc98f None None] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:37:28.729688 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ae5953-c47f-47d4-aac7-b7ee4a9dc98f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:37:28.747558 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.auth.core [None req-e6ae5953-c47f-47d4-aac7-b7ee4a9dc98f None None] MFA Rules not processed for user `1007f8e379b74e169b98fb7c3c0ba5b4`. Rule list: `[]` (Enabled: `True`). {{(pid=87057) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:37:28.748916 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.fernet_utils [None req-e6ae5953-c47f-47d4-aac7-b7ee4a9dc98f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87057) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:37:28.779440 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG dbcounter [-] [87058] Writing DB stats keystone:SELECT=100 {{(pid=87058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:37:28.818284 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-370565ac-c2e2-475e-9733-17d6da18897b None admin] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:37:28.838620 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 462/940] 10.222.0.20 () {58 vars in 1023 bytes} [Mon Aug 1 16:37:28 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 01 16:37:28.859563 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a5aa8d2-3de5-4b55-ae62-2c9daf66cfb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:37:28.860433 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5aa8d2-3de5-4b55-ae62-2c9daf66cfb6 None admin] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:37:28.860433 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5aa8d2-3de5-4b55-ae62-2c9daf66cfb6 None admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:37:28.860712 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5aa8d2-3de5-4b55-ae62-2c9daf66cfb6 None admin] PATH_INFO: `/v3/domains` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:37:28.861412 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a5aa8d2-3de5-4b55-ae62-2c9daf66cfb6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87058) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:37:28.861718 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a5aa8d2-3de5-4b55-ae62-2c9daf66cfb6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:37:28.862361 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ae5953-c47f-47d4-aac7-b7ee4a9dc98f None None] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:37:28.863127 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a5aa8d2-3de5-4b55-ae62-2c9daf66cfb6 None admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:37:28.877646 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 479/941] 10.222.0.20 () {56 vars in 1109 bytes} [Mon Aug 1 16:37:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:37:28.880130 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd507b9-8cda-42c9-99b3-7b4f3544cc27 None None] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:37:28.880692 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd507b9-8cda-42c9-99b3-7b4f3544cc27 None None] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:37:28.881070 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd507b9-8cda-42c9-99b3-7b4f3544cc27 None None] PATH_INFO: `/` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:37:28.882912 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 480/942] 10.222.0.20 () {54 vars in 926 bytes} [Mon Aug 1 16:37:28 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 16:37:28.884452 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd507b9-8cda-42c9-99b3-7b4f3544cc27 None None] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:37:28.971615 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ae5953-c47f-47d4-aac7-b7ee4a9dc98f None None] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:37:29.000491 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG dbcounter [-] [87057] Writing DB stats keystone:SELECT=100 {{(pid=87057) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:37:29.002629 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd507b9-8cda-42c9-99b3-7b4f3544cc27 None None] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:37:29.008339 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f49773b7-5275-4868-bf73-aaed5b03f77f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:37:29.008993 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49773b7-5275-4868-bf73-aaed5b03f77f None admin] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:37:29.009273 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49773b7-5275-4868-bf73-aaed5b03f77f None admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:37:29.009554 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49773b7-5275-4868-bf73-aaed5b03f77f None admin] PATH_INFO: `/v3/services/object-store` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:37:29.010300 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f49773b7-5275-4868-bf73-aaed5b03f77f None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:37:29.015329 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f49773b7-5275-4868-bf73-aaed5b03f77f None admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:37:29.020979 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application [None req-f49773b7-5275-4868-bf73-aaed5b03f77f None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 01 16:37:29.020979 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 16:37:29.020979 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 01 16:37:29.020979 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 01 16:37:29.020979 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 01 16:37:29.020979 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 01 16:37:29.020979 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 01 16:37:29.020979 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 01 16:37:29.020979 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 01 16:37:29.020979 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application Aug 01 16:37:29.020979 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 01 16:37:29.020979 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application Aug 01 16:37:29.020979 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 16:37:29.020979 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 16:37:29.020979 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 16:37:29.020979 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 16:37:29.020979 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 16:37:29.021954 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 16:37:29.021954 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 16:37:29.021954 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 16:37:29.021954 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 16:37:29.021954 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 16:37:29.021954 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 16:37:29.021954 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 01 16:37:29.021954 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 01 16:37:29.021954 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 01 16:37:29.021954 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 01 16:37:29.021954 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 16:37:29.021954 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 16:37:29.021954 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 16:37:29.021954 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 16:37:29.021954 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 16:37:29.021954 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 16:37:29.023558 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 16:37:29.023558 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application with Lock( Aug 01 16:37:29.023558 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 16:37:29.023558 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self._enter() Aug 01 16:37:29.023558 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 16:37:29.023558 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 16:37:29.023558 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 16:37:29.023558 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self.creator() Aug 01 16:37:29.023558 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 16:37:29.023558 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application created_value = creator( Aug 01 16:37:29.023558 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 01 16:37:29.023558 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 01 16:37:29.023558 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 01 16:37:29.023558 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application Aug 01 16:37:29.023558 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 463/943] 10.222.0.20 () {56 vars in 1126 bytes} [Mon Aug 1 16:37:28 2022] GET /identity/v3/services/object-store => generated 93 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 01 16:37:29.024736 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2273be-e14f-4592-9011-74dbc62939b5 None None] REQUEST_METHOD: `POST` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:37:29.024736 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2273be-e14f-4592-9011-74dbc62939b5 None None] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:37:29.024736 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2273be-e14f-4592-9011-74dbc62939b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:37:29.038290 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG dbcounter [-] [87058] Writing DB stats keystone:SELECT=100 {{(pid=87058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:37:29.047499 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.auth.core [None req-ab2273be-e14f-4592-9011-74dbc62939b5 None None] MFA Rules not processed for user `1007f8e379b74e169b98fb7c3c0ba5b4`. Rule list: `[]` (Enabled: `True`). {{(pid=87057) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:37:29.048735 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.fernet_utils [None req-ab2273be-e14f-4592-9011-74dbc62939b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87057) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:37:29.049407 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e09f5bef-7a7d-442a-a2ea-281358efe7e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:37:29.050175 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09f5bef-7a7d-442a-a2ea-281358efe7e3 None admin] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:37:29.050618 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09f5bef-7a7d-442a-a2ea-281358efe7e3 None admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:37:29.051073 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09f5bef-7a7d-442a-a2ea-281358efe7e3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:37:29.064044 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e09f5bef-7a7d-442a-a2ea-281358efe7e3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:37:29.067150 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e09f5bef-7a7d-442a-a2ea-281358efe7e3 None admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:37:29.083835 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application [None req-e09f5bef-7a7d-442a-a2ea-281358efe7e3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 16:37:29.083835 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 16:37:29.083835 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 16:37:29.083835 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 16:37:29.083835 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 16:37:29.083835 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 16:37:29.083835 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 16:37:29.083835 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 16:37:29.083835 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 16:37:29.083835 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 16:37:29.083835 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 16:37:29.083835 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 16:37:29.083835 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 01 16:37:29.083835 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 01 16:37:29.083835 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 01 16:37:29.085079 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 01 16:37:29.085079 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 16:37:29.085079 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 16:37:29.085079 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 16:37:29.085079 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 16:37:29.085079 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 16:37:29.085079 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 16:37:29.085079 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 16:37:29.085079 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application with Lock( Aug 01 16:37:29.085079 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 16:37:29.085079 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self._enter() Aug 01 16:37:29.085079 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 16:37:29.085079 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 16:37:29.085079 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 16:37:29.085079 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self.creator() Aug 01 16:37:29.085079 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 16:37:29.085079 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application created_value = creator( Aug 01 16:37:29.086741 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 01 16:37:29.086741 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 01 16:37:29.086741 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 01 16:37:29.086741 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 01 16:37:29.086741 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 01 16:37:29.086741 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 01 16:37:29.086741 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 16:37:29.086741 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application Aug 01 16:37:29.086741 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 481/944] 10.222.0.20 () {56 vars in 1111 bytes} [Mon Aug 1 16:37:28 2022] GET /identity/v3/projects/service => generated 88 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 01 16:37:29.086741 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e09f5bef-7a7d-442a-a2ea-281358efe7e3 None admin] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:37:29.130506 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 464/945] 10.222.0.20 () {58 vars in 1023 bytes} [Mon Aug 1 16:37:29 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 01 16:37:29.130506 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab2273be-e14f-4592-9011-74dbc62939b5 None None] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:37:29.194636 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e09f5bef-7a7d-442a-a2ea-281358efe7e3 None admin] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:37:29.230334 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97953ba1-cf3d-4522-98fd-41e1ded98602 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:37:29.230952 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97953ba1-cf3d-4522-98fd-41e1ded98602 None admin] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:37:29.231233 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97953ba1-cf3d-4522-98fd-41e1ded98602 None admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:37:29.231514 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97953ba1-cf3d-4522-98fd-41e1ded98602 None admin] PATH_INFO: `/v3/services` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:37:29.232190 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97953ba1-cf3d-4522-98fd-41e1ded98602 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=87058) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:37:29.232491 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97953ba1-cf3d-4522-98fd-41e1ded98602 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:37:29.233846 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97953ba1-cf3d-4522-98fd-41e1ded98602 None admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:37:29.244431 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 482/946] 10.222.0.20 () {56 vars in 1122 bytes} [Mon Aug 1 16:37:29 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:37:29.244431 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97953ba1-cf3d-4522-98fd-41e1ded98602 None admin] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:37:29.244662 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab2273be-e14f-4592-9011-74dbc62939b5 None None] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:37:29.267686 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG dbcounter [-] [87057] Writing DB stats keystone:SELECT=100 {{(pid=87057) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:37:29.279631 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2465d4ea-1930-4a3f-815a-a9f6f034e38b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:37:29.280250 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2465d4ea-1930-4a3f-815a-a9f6f034e38b None admin] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:37:29.280798 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2465d4ea-1930-4a3f-815a-a9f6f034e38b None admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:37:29.280798 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2465d4ea-1930-4a3f-815a-a9f6f034e38b None admin] PATH_INFO: `/v3/projects` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:37:29.281513 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2465d4ea-1930-4a3f-815a-a9f6f034e38b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87057) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:37:29.281808 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2465d4ea-1930-4a3f-815a-a9f6f034e38b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:37:29.283280 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2465d4ea-1930-4a3f-815a-a9f6f034e38b None admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:37:29.299595 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 465/947] 10.222.0.20 () {56 vars in 1148 bytes} [Mon Aug 1 16:37:29 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:37:29.299595 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2465d4ea-1930-4a3f-815a-a9f6f034e38b None admin] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:37:29.300754 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG dbcounter [-] [87058] Writing DB stats keystone:SELECT=100 {{(pid=87058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:37:29.357654 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97953ba1-cf3d-4522-98fd-41e1ded98602 None admin] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:37:29.396672 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f16616-9681-494b-a4bc-74339b04c264 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:37:29.397296 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f16616-9681-494b-a4bc-74339b04c264 None admin] REQUEST_METHOD: `POST` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:37:29.397572 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f16616-9681-494b-a4bc-74339b04c264 None admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:37:29.397846 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f16616-9681-494b-a4bc-74339b04c264 None admin] PATH_INFO: `/v3/projects` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:37:29.398567 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50f16616-9681-494b-a4bc-74339b04c264 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:37:29.400017 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50f16616-9681-494b-a4bc-74339b04c264 None admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:37:29.402634 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: WARNING keystone.server.flask.common [None req-50f16616-9681-494b-a4bc-74339b04c264 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Aug 01 16:37:29.408897 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2465d4ea-1930-4a3f-815a-a9f6f034e38b None admin] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:37:29.445705 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a22dcd-3c18-495c-ab86-29bc63f35bf4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:37:29.446271 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a22dcd-3c18-495c-ab86-29bc63f35bf4 None admin] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:37:29.446551 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a22dcd-3c18-495c-ab86-29bc63f35bf4 None admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:37:29.446805 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.sql.core [None req-50f16616-9681-494b-a4bc-74339b04c264 None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Aug 01 16:37:29.446805 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Aug 01 16:37:29.446805 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [parameters: {'id': '2f5c891c9097416991571d76961195aa', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Aug 01 16:37:29.446805 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=87058) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Aug 01 16:37:29.447105 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a22dcd-3c18-495c-ab86-29bc63f35bf4 None admin] PATH_INFO: `/v3/services` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:37:29.447475 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43a22dcd-3c18-495c-ab86-29bc63f35bf4 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=87057) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:37:29.447774 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43a22dcd-3c18-495c-ab86-29bc63f35bf4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:37:29.449119 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43a22dcd-3c18-495c-ab86-29bc63f35bf4 None admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:37:29.459296 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 466/948] 10.222.0.20 () {56 vars in 1122 bytes} [Mon Aug 1 16:37:29 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:37:29.459482 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application [None req-50f16616-9681-494b-a4bc-74339b04c264 None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 2f5c891c9097416991571d76961195aa.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 2f5c891c9097416991571d76961195aa. Aug 01 16:37:29.459482 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 16:37:29.459482 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Aug 01 16:37:29.459482 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 01 16:37:29.459482 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 01 16:37:29.459482 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 01 16:37:29.459482 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 01 16:37:29.459482 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application result = self._query(query) Aug 01 16:37:29.459482 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 01 16:37:29.459482 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application conn.query(q) Aug 01 16:37:29.459482 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 01 16:37:29.459482 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 01 16:37:29.459482 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 01 16:37:29.460270 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application result.read() Aug 01 16:37:29.460270 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 01 16:37:29.460270 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 01 16:37:29.460270 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 01 16:37:29.460270 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 01 16:37:29.460270 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 01 16:37:29.460270 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 01 16:37:29.460270 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 01 16:37:29.460270 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 01 16:37:29.460270 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Aug 01 16:37:29.460270 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application Aug 01 16:37:29.460270 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Aug 01 16:37:29.460270 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application Aug 01 16:37:29.460270 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 16:37:29.460270 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Aug 01 16:37:29.460270 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return method(*args, **kwargs) Aug 01 16:37:29.460270 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Aug 01 16:37:29.460270 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return project_ref.to_dict() Aug 01 16:37:29.461240 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 01 16:37:29.461240 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application next(self.gen) Aug 01 16:37:29.461240 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Aug 01 16:37:29.461240 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application yield resource Aug 01 16:37:29.461240 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 01 16:37:29.461240 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application next(self.gen) Aug 01 16:37:29.461240 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Aug 01 16:37:29.461240 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application self.session.rollback() Aug 01 16:37:29.461240 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Aug 01 16:37:29.461240 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application self.force_reraise() Aug 01 16:37:29.461240 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Aug 01 16:37:29.461240 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application raise self.value Aug 01 16:37:29.461240 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Aug 01 16:37:29.461240 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Aug 01 16:37:29.461240 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Aug 01 16:37:29.461240 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application session.commit() Aug 01 16:37:29.461240 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Aug 01 16:37:29.462182 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Aug 01 16:37:29.462182 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Aug 01 16:37:29.462182 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application self._prepare_impl() Aug 01 16:37:29.462182 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Aug 01 16:37:29.462182 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application self.session.flush() Aug 01 16:37:29.462182 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Aug 01 16:37:29.462182 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application self._flush(objects) Aug 01 16:37:29.462182 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Aug 01 16:37:29.462182 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Aug 01 16:37:29.462182 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Aug 01 16:37:29.462182 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application compat.raise_( Aug 01 16:37:29.462182 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 01 16:37:29.462182 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application raise exception Aug 01 16:37:29.462182 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Aug 01 16:37:29.462182 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application flush_context.execute() Aug 01 16:37:29.462182 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Aug 01 16:37:29.462182 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application rec.execute(self) Aug 01 16:37:29.463096 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Aug 01 16:37:29.463096 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Aug 01 16:37:29.463096 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Aug 01 16:37:29.463096 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application _emit_insert_statements( Aug 01 16:37:29.463096 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Aug 01 16:37:29.463096 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application c = connection._execute_20( Aug 01 16:37:29.463096 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Aug 01 16:37:29.463096 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Aug 01 16:37:29.463096 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Aug 01 16:37:29.463096 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Aug 01 16:37:29.463096 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Aug 01 16:37:29.463096 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application ret = self._execute_context( Aug 01 16:37:29.463096 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Aug 01 16:37:29.463096 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Aug 01 16:37:29.463096 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Aug 01 16:37:29.463889 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Aug 01 16:37:29.463889 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 01 16:37:29.463889 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application raise exception Aug 01 16:37:29.463889 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Aug 01 16:37:29.463889 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 01 16:37:29.463889 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 01 16:37:29.463889 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 01 16:37:29.463889 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 01 16:37:29.463889 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application result = self._query(query) Aug 01 16:37:29.463889 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 01 16:37:29.463889 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application conn.query(q) Aug 01 16:37:29.463889 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 01 16:37:29.463889 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 01 16:37:29.463889 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 01 16:37:29.463889 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application result.read() Aug 01 16:37:29.463889 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 01 16:37:29.463889 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 01 16:37:29.464754 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 01 16:37:29.464754 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 01 16:37:29.464754 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 01 16:37:29.464754 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 01 16:37:29.464754 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 01 16:37:29.464754 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 01 16:37:29.464754 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Aug 01 16:37:29.464754 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Aug 01 16:37:29.464754 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application [parameters: {'id': '2f5c891c9097416991571d76961195aa', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Aug 01 16:37:29.464754 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Aug 01 16:37:29.464754 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application Aug 01 16:37:29.464754 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 01 16:37:29.464754 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application Aug 01 16:37:29.464754 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 16:37:29.464754 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Aug 01 16:37:29.465576 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Aug 01 16:37:29.465576 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Aug 01 16:37:29.465576 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Aug 01 16:37:29.465576 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. Aug 01 16:37:29.465576 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application Aug 01 16:37:29.465576 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 01 16:37:29.465576 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application Aug 01 16:37:29.465576 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 16:37:29.465576 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 16:37:29.465576 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 16:37:29.465576 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 16:37:29.465576 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 16:37:29.465576 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 16:37:29.465576 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 16:37:29.465576 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 16:37:29.465576 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 16:37:29.465576 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 16:37:29.466904 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 16:37:29.466904 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post Aug 01 16:37:29.466904 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( Aug 01 16:37:29.466904 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 16:37:29.466904 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 16:37:29.466904 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Aug 01 16:37:29.466904 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application raise exception.Conflict( Aug 01 16:37:29.466904 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 2f5c891c9097416991571d76961195aa. Aug 01 16:37:29.466904 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application Aug 01 16:37:29.468579 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a22dcd-3c18-495c-ab86-29bc63f35bf4 None admin] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:37:29.468720 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 483/949] 10.222.0.20 () {60 vars in 1140 bytes} [Mon Aug 1 16:37:29 2022] POST /identity/v3/projects => generated 262 bytes in 215 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Aug 01 16:37:29.468720 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f16616-9681-494b-a4bc-74339b04c264 None admin] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:37:29.514399 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG dbcounter [-] [87057] Writing DB stats keystone:SELECT=100 {{(pid=87057) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:37:29.571102 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a22dcd-3c18-495c-ab86-29bc63f35bf4 None admin] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:37:29.571666 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f16616-9681-494b-a4bc-74339b04c264 None admin] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:37:29.578731 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG dbcounter [-] [87058] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:37:29.606214 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bdb4c2c-1596-47ad-99c1-816c2b3d5e3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:37:29.606880 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bdb4c2c-1596-47ad-99c1-816c2b3d5e3a None admin] REQUEST_METHOD: `PUT` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:37:29.607158 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bdb4c2c-1596-47ad-99c1-816c2b3d5e3a None admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:37:29.607470 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bdb4c2c-1596-47ad-99c1-816c2b3d5e3a None admin] PATH_INFO: `/v3/projects/7b992d6a65df4bf9b27c13192d57c5f0/users/b6a93d1c6c82489099adc8b413280fc0/roles/653d2ccc68e8415191e22643427e28bc` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:37:29.608605 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e317abc-4d8b-44b9-9af3-23505b76589b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:37:29.609219 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e317abc-4d8b-44b9-9af3-23505b76589b None admin] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:37:29.609498 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e317abc-4d8b-44b9-9af3-23505b76589b None admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:37:29.609608 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bdb4c2c-1596-47ad-99c1-816c2b3d5e3a None admin] RBAC: Authorizing `identity:create_grant(project_id=7b992d6a65df4bf9b27c13192d57c5f0, user_id=b6a93d1c6c82489099adc8b413280fc0, role_id=653d2ccc68e8415191e22643427e28bc)` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:37:29.609774 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e317abc-4d8b-44b9-9af3-23505b76589b None admin] PATH_INFO: `/v3/endpoints` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:37:29.610557 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e317abc-4d8b-44b9-9af3-23505b76589b None admin] RBAC: Adding query filter params (interface=public, service_id=4a938303141c4bdc8308a8422744fc85, region_id=RegionOne) {{(pid=87058) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:37:29.615043 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e317abc-4d8b-44b9-9af3-23505b76589b None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:37:29.615179 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bdb4c2c-1596-47ad-99c1-816c2b3d5e3a None admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:37:29.615285 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e317abc-4d8b-44b9-9af3-23505b76589b None admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:37:29.621265 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 484/950] 10.222.0.20 () {56 vars in 1251 bytes} [Mon Aug 1 16:37:29 2022] GET /identity/v3/endpoints?service_id=4a938303141c4bdc8308a8422744fc85&interface=public®ion_id=RegionOne => generated 194 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:37:29.622170 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e317abc-4d8b-44b9-9af3-23505b76589b None admin] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:37:29.683757 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 467/951] 10.222.0.20 () {58 vars in 1439 bytes} [Mon Aug 1 16:37:29 2022] PUT /identity/v3/projects/7b992d6a65df4bf9b27c13192d57c5f0/users/b6a93d1c6c82489099adc8b413280fc0/roles/653d2ccc68e8415191e22643427e28bc => generated 0 bytes in 229 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 16:37:29.684998 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bdb4c2c-1596-47ad-99c1-816c2b3d5e3a None admin] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:37:29.750505 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e317abc-4d8b-44b9-9af3-23505b76589b None admin] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:37:29.793604 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42c019fb-6f41-4e19-8e1b-e7834905124b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:37:29.794706 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c019fb-6f41-4e19-8e1b-e7834905124b None admin] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:37:29.795274 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c019fb-6f41-4e19-8e1b-e7834905124b None admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:37:29.795772 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c019fb-6f41-4e19-8e1b-e7834905124b None admin] PATH_INFO: `/v3/projects/service` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:37:29.805302 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42c019fb-6f41-4e19-8e1b-e7834905124b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:37:29.807248 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42c019fb-6f41-4e19-8e1b-e7834905124b None admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:37:29.820587 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application [None req-42c019fb-6f41-4e19-8e1b-e7834905124b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 16:37:29.820587 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 16:37:29.820587 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 16:37:29.820587 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 16:37:29.820587 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 16:37:29.820587 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 16:37:29.820587 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 16:37:29.820587 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 16:37:29.820587 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 16:37:29.820587 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 16:37:29.820587 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 16:37:29.820587 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 16:37:29.820587 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 01 16:37:29.820587 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 01 16:37:29.820587 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 01 16:37:29.821689 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 01 16:37:29.821689 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 16:37:29.821689 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 16:37:29.821689 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 16:37:29.821689 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 16:37:29.821689 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 16:37:29.821689 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 16:37:29.821689 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 16:37:29.821689 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application with Lock( Aug 01 16:37:29.821689 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 16:37:29.821689 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self._enter() Aug 01 16:37:29.821689 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 16:37:29.821689 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 16:37:29.821689 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 16:37:29.821689 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self.creator() Aug 01 16:37:29.821689 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 16:37:29.821689 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application created_value = creator( Aug 01 16:37:29.822713 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 01 16:37:29.822713 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 01 16:37:29.822713 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 01 16:37:29.822713 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 01 16:37:29.822713 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 01 16:37:29.822713 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 01 16:37:29.822713 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 16:37:29.822713 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application Aug 01 16:37:29.825355 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 485/952] 10.222.0.20 () {56 vars in 1111 bytes} [Mon Aug 1 16:37:29 2022] GET /identity/v3/projects/service => generated 88 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 16:37:29.830992 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42c019fb-6f41-4e19-8e1b-e7834905124b None admin] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:37:29.845020 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bdb4c2c-1596-47ad-99c1-816c2b3d5e3a None admin] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:37:29.855452 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG dbcounter [-] [87057] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87057) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:37:29.873113 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG dbcounter [-] [87058] Writing DB stats keystone:SELECT=100 {{(pid=87058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:37:29.893543 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ae8c602-c2e8-4b56-985a-391e6911d5a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:37:29.895476 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae8c602-c2e8-4b56-985a-391e6911d5a6 None admin] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:37:29.896540 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae8c602-c2e8-4b56-985a-391e6911d5a6 None admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:37:29.896815 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae8c602-c2e8-4b56-985a-391e6911d5a6 None admin] PATH_INFO: `/v3/services` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:37:29.897535 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ae8c602-c2e8-4b56-985a-391e6911d5a6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:37:29.898909 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ae8c602-c2e8-4b56-985a-391e6911d5a6 None admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:37:29.905302 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 468/953] 10.222.0.20 () {56 vars in 1087 bytes} [Mon Aug 1 16:37:29 2022] GET /identity/v3/services => generated 1497 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 16:37:29.969253 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42c019fb-6f41-4e19-8e1b-e7834905124b None admin] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:37:30.009823 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25770484-3bb1-4418-a750-aef50e68b233 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:37:30.010741 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25770484-3bb1-4418-a750-aef50e68b233 None admin] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:37:30.011174 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25770484-3bb1-4418-a750-aef50e68b233 None admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:37:30.011633 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25770484-3bb1-4418-a750-aef50e68b233 None admin] PATH_INFO: `/v3/projects` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:37:30.012535 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25770484-3bb1-4418-a750-aef50e68b233 None admin] RBAC: Adding query filter params (name=service) {{(pid=87058) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:37:30.013086 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25770484-3bb1-4418-a750-aef50e68b233 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:37:30.014737 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25770484-3bb1-4418-a750-aef50e68b233 None admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:37:30.036019 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 486/954] 10.222.0.20 () {56 vars in 1112 bytes} [Mon Aug 1 16:37:29 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 16:37:30.716698 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a401932-352a-4c7d-9f39-05c018910229 None None] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:37:30.717259 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a401932-352a-4c7d-9f39-05c018910229 None None] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:37:30.717539 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a401932-352a-4c7d-9f39-05c018910229 None None] PATH_INFO: `/` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:37:30.719292 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 469/955] 10.222.0.20 () {54 vars in 926 bytes} [Mon Aug 1 16:37:30 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 16:37:30.724539 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12957818-21bc-4396-aef2-bdd1aaa2d31d None None] REQUEST_METHOD: `POST` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:37:30.724955 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12957818-21bc-4396-aef2-bdd1aaa2d31d None None] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:37:30.725286 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12957818-21bc-4396-aef2-bdd1aaa2d31d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:37:30.744910 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.auth.core [None req-12957818-21bc-4396-aef2-bdd1aaa2d31d None None] MFA Rules not processed for user `1007f8e379b74e169b98fb7c3c0ba5b4`. Rule list: `[]` (Enabled: `True`). {{(pid=87058) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:37:30.746471 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.fernet_utils [None req-12957818-21bc-4396-aef2-bdd1aaa2d31d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:37:30.821110 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 487/956] 10.222.0.20 () {58 vars in 1023 bytes} [Mon Aug 1 16:37:30 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 01 16:37:30.843410 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a401932-352a-4c7d-9f39-05c018910229 None None] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:37:30.954963 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a401932-352a-4c7d-9f39-05c018910229 None None] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:37:30.992519 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff7658e-a0fe-4b2a-a555-c568bd5ba11f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:37:30.993067 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff7658e-a0fe-4b2a-a555-c568bd5ba11f None admin] REQUEST_METHOD: `POST` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:37:30.993351 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff7658e-a0fe-4b2a-a555-c568bd5ba11f None admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:37:30.993623 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff7658e-a0fe-4b2a-a555-c568bd5ba11f None admin] PATH_INFO: `/v3/services` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:37:30.994351 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cff7658e-a0fe-4b2a-a555-c568bd5ba11f None admin] RBAC: Authorizing `identity:create_service()` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:37:30.995719 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cff7658e-a0fe-4b2a-a555-c568bd5ba11f None admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:37:31.083805 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 470/957] 10.222.0.20 () {60 vars in 1141 bytes} [Mon Aug 1 16:37:30 2022] POST /identity/v3/services => generated 242 bytes in 240 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 16:37:31.781749 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80093548-955b-48e8-9955-b231d34fa85e None None] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:37:31.782306 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80093548-955b-48e8-9955-b231d34fa85e None None] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:37:31.782306 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80093548-955b-48e8-9955-b231d34fa85e None None] PATH_INFO: `/` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:37:31.783861 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 488/958] 10.222.0.20 () {54 vars in 926 bytes} [Mon Aug 1 16:37:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 16:37:31.790144 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c732be7-ffb0-4fae-9a2a-76649b775370 None None] REQUEST_METHOD: `POST` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:37:31.790608 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c732be7-ffb0-4fae-9a2a-76649b775370 None None] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:37:31.790685 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c732be7-ffb0-4fae-9a2a-76649b775370 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:37:31.809686 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.auth.core [None req-1c732be7-ffb0-4fae-9a2a-76649b775370 None None] MFA Rules not processed for user `1007f8e379b74e169b98fb7c3c0ba5b4`. Rule list: `[]` (Enabled: `True`). {{(pid=87057) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:37:31.810931 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.fernet_utils [None req-1c732be7-ffb0-4fae-9a2a-76649b775370 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87057) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:37:31.837394 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG dbcounter [-] [87057] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87057) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:37:31.889677 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 471/959] 10.222.0.20 () {58 vars in 1023 bytes} [Mon Aug 1 16:37:31 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 01 16:37:31.926363 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80093548-955b-48e8-9955-b231d34fa85e None None] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:37:31.951223 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG dbcounter [-] [87058] Writing DB stats keystone:SELECT=100 {{(pid=87058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:37:31.983980 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28003bd1-5426-4e05-905c-003b2d6d0a38 None None] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:37:31.984322 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28003bd1-5426-4e05-905c-003b2d6d0a38 None None] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:37:31.984542 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28003bd1-5426-4e05-905c-003b2d6d0a38 None None] PATH_INFO: `/` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:37:31.986087 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 472/960] 10.222.0.20 () {54 vars in 926 bytes} [Mon Aug 1 16:37:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 16:37:31.992146 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ffdbe1-0607-4db2-af2d-2c312b845d65 None None] REQUEST_METHOD: `POST` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:37:31.992687 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ffdbe1-0607-4db2-af2d-2c312b845d65 None None] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:37:31.993354 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ffdbe1-0607-4db2-af2d-2c312b845d65 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:37:32.014126 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.auth.core [None req-e7ffdbe1-0607-4db2-af2d-2c312b845d65 None None] MFA Rules not processed for user `1007f8e379b74e169b98fb7c3c0ba5b4`. Rule list: `[]` (Enabled: `True`). {{(pid=87057) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:37:32.015904 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.fernet_utils [None req-e7ffdbe1-0607-4db2-af2d-2c312b845d65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87057) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:37:32.051144 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80093548-955b-48e8-9955-b231d34fa85e None None] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:37:32.087742 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bcb5f52-a6bc-47a2-917c-1cfab5145163 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:37:32.088497 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bcb5f52-a6bc-47a2-917c-1cfab5145163 None admin] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:37:32.088784 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bcb5f52-a6bc-47a2-917c-1cfab5145163 None admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:37:32.089133 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bcb5f52-a6bc-47a2-917c-1cfab5145163 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:37:32.098393 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 473/961] 10.222.0.20 () {58 vars in 1023 bytes} [Mon Aug 1 16:37:31 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 01 16:37:32.100561 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bcb5f52-a6bc-47a2-917c-1cfab5145163 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 16:37:32.105418 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application [None req-9bcb5f52-a6bc-47a2-917c-1cfab5145163 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 01 16:37:32.105418 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 16:37:32.105418 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 16:37:32.105418 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 16:37:32.105418 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 16:37:32.105418 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 16:37:32.105418 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 16:37:32.105418 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 16:37:32.105418 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 16:37:32.105418 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 16:37:32.105418 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 16:37:32.105418 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 16:37:32.105418 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 16:37:32.105418 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 16:37:32.105418 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 16:37:32.105418 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 16:37:32.106377 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 16:37:32.106377 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 16:37:32.106377 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 16:37:32.106377 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 16:37:32.106377 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 16:37:32.106377 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 16:37:32.106377 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 16:37:32.106377 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 16:37:32.106377 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 16:37:32.106377 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 16:37:32.106377 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 16:37:32.106377 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application with Lock( Aug 01 16:37:32.106377 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 16:37:32.106377 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self._enter() Aug 01 16:37:32.106377 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 16:37:32.106377 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 16:37:32.107231 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 16:37:32.107231 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self.creator() Aug 01 16:37:32.107231 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 16:37:32.107231 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application created_value = creator( Aug 01 16:37:32.107231 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 16:37:32.107231 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 16:37:32.107231 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 16:37:32.107231 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 16:37:32.107231 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 16:37:32.107231 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 16:37:32.107231 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 01 16:37:32.107231 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application Aug 01 16:37:32.110953 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 489/962] 10.222.0.20 () {56 vars in 1120 bytes} [Mon Aug 1 16:37:31 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 16:37:32.117133 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ffdbe1-0607-4db2-af2d-2c312b845d65 None None] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:37:32.118663 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bcb5f52-a6bc-47a2-917c-1cfab5145163 None admin] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:37:32.213656 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG dbcounter [-] [87058] Writing DB stats keystone:SELECT=100 {{(pid=87058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:37:32.218173 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG dbcounter [-] [87057] Writing DB stats keystone:SELECT=100 {{(pid=87057) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:37:32.234625 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bcb5f52-a6bc-47a2-917c-1cfab5145163 None admin] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:37:32.237926 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ffdbe1-0607-4db2-af2d-2c312b845d65 None None] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:37:32.271426 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f12042-a823-4d43-8e9f-fcc81bc19922 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:37:32.272387 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f12042-a823-4d43-8e9f-fcc81bc19922 None admin] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:37:32.272668 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f12042-a823-4d43-8e9f-fcc81bc19922 None admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:37:32.272943 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f12042-a823-4d43-8e9f-fcc81bc19922 None admin] PATH_INFO: `/v3/roles` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:37:32.273657 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6f12042-a823-4d43-8e9f-fcc81bc19922 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=87058) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:37:32.273957 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6f12042-a823-4d43-8e9f-fcc81bc19922 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:37:32.275359 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6f12042-a823-4d43-8e9f-fcc81bc19922 None admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:37:32.277156 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e10dd88-8e8e-4cd4-b252-cbf92f0cd089 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:37:32.277805 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e10dd88-8e8e-4cd4-b252-cbf92f0cd089 None admin] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:37:32.278105 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e10dd88-8e8e-4cd4-b252-cbf92f0cd089 None admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:37:32.278405 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e10dd88-8e8e-4cd4-b252-cbf92f0cd089 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:37:32.279030 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e10dd88-8e8e-4cd4-b252-cbf92f0cd089 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:37:32.280385 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e10dd88-8e8e-4cd4-b252-cbf92f0cd089 None admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:37:32.284085 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 490/963] 10.222.0.20 () {56 vars in 1115 bytes} [Mon Aug 1 16:37:32 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:37:32.285079 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34b1914-2472-4e10-8703-e50f52f1e76c None None] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:37:32.285432 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34b1914-2472-4e10-8703-e50f52f1e76c None None] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:37:32.285706 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34b1914-2472-4e10-8703-e50f52f1e76c None None] PATH_INFO: `/` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:37:32.287570 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 491/964] 10.222.0.20 () {54 vars in 926 bytes} [Mon Aug 1 16:37:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 16:37:32.290881 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c34b1914-2472-4e10-8703-e50f52f1e76c None None] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:37:32.293752 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application [None req-5e10dd88-8e8e-4cd4-b252-cbf92f0cd089 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 01 16:37:32.293752 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 16:37:32.293752 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 01 16:37:32.293752 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 01 16:37:32.293752 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 01 16:37:32.293752 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 01 16:37:32.293752 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 01 16:37:32.293752 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 01 16:37:32.293752 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 01 16:37:32.293752 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application Aug 01 16:37:32.293752 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 01 16:37:32.293752 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application Aug 01 16:37:32.293752 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 16:37:32.293752 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 16:37:32.293752 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 16:37:32.293752 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 16:37:32.293752 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 16:37:32.295095 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 16:37:32.295095 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 16:37:32.295095 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 16:37:32.295095 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 16:37:32.295095 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 16:37:32.295095 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 16:37:32.295095 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 01 16:37:32.295095 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 01 16:37:32.295095 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 01 16:37:32.295095 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 01 16:37:32.295095 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 16:37:32.295095 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 16:37:32.295095 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 16:37:32.295095 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 16:37:32.295095 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 16:37:32.295095 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 16:37:32.296259 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 16:37:32.296259 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application with Lock( Aug 01 16:37:32.296259 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 16:37:32.296259 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self._enter() Aug 01 16:37:32.296259 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 16:37:32.296259 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 16:37:32.296259 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 16:37:32.296259 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self.creator() Aug 01 16:37:32.296259 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 16:37:32.296259 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application created_value = creator( Aug 01 16:37:32.296259 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 01 16:37:32.296259 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 01 16:37:32.296259 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 01 16:37:32.296259 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application Aug 01 16:37:32.296259 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 474/965] 10.222.0.20 () {56 vars in 1126 bytes} [Mon Aug 1 16:37:32 2022] GET /identity/v3/services/object-store => generated 93 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 16:37:32.297285 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4784be-89b3-4eec-9aa8-7b9969e67fe0 None None] REQUEST_METHOD: `POST` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:37:32.297285 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4784be-89b3-4eec-9aa8-7b9969e67fe0 None None] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:37:32.297285 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4784be-89b3-4eec-9aa8-7b9969e67fe0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:37:32.315583 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.auth.core [None req-6b4784be-89b3-4eec-9aa8-7b9969e67fe0 None None] MFA Rules not processed for user `1007f8e379b74e169b98fb7c3c0ba5b4`. Rule list: `[]` (Enabled: `True`). {{(pid=87057) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:37:32.316698 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.fernet_utils [None req-6b4784be-89b3-4eec-9aa8-7b9969e67fe0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87057) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:37:32.388563 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 475/966] 10.222.0.20 () {58 vars in 1023 bytes} [Mon Aug 1 16:37:32 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 01 16:37:32.398907 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b4784be-89b3-4eec-9aa8-7b9969e67fe0 None None] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:37:32.408630 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c34b1914-2472-4e10-8703-e50f52f1e76c None None] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:37:32.446043 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-589ca7d2-872f-4840-a103-85cee4cbf3c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:37:32.446707 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589ca7d2-872f-4840-a103-85cee4cbf3c1 None admin] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:37:32.446988 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589ca7d2-872f-4840-a103-85cee4cbf3c1 None admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:37:32.447257 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589ca7d2-872f-4840-a103-85cee4cbf3c1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:37:32.458142 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-589ca7d2-872f-4840-a103-85cee4cbf3c1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:37:32.459957 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-589ca7d2-872f-4840-a103-85cee4cbf3c1 None admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:37:32.462055 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG dbcounter [-] [87058] Writing DB stats keystone:SELECT=100 {{(pid=87058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:37:32.475068 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application [None req-589ca7d2-872f-4840-a103-85cee4cbf3c1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 16:37:32.475068 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 16:37:32.475068 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 16:37:32.475068 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 16:37:32.475068 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 16:37:32.475068 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 16:37:32.475068 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 16:37:32.475068 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 16:37:32.475068 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 16:37:32.475068 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 16:37:32.475068 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 16:37:32.475068 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 16:37:32.475068 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 16:37:32.475068 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 16:37:32.475068 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 16:37:32.479423 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 16:37:32.479423 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 16:37:32.479423 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 16:37:32.479423 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 16:37:32.479423 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 16:37:32.479423 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 16:37:32.479423 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 16:37:32.479423 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 16:37:32.479423 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application with Lock( Aug 01 16:37:32.479423 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 16:37:32.479423 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self._enter() Aug 01 16:37:32.479423 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 16:37:32.479423 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 16:37:32.479423 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 16:37:32.479423 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self.creator() Aug 01 16:37:32.479423 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 16:37:32.479423 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application created_value = creator( Aug 01 16:37:32.480537 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 16:37:32.480537 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 16:37:32.480537 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 16:37:32.480537 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application Aug 01 16:37:32.480537 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 492/967] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Aug 1 16:37:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 16:37:32.480537 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-589ca7d2-872f-4840-a103-85cee4cbf3c1 None admin] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:37:32.499014 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG dbcounter [-] [87057] Writing DB stats keystone:SELECT=100 {{(pid=87057) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:37:32.518238 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b4784be-89b3-4eec-9aa8-7b9969e67fe0 None None] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:37:32.554677 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c56d3bab-1d15-4470-8f20-29b82e93105f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:37:32.555325 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56d3bab-1d15-4470-8f20-29b82e93105f None admin] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:37:32.555604 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56d3bab-1d15-4470-8f20-29b82e93105f None admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:37:32.555882 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56d3bab-1d15-4470-8f20-29b82e93105f None admin] PATH_INFO: `/v3/services` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:37:32.556566 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c56d3bab-1d15-4470-8f20-29b82e93105f None admin] RBAC: Adding query filter params (name=object-store) {{(pid=87057) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:37:32.556872 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c56d3bab-1d15-4470-8f20-29b82e93105f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:37:32.558267 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c56d3bab-1d15-4470-8f20-29b82e93105f None admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:37:32.563176 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 476/968] 10.222.0.20 () {56 vars in 1122 bytes} [Mon Aug 1 16:37:32 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:37:32.563961 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c56d3bab-1d15-4470-8f20-29b82e93105f None admin] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:37:32.595287 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-589ca7d2-872f-4840-a103-85cee4cbf3c1 None admin] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:37:32.632983 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17fbecc3-b587-41ca-9ea9-fe850cad3ea3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:37:32.633606 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17fbecc3-b587-41ca-9ea9-fe850cad3ea3 None admin] REQUEST_METHOD: `POST` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:37:32.633888 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17fbecc3-b587-41ca-9ea9-fe850cad3ea3 None admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:37:32.634374 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17fbecc3-b587-41ca-9ea9-fe850cad3ea3 None admin] PATH_INFO: `/v3/users` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:37:32.635056 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17fbecc3-b587-41ca-9ea9-fe850cad3ea3 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:37:32.636490 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17fbecc3-b587-41ca-9ea9-fe850cad3ea3 None admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:37:32.671403 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c56d3bab-1d15-4470-8f20-29b82e93105f None admin] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:37:32.686909 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.sql.core [None req-17fbecc3-b587-41ca-9ea9-fe850cad3ea3 None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Aug 01 16:37:32.686909 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Aug 01 16:37:32.686909 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [parameters: {'user_id': 'ae6204ff131d4dc9950dac6d3dba905e', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Aug 01 16:37:32.686909 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=87058) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Aug 01 16:37:32.694552 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application [None req-17fbecc3-b587-41ca-9ea9-fe850cad3ea3 None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Aug 01 16:37:32.694552 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 16:37:32.694552 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Aug 01 16:37:32.694552 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 01 16:37:32.694552 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 01 16:37:32.694552 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 01 16:37:32.694552 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 01 16:37:32.694552 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application result = self._query(query) Aug 01 16:37:32.694552 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 01 16:37:32.694552 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application conn.query(q) Aug 01 16:37:32.694552 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 01 16:37:32.694552 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 01 16:37:32.694552 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 01 16:37:32.694552 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application result.read() Aug 01 16:37:32.694552 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 01 16:37:32.697258 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 01 16:37:32.697258 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 01 16:37:32.697258 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 01 16:37:32.697258 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 01 16:37:32.697258 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 01 16:37:32.697258 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 01 16:37:32.697258 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 01 16:37:32.697258 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Aug 01 16:37:32.697258 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application Aug 01 16:37:32.697258 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Aug 01 16:37:32.697258 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application Aug 01 16:37:32.697258 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 16:37:32.697258 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Aug 01 16:37:32.697258 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return method(*args, **kwargs) Aug 01 16:37:32.697258 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user Aug 01 16:37:32.697258 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) Aug 01 16:37:32.697258 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 01 16:37:32.697258 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application next(self.gen) Aug 01 16:37:32.698267 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Aug 01 16:37:32.698267 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application yield resource Aug 01 16:37:32.698267 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 01 16:37:32.698267 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application next(self.gen) Aug 01 16:37:32.698267 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Aug 01 16:37:32.698267 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application self.session.rollback() Aug 01 16:37:32.698267 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Aug 01 16:37:32.698267 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application self.force_reraise() Aug 01 16:37:32.698267 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Aug 01 16:37:32.698267 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application raise self.value Aug 01 16:37:32.698267 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Aug 01 16:37:32.698267 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Aug 01 16:37:32.698267 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Aug 01 16:37:32.698267 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application session.commit() Aug 01 16:37:32.698267 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Aug 01 16:37:32.698267 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Aug 01 16:37:32.698267 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Aug 01 16:37:32.699245 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application self._prepare_impl() Aug 01 16:37:32.699245 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Aug 01 16:37:32.699245 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application self.session.flush() Aug 01 16:37:32.699245 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Aug 01 16:37:32.699245 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application self._flush(objects) Aug 01 16:37:32.699245 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Aug 01 16:37:32.699245 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Aug 01 16:37:32.699245 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Aug 01 16:37:32.699245 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application compat.raise_( Aug 01 16:37:32.699245 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 01 16:37:32.699245 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application raise exception Aug 01 16:37:32.699245 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Aug 01 16:37:32.699245 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application flush_context.execute() Aug 01 16:37:32.699245 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Aug 01 16:37:32.699245 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application rec.execute(self) Aug 01 16:37:32.699245 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Aug 01 16:37:32.699245 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Aug 01 16:37:32.700175 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Aug 01 16:37:32.700175 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application _emit_insert_statements( Aug 01 16:37:32.700175 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements Aug 01 16:37:32.700175 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application result = connection._execute_20( Aug 01 16:37:32.700175 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Aug 01 16:37:32.700175 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Aug 01 16:37:32.700175 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Aug 01 16:37:32.700175 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Aug 01 16:37:32.700175 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Aug 01 16:37:32.700175 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application ret = self._execute_context( Aug 01 16:37:32.700175 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Aug 01 16:37:32.700175 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Aug 01 16:37:32.700175 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Aug 01 16:37:32.700175 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Aug 01 16:37:32.700175 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 01 16:37:32.700175 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application raise exception Aug 01 16:37:32.702415 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Aug 01 16:37:32.702415 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 01 16:37:32.702415 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 01 16:37:32.702415 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 01 16:37:32.702415 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 01 16:37:32.702415 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application result = self._query(query) Aug 01 16:37:32.702415 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 01 16:37:32.702415 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application conn.query(q) Aug 01 16:37:32.702415 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 01 16:37:32.702415 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 01 16:37:32.702415 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 01 16:37:32.702415 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application result.read() Aug 01 16:37:32.702415 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 01 16:37:32.702415 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 01 16:37:32.702415 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 01 16:37:32.702415 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 01 16:37:32.704269 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 01 16:37:32.704269 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 01 16:37:32.704269 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 01 16:37:32.704269 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 01 16:37:32.704269 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Aug 01 16:37:32.704269 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Aug 01 16:37:32.704269 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application [parameters: {'user_id': 'ae6204ff131d4dc9950dac6d3dba905e', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Aug 01 16:37:32.704269 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Aug 01 16:37:32.704269 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application Aug 01 16:37:32.704269 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 01 16:37:32.704269 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application Aug 01 16:37:32.704269 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 16:37:32.704269 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 16:37:32.704269 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 16:37:32.704269 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 16:37:32.706164 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 16:37:32.706164 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 16:37:32.706164 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 16:37:32.706164 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 16:37:32.706164 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 16:37:32.706164 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 16:37:32.706164 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 16:37:32.706164 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post Aug 01 16:37:32.706164 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( Aug 01 16:37:32.706164 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 16:37:32.706164 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 16:37:32.706164 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 01 16:37:32.706164 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 16:37:32.706164 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 01 16:37:32.706164 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 16:37:32.706164 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user Aug 01 16:37:32.706164 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) Aug 01 16:37:32.709184 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects Aug 01 16:37:32.709184 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) Aug 01 16:37:32.709184 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Aug 01 16:37:32.709184 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Aug 01 16:37:32.709184 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Aug 01 16:37:32.709184 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application Aug 01 16:37:32.709184 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 493/969] 10.222.0.20 () {60 vars in 1131 bytes} [Mon Aug 1 16:37:32 2022] POST /identity/v3/users => generated 160 bytes in 219 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Aug 01 16:37:32.709184 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17fbecc3-b587-41ca-9ea9-fe850cad3ea3 None admin] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:37:32.710775 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51929a25-f675-4e25-9fa2-c1b7f6161ce5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:37:32.711398 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51929a25-f675-4e25-9fa2-c1b7f6161ce5 None admin] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:37:32.711681 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51929a25-f675-4e25-9fa2-c1b7f6161ce5 None admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:37:32.712012 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51929a25-f675-4e25-9fa2-c1b7f6161ce5 None admin] PATH_INFO: `/v3/domains` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:37:32.712708 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51929a25-f675-4e25-9fa2-c1b7f6161ce5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87057) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:37:32.713007 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51929a25-f675-4e25-9fa2-c1b7f6161ce5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:37:32.714402 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51929a25-f675-4e25-9fa2-c1b7f6161ce5 None admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:37:32.727939 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 477/970] 10.222.0.20 () {56 vars in 1109 bytes} [Mon Aug 1 16:37:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:37:32.728917 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51929a25-f675-4e25-9fa2-c1b7f6161ce5 None admin] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:37:32.736812 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG dbcounter [-] [87057] Writing DB stats keystone:SELECT=100 {{(pid=87057) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:37:32.769069 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG dbcounter [-] [87058] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=87058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:37:32.815236 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17fbecc3-b587-41ca-9ea9-fe850cad3ea3 None admin] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:37:32.842119 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51929a25-f675-4e25-9fa2-c1b7f6161ce5 None admin] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:37:32.851407 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-944301ec-8241-4e5f-b664-bfebbc98d641 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:37:32.851964 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944301ec-8241-4e5f-b664-bfebbc98d641 None admin] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:37:32.852234 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944301ec-8241-4e5f-b664-bfebbc98d641 None admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:37:32.852500 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944301ec-8241-4e5f-b664-bfebbc98d641 None admin] PATH_INFO: `/v3/services` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:37:32.853176 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-944301ec-8241-4e5f-b664-bfebbc98d641 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=87058) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:37:32.853469 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-944301ec-8241-4e5f-b664-bfebbc98d641 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:37:32.855060 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-944301ec-8241-4e5f-b664-bfebbc98d641 None admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:37:32.864117 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 494/971] 10.222.0.20 () {56 vars in 1122 bytes} [Mon Aug 1 16:37:32 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:37:32.869893 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-944301ec-8241-4e5f-b664-bfebbc98d641 None admin] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:37:32.881139 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ebc38e7-ee6d-46b1-b09a-c9ca20a94305 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:37:32.881799 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ebc38e7-ee6d-46b1-b09a-c9ca20a94305 None admin] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:37:32.882514 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ebc38e7-ee6d-46b1-b09a-c9ca20a94305 None admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:37:32.882848 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ebc38e7-ee6d-46b1-b09a-c9ca20a94305 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:37:32.891181 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ebc38e7-ee6d-46b1-b09a-c9ca20a94305 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:37:32.892635 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ebc38e7-ee6d-46b1-b09a-c9ca20a94305 None admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:37:32.904767 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application [None req-3ebc38e7-ee6d-46b1-b09a-c9ca20a94305 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 01 16:37:32.904767 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 16:37:32.904767 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 16:37:32.904767 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 16:37:32.904767 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 16:37:32.904767 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 16:37:32.904767 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 16:37:32.904767 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 16:37:32.904767 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 16:37:32.904767 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 16:37:32.904767 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 16:37:32.904767 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 16:37:32.904767 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 01 16:37:32.904767 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 01 16:37:32.904767 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 01 16:37:32.904767 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 01 16:37:32.905810 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 16:37:32.905810 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 16:37:32.905810 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 01 16:37:32.905810 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 16:37:32.905810 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 01 16:37:32.905810 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 16:37:32.905810 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 16:37:32.905810 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 16:37:32.905810 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 16:37:32.905810 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 16:37:32.905810 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 16:37:32.905810 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application with Lock( Aug 01 16:37:32.905810 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 16:37:32.905810 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self._enter() Aug 01 16:37:32.905810 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 16:37:32.905810 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 16:37:32.905810 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 16:37:32.905810 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self.creator() Aug 01 16:37:32.907418 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 16:37:32.907418 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application created_value = creator( Aug 01 16:37:32.907418 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 01 16:37:32.907418 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 01 16:37:32.907418 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 01 16:37:32.907418 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 01 16:37:32.907418 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 01 16:37:32.907418 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 01 16:37:32.907418 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 01 16:37:32.907418 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application Aug 01 16:37:32.907418 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 478/972] 10.222.0.20 () {56 vars in 1099 bytes} [Mon Aug 1 16:37:32 2022] GET /identity/v3/users/cinder => generated 84 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 16:37:32.907418 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ebc38e7-ee6d-46b1-b09a-c9ca20a94305 None admin] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:37:32.974553 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG dbcounter [-] [87057] Writing DB stats keystone:SELECT=100 {{(pid=87057) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:37:32.994220 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-944301ec-8241-4e5f-b664-bfebbc98d641 None admin] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:37:33.015053 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG dbcounter [-] [87058] Writing DB stats keystone:SELECT=100 {{(pid=87058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:37:33.017212 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ebc38e7-ee6d-46b1-b09a-c9ca20a94305 None admin] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:37:33.034376 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-631af198-25b8-4ce5-8770-a3c09cb7eba9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:37:33.035035 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-631af198-25b8-4ce5-8770-a3c09cb7eba9 None admin] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:37:33.035314 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-631af198-25b8-4ce5-8770-a3c09cb7eba9 None admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:37:33.035586 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-631af198-25b8-4ce5-8770-a3c09cb7eba9 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:37:33.043358 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-631af198-25b8-4ce5-8770-a3c09cb7eba9 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:37:33.044798 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-631af198-25b8-4ce5-8770-a3c09cb7eba9 None admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:37:33.055627 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41aebcd6-59ea-471f-8cea-83bafa52c573 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:37:33.056252 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41aebcd6-59ea-471f-8cea-83bafa52c573 None admin] REQUEST_METHOD: `POST` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:37:33.056537 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41aebcd6-59ea-471f-8cea-83bafa52c573 None admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:37:33.056859 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41aebcd6-59ea-471f-8cea-83bafa52c573 None admin] PATH_INFO: `/v3/endpoints` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:37:33.056980 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application [None req-631af198-25b8-4ce5-8770-a3c09cb7eba9 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 01 16:37:33.056980 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 16:37:33.056980 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 16:37:33.056980 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 16:37:33.056980 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 16:37:33.056980 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 16:37:33.056980 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 16:37:33.056980 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 16:37:33.056980 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 16:37:33.056980 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 16:37:33.056980 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 16:37:33.056980 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 16:37:33.056980 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 01 16:37:33.056980 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 01 16:37:33.056980 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 01 16:37:33.056980 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 01 16:37:33.057907 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 16:37:33.057907 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 16:37:33.057907 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 01 16:37:33.057907 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 16:37:33.057907 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 01 16:37:33.057907 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 16:37:33.057907 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 16:37:33.057907 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 16:37:33.057907 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 16:37:33.057907 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 16:37:33.057907 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 16:37:33.057907 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application with Lock( Aug 01 16:37:33.057907 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 16:37:33.057907 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self._enter() Aug 01 16:37:33.057907 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 16:37:33.057907 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 16:37:33.057907 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 16:37:33.057907 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self.creator() Aug 01 16:37:33.059361 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 16:37:33.059361 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application created_value = creator( Aug 01 16:37:33.059361 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 01 16:37:33.059361 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 01 16:37:33.059361 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 01 16:37:33.059361 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 01 16:37:33.059361 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 01 16:37:33.059361 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 01 16:37:33.059361 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 01 16:37:33.059361 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application Aug 01 16:37:33.059994 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41aebcd6-59ea-471f-8cea-83bafa52c573 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:37:33.059994 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41aebcd6-59ea-471f-8cea-83bafa52c573 None admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:37:33.062816 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 495/973] 10.222.0.20 () {56 vars in 1117 bytes} [Mon Aug 1 16:37:32 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 16:37:33.063424 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-631af198-25b8-4ce5-8770-a3c09cb7eba9 None admin] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:37:33.133482 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 479/974] 10.222.0.20 () {60 vars in 1144 bytes} [Mon Aug 1 16:37:32 2022] POST /identity/v3/endpoints => generated 349 bytes in 227 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 16:37:33.134595 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41aebcd6-59ea-471f-8cea-83bafa52c573 None admin] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:37:33.180879 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-631af198-25b8-4ce5-8770-a3c09cb7eba9 None admin] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:37:33.226917 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e28161-8097-40b9-903a-ab43cd92931e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:37:33.227550 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e28161-8097-40b9-903a-ab43cd92931e None admin] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:37:33.227832 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e28161-8097-40b9-903a-ab43cd92931e None admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:37:33.228106 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e28161-8097-40b9-903a-ab43cd92931e None admin] PATH_INFO: `/v3/users` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:37:33.228879 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7e28161-8097-40b9-903a-ab43cd92931e None admin] RBAC: Adding query filter params (name=cinder) {{(pid=87058) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:37:33.229184 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7e28161-8097-40b9-903a-ab43cd92931e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:37:33.230678 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7e28161-8097-40b9-903a-ab43cd92931e None admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:37:33.267919 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41aebcd6-59ea-471f-8cea-83bafa52c573 None admin] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:37:33.268587 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 496/975] 10.222.0.20 () {56 vars in 1101 bytes} [Mon Aug 1 16:37:33 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:37:33.269990 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91669d23-96df-4e31-9763-1dc4cd4d99dd None None] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:37:33.270310 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91669d23-96df-4e31-9763-1dc4cd4d99dd None None] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:37:33.270727 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91669d23-96df-4e31-9763-1dc4cd4d99dd None None] PATH_INFO: `/` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:37:33.272342 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 497/976] 10.222.0.20 () {54 vars in 926 bytes} [Mon Aug 1 16:37:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 16:37:33.277988 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3014677-e6dc-46d7-98bd-9ec8c05f00ba None None] REQUEST_METHOD: `POST` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:37:33.278360 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3014677-e6dc-46d7-98bd-9ec8c05f00ba None None] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:37:33.278564 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3014677-e6dc-46d7-98bd-9ec8c05f00ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:37:33.299282 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.auth.core [None req-a3014677-e6dc-46d7-98bd-9ec8c05f00ba None None] MFA Rules not processed for user `1007f8e379b74e169b98fb7c3c0ba5b4`. Rule list: `[]` (Enabled: `True`). {{(pid=87058) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:37:33.300880 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.fernet_utils [None req-a3014677-e6dc-46d7-98bd-9ec8c05f00ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:37:33.301024 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG dbcounter [-] [87057] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87057) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:37:33.328321 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de4b1d75-22eb-42c6-a5bf-30af32b56f9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:37:33.328974 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4b1d75-22eb-42c6-a5bf-30af32b56f9b None admin] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:37:33.329260 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4b1d75-22eb-42c6-a5bf-30af32b56f9b None admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:37:33.329536 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4b1d75-22eb-42c6-a5bf-30af32b56f9b None admin] PATH_INFO: `/v3/users` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:37:33.330404 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de4b1d75-22eb-42c6-a5bf-30af32b56f9b None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=87057) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:37:33.330717 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de4b1d75-22eb-42c6-a5bf-30af32b56f9b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:37:33.332231 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de4b1d75-22eb-42c6-a5bf-30af32b56f9b None admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:37:33.344040 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG dbcounter [-] [87058] Writing DB stats keystone:SELECT=100 {{(pid=87058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:37:33.354923 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 480/977] 10.222.0.20 () {56 vars in 1149 bytes} [Mon Aug 1 16:37:33 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:37:33.360995 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de4b1d75-22eb-42c6-a5bf-30af32b56f9b None admin] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:37:33.406828 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 498/978] 10.222.0.20 () {58 vars in 1023 bytes} [Mon Aug 1 16:37:33 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 01 16:37:33.439476 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3014677-e6dc-46d7-98bd-9ec8c05f00ba None None] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:37:33.504187 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de4b1d75-22eb-42c6-a5bf-30af32b56f9b None admin] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:37:33.549681 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a5d816-fb9f-4510-bbf5-f41270352975 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:37:33.550357 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a5d816-fb9f-4510-bbf5-f41270352975 None admin] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:37:33.550639 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a5d816-fb9f-4510-bbf5-f41270352975 None admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:37:33.550911 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a5d816-fb9f-4510-bbf5-f41270352975 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:37:33.573908 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3014677-e6dc-46d7-98bd-9ec8c05f00ba None None] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:37:33.610305 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e926f7c-62b6-4406-bfdc-ac04f858650f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:37:33.610941 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG dbcounter [-] [87057] Writing DB stats keystone:SELECT=100 {{(pid=87057) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:37:33.611337 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e926f7c-62b6-4406-bfdc-ac04f858650f None admin] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:37:33.611621 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e926f7c-62b6-4406-bfdc-ac04f858650f None admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:37:33.612008 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e926f7c-62b6-4406-bfdc-ac04f858650f None admin] PATH_INFO: `/v3/services/compute` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:37:33.612692 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e926f7c-62b6-4406-bfdc-ac04f858650f None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:37:33.614196 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e926f7c-62b6-4406-bfdc-ac04f858650f None admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:37:33.627189 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application [None req-6e926f7c-62b6-4406-bfdc-ac04f858650f None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Aug 01 16:37:33.627189 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 16:37:33.627189 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 01 16:37:33.627189 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 01 16:37:33.627189 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 01 16:37:33.627189 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 01 16:37:33.627189 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 01 16:37:33.627189 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 01 16:37:33.627189 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 01 16:37:33.627189 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application Aug 01 16:37:33.627189 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 01 16:37:33.627189 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application Aug 01 16:37:33.627189 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 16:37:33.627189 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 16:37:33.627189 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 16:37:33.627189 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 16:37:33.627189 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 16:37:33.633919 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 16:37:33.633919 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 16:37:33.633919 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 16:37:33.633919 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 16:37:33.633919 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 16:37:33.633919 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 16:37:33.633919 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 01 16:37:33.633919 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 01 16:37:33.633919 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 01 16:37:33.633919 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 01 16:37:33.633919 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 16:37:33.633919 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 16:37:33.633919 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 16:37:33.633919 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 16:37:33.633919 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 16:37:33.633919 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 16:37:33.635089 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 16:37:33.635089 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application with Lock( Aug 01 16:37:33.635089 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 16:37:33.635089 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self._enter() Aug 01 16:37:33.635089 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 16:37:33.635089 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 16:37:33.635089 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 16:37:33.635089 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self.creator() Aug 01 16:37:33.635089 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 16:37:33.635089 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application created_value = creator( Aug 01 16:37:33.635089 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 01 16:37:33.635089 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 01 16:37:33.635089 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 01 16:37:33.635089 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application Aug 01 16:37:33.635089 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 499/979] 10.222.0.20 () {56 vars in 1111 bytes} [Mon Aug 1 16:37:33 2022] GET /identity/v3/services/compute => generated 88 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 01 16:37:33.636077 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e926f7c-62b6-4406-bfdc-ac04f858650f None admin] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:37:33.644150 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8a5d816-fb9f-4510-bbf5-f41270352975 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:37:33.646310 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8a5d816-fb9f-4510-bbf5-f41270352975 None admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:37:33.673927 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG dbcounter [-] [87058] Writing DB stats keystone:SELECT=100 {{(pid=87058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:37:33.749870 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e926f7c-62b6-4406-bfdc-ac04f858650f None admin] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:37:33.755750 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 481/980] 10.222.0.20 () {58 vars in 1282 bytes} [Mon Aug 1 16:37:33 2022] GET /identity/v3/auth/tokens => generated 1560 bytes in 395 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 01 16:37:33.761385 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a5d816-fb9f-4510-bbf5-f41270352975 None admin] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:37:33.792036 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c172362c-5255-4be9-97cf-374a19186a8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:37:33.793059 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c172362c-5255-4be9-97cf-374a19186a8d None admin] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:37:33.793480 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c172362c-5255-4be9-97cf-374a19186a8d None admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:37:33.793755 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c172362c-5255-4be9-97cf-374a19186a8d None admin] PATH_INFO: `/v3/services` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:37:33.794469 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c172362c-5255-4be9-97cf-374a19186a8d None admin] RBAC: Adding query filter params (name=compute) {{(pid=87058) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:37:33.794772 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c172362c-5255-4be9-97cf-374a19186a8d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:37:33.796126 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c172362c-5255-4be9-97cf-374a19186a8d None admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:37:33.801086 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 500/981] 10.222.0.20 () {56 vars in 1112 bytes} [Mon Aug 1 16:37:33 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 16:37:33.806620 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c172362c-5255-4be9-97cf-374a19186a8d None admin] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:37:33.844032 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG dbcounter [-] [87057] Writing DB stats keystone:SELECT=100 {{(pid=87057) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:37:33.866630 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a5d816-fb9f-4510-bbf5-f41270352975 None admin] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:37:33.908992 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG dbcounter [-] [87058] Writing DB stats keystone:SELECT=100 {{(pid=87058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:37:33.910718 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a4eb611-3f0a-440f-b6d5-1ca17b154be9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:37:33.911992 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4eb611-3f0a-440f-b6d5-1ca17b154be9 None admin] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:37:33.912347 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4eb611-3f0a-440f-b6d5-1ca17b154be9 None admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:37:33.912768 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4eb611-3f0a-440f-b6d5-1ca17b154be9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:37:33.919246 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c172362c-5255-4be9-97cf-374a19186a8d None admin] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:37:33.923532 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a4eb611-3f0a-440f-b6d5-1ca17b154be9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:37:33.925051 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a4eb611-3f0a-440f-b6d5-1ca17b154be9 None admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:37:33.940514 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application [None req-8a4eb611-3f0a-440f-b6d5-1ca17b154be9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 16:37:33.940514 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 16:37:33.940514 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 16:37:33.940514 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 16:37:33.940514 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 16:37:33.940514 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 16:37:33.940514 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 16:37:33.940514 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 16:37:33.940514 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 16:37:33.940514 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 16:37:33.940514 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 16:37:33.940514 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 16:37:33.940514 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 16:37:33.940514 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 16:37:33.940514 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 16:37:33.941656 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 16:37:33.941656 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 16:37:33.941656 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 16:37:33.941656 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 16:37:33.941656 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 16:37:33.941656 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 16:37:33.941656 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 16:37:33.941656 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 16:37:33.941656 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application with Lock( Aug 01 16:37:33.941656 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 16:37:33.941656 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self._enter() Aug 01 16:37:33.941656 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 16:37:33.941656 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 16:37:33.941656 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 16:37:33.941656 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self.creator() Aug 01 16:37:33.941656 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 16:37:33.941656 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application created_value = creator( Aug 01 16:37:33.942657 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 16:37:33.942657 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 16:37:33.942657 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 16:37:33.942657 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application Aug 01 16:37:33.942657 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 482/982] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Aug 1 16:37:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 01 16:37:33.947496 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a4eb611-3f0a-440f-b6d5-1ca17b154be9 None admin] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:37:33.957631 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04e733db-31bf-48d4-9907-4b67be70c5d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:37:33.959211 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e733db-31bf-48d4-9907-4b67be70c5d2 None admin] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:37:33.959638 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e733db-31bf-48d4-9907-4b67be70c5d2 None admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:37:33.960054 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e733db-31bf-48d4-9907-4b67be70c5d2 None admin] PATH_INFO: `/v3/services` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:37:33.960970 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04e733db-31bf-48d4-9907-4b67be70c5d2 None admin] RBAC: Adding query filter params (type=compute) {{(pid=87058) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:37:33.961442 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04e733db-31bf-48d4-9907-4b67be70c5d2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:37:33.962837 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04e733db-31bf-48d4-9907-4b67be70c5d2 None admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:37:33.973088 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 501/983] 10.222.0.20 () {56 vars in 1112 bytes} [Mon Aug 1 16:37:33 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 16:37:33.973971 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04e733db-31bf-48d4-9907-4b67be70c5d2 None admin] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:37:34.049974 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a4eb611-3f0a-440f-b6d5-1ca17b154be9 None admin] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:37:34.078449 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG dbcounter [-] [87057] Writing DB stats keystone:SELECT=100 {{(pid=87057) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:37:34.084915 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8f6e55a-0fd3-475a-857f-845f167633b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:37:34.085533 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f6e55a-0fd3-475a-857f-845f167633b1 None admin] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:37:34.085885 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f6e55a-0fd3-475a-857f-845f167633b1 None admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:37:34.091551 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f6e55a-0fd3-475a-857f-845f167633b1 None admin] PATH_INFO: `/v3/domains` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:37:34.091551 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8f6e55a-0fd3-475a-857f-845f167633b1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87057) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:37:34.091551 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8f6e55a-0fd3-475a-857f-845f167633b1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:37:34.091830 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04e733db-31bf-48d4-9907-4b67be70c5d2 None admin] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:37:34.091921 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8f6e55a-0fd3-475a-857f-845f167633b1 None admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:37:34.100746 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 483/984] 10.222.0.20 () {56 vars in 1109 bytes} [Mon Aug 1 16:37:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 16:37:34.106520 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8f6e55a-0fd3-475a-857f-845f167633b1 None admin] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:37:34.130762 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ce6bb9-837f-43ee-b68c-693eb337668f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:37:34.131467 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ce6bb9-837f-43ee-b68c-693eb337668f None admin] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:37:34.131828 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ce6bb9-837f-43ee-b68c-693eb337668f None admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:37:34.132174 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ce6bb9-837f-43ee-b68c-693eb337668f None admin] PATH_INFO: `/v3/endpoints` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:37:34.133015 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96ce6bb9-837f-43ee-b68c-693eb337668f None admin] RBAC: Adding query filter params (interface=public, service_id=b3a8987c81bb49fd9e51dbafcbc00d5f, region_id=RegionOne) {{(pid=87058) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:37:34.133398 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96ce6bb9-837f-43ee-b68c-693eb337668f None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:37:34.134868 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96ce6bb9-837f-43ee-b68c-693eb337668f None admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:37:34.140540 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 502/985] 10.222.0.20 () {56 vars in 1251 bytes} [Mon Aug 1 16:37:33 2022] GET /identity/v3/endpoints?service_id=b3a8987c81bb49fd9e51dbafcbc00d5f&interface=public®ion_id=RegionOne => generated 194 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:37:34.145687 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ce6bb9-837f-43ee-b68c-693eb337668f None admin] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:37:34.170687 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG dbcounter [-] [87058] Writing DB stats keystone:SELECT=100 {{(pid=87058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:37:34.218375 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8f6e55a-0fd3-475a-857f-845f167633b1 None admin] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:37:34.254819 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b4815d0-5ed0-422d-8552-51df1bf53930 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:37:34.255477 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4815d0-5ed0-422d-8552-51df1bf53930 None admin] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:37:34.255757 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4815d0-5ed0-422d-8552-51df1bf53930 None admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:37:34.256032 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4815d0-5ed0-422d-8552-51df1bf53930 None admin] PATH_INFO: `/v3/projects/service` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:37:34.256289 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ce6bb9-837f-43ee-b68c-693eb337668f None admin] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:37:34.262873 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b4815d0-5ed0-422d-8552-51df1bf53930 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:37:34.264657 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b4815d0-5ed0-422d-8552-51df1bf53930 None admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:37:34.276448 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application [None req-9b4815d0-5ed0-422d-8552-51df1bf53930 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 16:37:34.276448 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 16:37:34.276448 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 16:37:34.276448 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 16:37:34.276448 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 16:37:34.276448 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 16:37:34.276448 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 16:37:34.276448 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 16:37:34.276448 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 16:37:34.276448 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 16:37:34.276448 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 16:37:34.276448 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 16:37:34.276448 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 01 16:37:34.276448 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 01 16:37:34.276448 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 01 16:37:34.277362 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 01 16:37:34.277362 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 16:37:34.277362 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 16:37:34.277362 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 16:37:34.277362 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 16:37:34.277362 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 16:37:34.277362 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 16:37:34.277362 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 16:37:34.277362 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application with Lock( Aug 01 16:37:34.277362 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 16:37:34.277362 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self._enter() Aug 01 16:37:34.277362 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 16:37:34.277362 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 16:37:34.277362 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 16:37:34.277362 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self.creator() Aug 01 16:37:34.277362 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 16:37:34.277362 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application created_value = creator( Aug 01 16:37:34.278295 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 01 16:37:34.278295 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 01 16:37:34.278295 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 01 16:37:34.278295 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 01 16:37:34.278295 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 01 16:37:34.278295 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 01 16:37:34.278295 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 16:37:34.278295 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application Aug 01 16:37:34.278295 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 484/986] 10.222.0.20 () {56 vars in 1111 bytes} [Mon Aug 1 16:37:34 2022] GET /identity/v3/projects/service => generated 88 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 01 16:37:34.283847 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b4815d0-5ed0-422d-8552-51df1bf53930 None admin] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:37:34.300841 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4ed8177-9571-4cb6-96ae-caa433c6bdcf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:37:34.301483 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ed8177-9571-4cb6-96ae-caa433c6bdcf None admin] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:37:34.301830 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ed8177-9571-4cb6-96ae-caa433c6bdcf None admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:37:34.302168 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ed8177-9571-4cb6-96ae-caa433c6bdcf None admin] PATH_INFO: `/v3/services` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:37:34.302927 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4ed8177-9571-4cb6-96ae-caa433c6bdcf None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:37:34.304258 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4ed8177-9571-4cb6-96ae-caa433c6bdcf None admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:37:34.322077 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 503/987] 10.222.0.20 () {56 vars in 1087 bytes} [Mon Aug 1 16:37:34 2022] GET /identity/v3/services => generated 1727 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 01 16:37:34.353125 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG dbcounter [-] [87057] Writing DB stats keystone:SELECT=100 {{(pid=87057) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:37:34.410268 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b4815d0-5ed0-422d-8552-51df1bf53930 None admin] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:37:34.447764 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5351df1b-b7cb-4467-87e8-e15ef85be863 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:37:34.448404 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5351df1b-b7cb-4467-87e8-e15ef85be863 None admin] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:37:34.448691 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5351df1b-b7cb-4467-87e8-e15ef85be863 None admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:37:34.448965 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5351df1b-b7cb-4467-87e8-e15ef85be863 None admin] PATH_INFO: `/v3/projects` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:37:34.449732 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5351df1b-b7cb-4467-87e8-e15ef85be863 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87057) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:37:34.450054 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5351df1b-b7cb-4467-87e8-e15ef85be863 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:37:34.451543 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5351df1b-b7cb-4467-87e8-e15ef85be863 None admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:37:34.465590 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 485/988] 10.222.0.20 () {56 vars in 1148 bytes} [Mon Aug 1 16:37:34 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 16:37:34.471698 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4ed8177-9571-4cb6-96ae-caa433c6bdcf None admin] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:37:34.560034 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG dbcounter [-] [87058] Writing DB stats keystone:SELECT=100 {{(pid=87058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:37:34.577018 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4ed8177-9571-4cb6-96ae-caa433c6bdcf None admin] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:37:34.612651 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20d3051d-d6eb-4c4f-bc95-7538bba5dbad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:37:34.613272 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d3051d-d6eb-4c4f-bc95-7538bba5dbad None admin] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:37:34.613553 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d3051d-d6eb-4c4f-bc95-7538bba5dbad None admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:37:34.613830 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d3051d-d6eb-4c4f-bc95-7538bba5dbad None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:37:34.614682 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20d3051d-d6eb-4c4f-bc95-7538bba5dbad None admin] RBAC: Adding query filter params (role.id=653d2ccc68e8415191e22643427e28bc, scope.project.id=7b992d6a65df4bf9b27c13192d57c5f0, user.id=b6a93d1c6c82489099adc8b413280fc0) {{(pid=87058) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:37:34.614995 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20d3051d-d6eb-4c4f-bc95-7538bba5dbad None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:37:34.616538 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20d3051d-d6eb-4c4f-bc95-7538bba5dbad None admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:37:34.630058 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 504/989] 10.222.0.20 () {56 vars in 1374 bytes} [Mon Aug 1 16:37:34 2022] GET /identity/v3/role_assignments?user.id=b6a93d1c6c82489099adc8b413280fc0&scope.project.id=7b992d6a65df4bf9b27c13192d57c5f0&role.id=653d2ccc68e8415191e22643427e28bc => generated 608 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:37:35.201159 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db4389d-70c0-409b-909c-09a64a5ee938 None None] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:37:35.201671 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db4389d-70c0-409b-909c-09a64a5ee938 None None] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:37:35.201671 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db4389d-70c0-409b-909c-09a64a5ee938 None None] PATH_INFO: `/` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:37:35.203156 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 486/990] 10.222.0.20 () {54 vars in 926 bytes} [Mon Aug 1 16:37:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 16:37:35.208801 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2d5c58-3f2a-4505-ae54-87c1bb9a1e10 None None] REQUEST_METHOD: `POST` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:37:35.209206 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2d5c58-3f2a-4505-ae54-87c1bb9a1e10 None None] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:37:35.209371 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2d5c58-3f2a-4505-ae54-87c1bb9a1e10 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:37:35.229851 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.auth.core [None req-4c2d5c58-3f2a-4505-ae54-87c1bb9a1e10 None None] MFA Rules not processed for user `1007f8e379b74e169b98fb7c3c0ba5b4`. Rule list: `[]` (Enabled: `True`). {{(pid=87058) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:37:35.231059 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.fernet_utils [None req-4c2d5c58-3f2a-4505-ae54-87c1bb9a1e10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:37:35.299532 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 505/991] 10.222.0.20 () {58 vars in 1023 bytes} [Mon Aug 1 16:37:35 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 01 16:37:35.322795 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db4389d-70c0-409b-909c-09a64a5ee938 None None] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:37:35.433958 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db4389d-70c0-409b-909c-09a64a5ee938 None None] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:37:35.447222 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG dbcounter [-] [87057] Writing DB stats keystone:SELECT=100 {{(pid=87057) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:37:35.477363 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3d1bf7f-3d17-40e1-bff1-0f7d9a053ddc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:37:35.478398 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d1bf7f-3d17-40e1-bff1-0f7d9a053ddc None admin] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:37:35.478997 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d1bf7f-3d17-40e1-bff1-0f7d9a053ddc None admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:37:35.479533 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d1bf7f-3d17-40e1-bff1-0f7d9a053ddc None admin] PATH_INFO: `/v3/services/object-store` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:37:35.480589 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3d1bf7f-3d17-40e1-bff1-0f7d9a053ddc None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:37:35.481959 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3d1bf7f-3d17-40e1-bff1-0f7d9a053ddc None admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:37:35.491619 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application [None req-f3d1bf7f-3d17-40e1-bff1-0f7d9a053ddc None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 01 16:37:35.491619 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 16:37:35.491619 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 01 16:37:35.491619 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 01 16:37:35.491619 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 01 16:37:35.491619 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 01 16:37:35.491619 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 01 16:37:35.491619 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 01 16:37:35.491619 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 01 16:37:35.491619 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application Aug 01 16:37:35.491619 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 01 16:37:35.491619 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application Aug 01 16:37:35.491619 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 16:37:35.491619 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 16:37:35.491619 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 16:37:35.491619 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 16:37:35.491619 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 16:37:35.492626 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 16:37:35.492626 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 16:37:35.492626 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 16:37:35.492626 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 16:37:35.492626 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 16:37:35.492626 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 16:37:35.492626 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 01 16:37:35.492626 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 01 16:37:35.492626 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 01 16:37:35.492626 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 01 16:37:35.492626 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 16:37:35.492626 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 16:37:35.492626 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 16:37:35.492626 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 16:37:35.492626 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 16:37:35.492626 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 16:37:35.493621 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 16:37:35.493621 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application with Lock( Aug 01 16:37:35.493621 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 16:37:35.493621 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self._enter() Aug 01 16:37:35.493621 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 16:37:35.493621 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 16:37:35.493621 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 16:37:35.493621 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self.creator() Aug 01 16:37:35.493621 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 16:37:35.493621 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application created_value = creator( Aug 01 16:37:35.493621 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 01 16:37:35.493621 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 01 16:37:35.493621 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 01 16:37:35.493621 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application Aug 01 16:37:35.493621 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 487/992] 10.222.0.20 () {56 vars in 1126 bytes} [Mon Aug 1 16:37:35 2022] GET /identity/v3/services/object-store => generated 93 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 16:37:35.498689 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c2d5c58-3f2a-4505-ae54-87c1bb9a1e10 None None] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:37:35.581023 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG dbcounter [-] [87058] Writing DB stats keystone:SELECT=100 {{(pid=87058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:37:35.602777 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c2d5c58-3f2a-4505-ae54-87c1bb9a1e10 None None] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:37:35.647647 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2798cc23-a9b4-4d75-82c8-5ed0f0241f2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:37:35.648272 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2798cc23-a9b4-4d75-82c8-5ed0f0241f2c None admin] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:37:35.648552 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2798cc23-a9b4-4d75-82c8-5ed0f0241f2c None admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:37:35.648821 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2798cc23-a9b4-4d75-82c8-5ed0f0241f2c None admin] PATH_INFO: `/v3/services` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:37:35.649500 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2798cc23-a9b4-4d75-82c8-5ed0f0241f2c None admin] RBAC: Adding query filter params (name=object-store) {{(pid=87058) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:37:35.649800 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2798cc23-a9b4-4d75-82c8-5ed0f0241f2c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:37:35.651187 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2798cc23-a9b4-4d75-82c8-5ed0f0241f2c None admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:37:35.661016 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 506/993] 10.222.0.20 () {56 vars in 1122 bytes} [Mon Aug 1 16:37:35 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:37:35.661927 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3d1bf7f-3d17-40e1-bff1-0f7d9a053ddc None admin] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:37:35.765810 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3d1bf7f-3d17-40e1-bff1-0f7d9a053ddc None admin] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:37:35.802471 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a2a249a-f175-4dda-9293-31fada8e67d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:37:35.803098 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2a249a-f175-4dda-9293-31fada8e67d6 None admin] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:37:35.803388 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2a249a-f175-4dda-9293-31fada8e67d6 None admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:37:35.803663 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2a249a-f175-4dda-9293-31fada8e67d6 None admin] PATH_INFO: `/v3/services` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:37:35.804349 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a2a249a-f175-4dda-9293-31fada8e67d6 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=87057) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:37:35.804652 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a2a249a-f175-4dda-9293-31fada8e67d6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:37:35.806020 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a2a249a-f175-4dda-9293-31fada8e67d6 None admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:37:35.815894 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 488/994] 10.222.0.20 () {56 vars in 1122 bytes} [Mon Aug 1 16:37:35 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 16:37:35.816707 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2798cc23-a9b4-4d75-82c8-5ed0f0241f2c None admin] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:37:35.926455 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2798cc23-a9b4-4d75-82c8-5ed0f0241f2c None admin] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:37:35.962423 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e64e32d0-ed63-419c-82b5-73e2a5926f41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:37:35.963038 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64e32d0-ed63-419c-82b5-73e2a5926f41 None admin] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:37:35.963316 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64e32d0-ed63-419c-82b5-73e2a5926f41 None admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:37:35.963589 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64e32d0-ed63-419c-82b5-73e2a5926f41 None admin] PATH_INFO: `/v3/endpoints` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:37:35.964348 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e64e32d0-ed63-419c-82b5-73e2a5926f41 None admin] RBAC: Adding query filter params (interface=admin, service_id=4a938303141c4bdc8308a8422744fc85, region_id=RegionOne) {{(pid=87058) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:37:35.964654 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e64e32d0-ed63-419c-82b5-73e2a5926f41 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:37:35.966019 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e64e32d0-ed63-419c-82b5-73e2a5926f41 None admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:37:35.976010 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 507/995] 10.222.0.20 () {56 vars in 1249 bytes} [Mon Aug 1 16:37:35 2022] GET /identity/v3/endpoints?service_id=4a938303141c4bdc8308a8422744fc85&interface=admin®ion_id=RegionOne => generated 193 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:37:35.976884 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a2a249a-f175-4dda-9293-31fada8e67d6 None admin] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:37:36.018932 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG dbcounter [-] [87057] Writing DB stats keystone:SELECT=100 {{(pid=87057) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:37:36.077195 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a2a249a-f175-4dda-9293-31fada8e67d6 None admin] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:37:36.111914 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-888d48de-8811-4e64-a28e-6eb2622b5d54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:37:36.112526 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888d48de-8811-4e64-a28e-6eb2622b5d54 None admin] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:37:36.112802 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888d48de-8811-4e64-a28e-6eb2622b5d54 None admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:37:36.113070 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888d48de-8811-4e64-a28e-6eb2622b5d54 None admin] PATH_INFO: `/v3/services` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:37:36.113717 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-888d48de-8811-4e64-a28e-6eb2622b5d54 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:37:36.115684 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-888d48de-8811-4e64-a28e-6eb2622b5d54 None admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:37:36.130608 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 489/996] 10.222.0.20 () {56 vars in 1087 bytes} [Mon Aug 1 16:37:35 2022] GET /identity/v3/services => generated 1727 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 01 16:37:36.357937 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464f85be-0286-4f85-8477-270e1e86ddac None None] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:37:36.358488 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464f85be-0286-4f85-8477-270e1e86ddac None None] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:37:36.358488 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464f85be-0286-4f85-8477-270e1e86ddac None None] PATH_INFO: `/` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:37:36.359837 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 508/997] 10.222.0.20 () {54 vars in 926 bytes} [Mon Aug 1 16:37:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 16:37:36.365639 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f787e8-d7e4-43d3-a861-62839a921de6 None None] REQUEST_METHOD: `POST` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:37:36.366202 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f787e8-d7e4-43d3-a861-62839a921de6 None None] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:37:36.366202 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f787e8-d7e4-43d3-a861-62839a921de6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:37:36.384757 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.auth.core [None req-d7f787e8-d7e4-43d3-a861-62839a921de6 None None] MFA Rules not processed for user `1007f8e379b74e169b98fb7c3c0ba5b4`. Rule list: `[]` (Enabled: `True`). {{(pid=87057) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:37:36.386329 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.fernet_utils [None req-d7f787e8-d7e4-43d3-a861-62839a921de6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87057) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:37:36.455680 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 490/998] 10.222.0.20 () {58 vars in 1023 bytes} [Mon Aug 1 16:37:36 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 01 16:37:36.477683 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-464f85be-0286-4f85-8477-270e1e86ddac None None] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:37:36.487167 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG dbcounter [-] [87058] Writing DB stats keystone:SELECT=100 {{(pid=87058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:37:36.583536 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-464f85be-0286-4f85-8477-270e1e86ddac None None] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:37:36.618360 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e371b1-5787-408e-92df-0fd5643aaab8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:37:36.618921 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e371b1-5787-408e-92df-0fd5643aaab8 None admin] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:37:36.619199 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e371b1-5787-408e-92df-0fd5643aaab8 None admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:37:36.619466 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e371b1-5787-408e-92df-0fd5643aaab8 None admin] PATH_INFO: `/v3/services/compute` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:37:36.620089 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98e371b1-5787-408e-92df-0fd5643aaab8 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:37:36.621446 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98e371b1-5787-408e-92df-0fd5643aaab8 None admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:37:36.630405 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application [None req-98e371b1-5787-408e-92df-0fd5643aaab8 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Aug 01 16:37:36.630405 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 16:37:36.630405 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 01 16:37:36.630405 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 01 16:37:36.630405 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 01 16:37:36.630405 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 01 16:37:36.630405 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 01 16:37:36.630405 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 01 16:37:36.630405 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 01 16:37:36.630405 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application Aug 01 16:37:36.630405 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 01 16:37:36.630405 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application Aug 01 16:37:36.630405 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 16:37:36.630405 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 16:37:36.630405 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 16:37:36.630405 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 16:37:36.630405 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 16:37:36.631447 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 16:37:36.631447 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 16:37:36.631447 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 16:37:36.631447 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 16:37:36.631447 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 16:37:36.631447 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 16:37:36.631447 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 01 16:37:36.631447 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 01 16:37:36.631447 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 01 16:37:36.631447 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 01 16:37:36.631447 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 16:37:36.631447 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 16:37:36.631447 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 16:37:36.631447 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 16:37:36.631447 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 16:37:36.631447 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 16:37:36.632357 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 16:37:36.632357 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application with Lock( Aug 01 16:37:36.632357 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 16:37:36.632357 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self._enter() Aug 01 16:37:36.632357 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 16:37:36.632357 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 16:37:36.632357 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 16:37:36.632357 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self.creator() Aug 01 16:37:36.632357 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 16:37:36.632357 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application created_value = creator( Aug 01 16:37:36.632357 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 01 16:37:36.632357 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 01 16:37:36.632357 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 01 16:37:36.632357 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application Aug 01 16:37:36.632357 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 509/999] 10.222.0.20 () {56 vars in 1111 bytes} [Mon Aug 1 16:37:36 2022] GET /identity/v3/services/compute => generated 88 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 16:37:36.638249 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7f787e8-d7e4-43d3-a861-62839a921de6 None None] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:37:36.670713 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG dbcounter [-] [87057] Writing DB stats keystone:SELECT=100 {{(pid=87057) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:37:36.672106 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc93b363-4c7b-4144-a532-ee76bd69c70b None None] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:37:36.672227 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc93b363-4c7b-4144-a532-ee76bd69c70b None None] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:37:36.672597 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc93b363-4c7b-4144-a532-ee76bd69c70b None None] PATH_INFO: `/` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:37:36.673968 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 510/1000] 10.222.0.20 () {54 vars in 926 bytes} [Mon Aug 1 16:37:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 01 16:37:36.679306 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2dcbf8-1531-457f-98b0-de52e9b805c8 None None] REQUEST_METHOD: `POST` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:37:36.679590 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2dcbf8-1531-457f-98b0-de52e9b805c8 None None] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:37:36.679864 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2dcbf8-1531-457f-98b0-de52e9b805c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:37:36.698906 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.auth.core [None req-8a2dcbf8-1531-457f-98b0-de52e9b805c8 None None] MFA Rules not processed for user `1007f8e379b74e169b98fb7c3c0ba5b4`. Rule list: `[]` (Enabled: `True`). {{(pid=87058) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:37:36.700104 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.fernet_utils [None req-8a2dcbf8-1531-457f-98b0-de52e9b805c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:37:36.746045 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7f787e8-d7e4-43d3-a861-62839a921de6 None None] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:37:36.767773 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG dbcounter [-] [87058] Writing DB stats keystone:SELECT=100 {{(pid=87058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:37:36.773865 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 511/1001] 10.222.0.20 () {58 vars in 1023 bytes} [Mon Aug 1 16:37:36 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 01 16:37:36.782886 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d824ff7-85af-4de3-8d41-1c752d220d89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:37:36.783848 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d824ff7-85af-4de3-8d41-1c752d220d89 None admin] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:37:36.784314 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d824ff7-85af-4de3-8d41-1c752d220d89 None admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:37:36.784787 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d824ff7-85af-4de3-8d41-1c752d220d89 None admin] PATH_INFO: `/v3/services` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:37:36.785647 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d824ff7-85af-4de3-8d41-1c752d220d89 None admin] RBAC: Adding query filter params (name=compute) {{(pid=87057) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:37:36.786297 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d824ff7-85af-4de3-8d41-1c752d220d89 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:37:36.787888 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d824ff7-85af-4de3-8d41-1c752d220d89 None admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:37:36.793460 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 491/1002] 10.222.0.20 () {56 vars in 1112 bytes} [Mon Aug 1 16:37:36 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:37:36.797894 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a2dcbf8-1531-457f-98b0-de52e9b805c8 None None] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:37:36.798527 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d824ff7-85af-4de3-8d41-1c752d220d89 None admin] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:37:36.906494 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a2dcbf8-1531-457f-98b0-de52e9b805c8 None None] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:37:36.907396 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG dbcounter [-] [87057] Writing DB stats keystone:SELECT=100 {{(pid=87057) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:37:36.914298 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d824ff7-85af-4de3-8d41-1c752d220d89 None admin] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:37:36.945291 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9942a085-f282-4a74-b447-411c2e6cdcf6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:37:36.946045 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9942a085-f282-4a74-b447-411c2e6cdcf6 None admin] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:37:36.946348 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9942a085-f282-4a74-b447-411c2e6cdcf6 None admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:37:36.946612 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9942a085-f282-4a74-b447-411c2e6cdcf6 None admin] PATH_INFO: `/v3/services/image` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:37:36.947240 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9942a085-f282-4a74-b447-411c2e6cdcf6 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:37:36.948608 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9942a085-f282-4a74-b447-411c2e6cdcf6 None admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:37:36.951439 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49f7d547-8763-4337-984d-33e329eade4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:37:36.952160 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f7d547-8763-4337-984d-33e329eade4b None admin] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:37:36.952440 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f7d547-8763-4337-984d-33e329eade4b None admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:37:36.952614 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f7d547-8763-4337-984d-33e329eade4b None admin] PATH_INFO: `/v3/services` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:37:36.953269 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49f7d547-8763-4337-984d-33e329eade4b None admin] RBAC: Adding query filter params (type=compute) {{(pid=87057) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:37:36.953571 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49f7d547-8763-4337-984d-33e329eade4b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:37:36.954975 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49f7d547-8763-4337-984d-33e329eade4b None admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:37:36.957582 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application [None req-9942a085-f282-4a74-b447-411c2e6cdcf6 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Aug 01 16:37:36.957582 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 16:37:36.957582 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 01 16:37:36.957582 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 01 16:37:36.957582 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 01 16:37:36.957582 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 01 16:37:36.957582 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 01 16:37:36.957582 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 01 16:37:36.957582 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 01 16:37:36.957582 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application Aug 01 16:37:36.957582 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 01 16:37:36.957582 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application Aug 01 16:37:36.957582 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 16:37:36.957582 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 16:37:36.957582 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 16:37:36.957582 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 16:37:36.957582 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 16:37:36.958878 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 16:37:36.958878 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 16:37:36.958878 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 16:37:36.958878 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 16:37:36.958878 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 16:37:36.958878 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 16:37:36.958878 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 01 16:37:36.958878 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 01 16:37:36.958878 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 01 16:37:36.958878 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 01 16:37:36.958878 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 16:37:36.958878 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 16:37:36.958878 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 16:37:36.958878 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 16:37:36.958878 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 16:37:36.958878 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 16:37:36.959743 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 16:37:36.959743 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application with Lock( Aug 01 16:37:36.959743 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 16:37:36.959743 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self._enter() Aug 01 16:37:36.959743 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 16:37:36.959743 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 16:37:36.959743 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 16:37:36.959743 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self.creator() Aug 01 16:37:36.959743 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 16:37:36.959743 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application created_value = creator( Aug 01 16:37:36.959743 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 01 16:37:36.959743 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 01 16:37:36.959743 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 01 16:37:36.959743 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application Aug 01 16:37:36.959743 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 512/1003] 10.222.0.20 () {56 vars in 1105 bytes} [Mon Aug 1 16:37:36 2022] GET /identity/v3/services/image => generated 86 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 16:37:36.967826 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 492/1004] 10.222.0.20 () {56 vars in 1112 bytes} [Mon Aug 1 16:37:36 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 16:37:36.968510 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9942a085-f282-4a74-b447-411c2e6cdcf6 None admin] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:37:36.968618 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49f7d547-8763-4337-984d-33e329eade4b None admin] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:37:37.035323 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG dbcounter [-] [87058] Writing DB stats keystone:SELECT=100 {{(pid=87058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:37:37.074109 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49f7d547-8763-4337-984d-33e329eade4b None admin] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:37:37.078901 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9942a085-f282-4a74-b447-411c2e6cdcf6 None admin] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:37:37.112232 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-236e725b-0425-4cdf-a124-521a2d96de85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:37:37.112856 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236e725b-0425-4cdf-a124-521a2d96de85 None admin] REQUEST_METHOD: `POST` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:37:37.113139 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236e725b-0425-4cdf-a124-521a2d96de85 None admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:37:37.113421 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236e725b-0425-4cdf-a124-521a2d96de85 None admin] PATH_INFO: `/v3/endpoints` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:37:37.114220 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-236e725b-0425-4cdf-a124-521a2d96de85 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:37:37.115606 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-236e725b-0425-4cdf-a124-521a2d96de85 None admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:37:37.118730 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a61635-13ee-44f7-8e15-647fe39cc61d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:37:37.119566 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a61635-13ee-44f7-8e15-647fe39cc61d None admin] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:37:37.120063 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a61635-13ee-44f7-8e15-647fe39cc61d None admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:37:37.120460 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a61635-13ee-44f7-8e15-647fe39cc61d None admin] PATH_INFO: `/v3/services` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:37:37.121214 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6a61635-13ee-44f7-8e15-647fe39cc61d None admin] RBAC: Adding query filter params (name=image) {{(pid=87058) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:37:37.121454 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6a61635-13ee-44f7-8e15-647fe39cc61d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:37:37.122836 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6a61635-13ee-44f7-8e15-647fe39cc61d None admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:37:37.127774 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 513/1005] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Aug 1 16:37:36 2022] GET /identity/v3/services?name=image => generated 122 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 16:37:37.134291 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a61635-13ee-44f7-8e15-647fe39cc61d None admin] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:37:37.190694 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 493/1006] 10.222.0.20 () {60 vars in 1144 bytes} [Mon Aug 1 16:37:36 2022] POST /identity/v3/endpoints => generated 334 bytes in 222 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 16:37:37.245952 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a61635-13ee-44f7-8e15-647fe39cc61d None admin] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:37:37.277037 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG dbcounter [-] [87058] Writing DB stats keystone:SELECT=100 {{(pid=87058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:37:37.292832 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-329d573c-2a1b-4cc5-8579-fa6859abd87b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:37:37.293464 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329d573c-2a1b-4cc5-8579-fa6859abd87b None admin] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:37:37.293744 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329d573c-2a1b-4cc5-8579-fa6859abd87b None admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:37:37.294010 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329d573c-2a1b-4cc5-8579-fa6859abd87b None admin] PATH_INFO: `/v3/services` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:37:37.294720 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-329d573c-2a1b-4cc5-8579-fa6859abd87b None admin] RBAC: Adding query filter params (type=image) {{(pid=87058) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:37:37.295020 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-329d573c-2a1b-4cc5-8579-fa6859abd87b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:37:37.296392 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-329d573c-2a1b-4cc5-8579-fa6859abd87b None admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:37:37.301766 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 514/1007] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Aug 1 16:37:37 2022] GET /identity/v3/services?type=image => generated 122 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 16:37:38.131080 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0288ae50-488b-407a-82a5-3ff79455939c None None] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:37:38.131620 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0288ae50-488b-407a-82a5-3ff79455939c None None] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:37:38.131620 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0288ae50-488b-407a-82a5-3ff79455939c None None] PATH_INFO: `/` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:37:38.132984 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 494/1008] 10.222.0.20 () {54 vars in 926 bytes} [Mon Aug 1 16:37:38 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 16:37:38.138267 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7924b30c-397b-4c1c-96bd-2cf7eff927a7 None None] REQUEST_METHOD: `POST` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:37:38.138650 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7924b30c-397b-4c1c-96bd-2cf7eff927a7 None None] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:37:38.138811 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7924b30c-397b-4c1c-96bd-2cf7eff927a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:37:38.157509 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.auth.core [None req-7924b30c-397b-4c1c-96bd-2cf7eff927a7 None None] MFA Rules not processed for user `1007f8e379b74e169b98fb7c3c0ba5b4`. Rule list: `[]` (Enabled: `True`). {{(pid=87058) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:37:38.158664 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.fernet_utils [None req-7924b30c-397b-4c1c-96bd-2cf7eff927a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:37:38.224591 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 515/1009] 10.222.0.20 () {58 vars in 1023 bytes} [Mon Aug 1 16:37:38 2022] POST /identity/v3/auth/tokens => generated 1716 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 01 16:37:38.246778 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0288ae50-488b-407a-82a5-3ff79455939c None None] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:37:38.266714 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG dbcounter [-] [87057] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87057) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:37:38.352410 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0288ae50-488b-407a-82a5-3ff79455939c None None] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:37:38.387053 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ca4a6d-76d0-41b5-a34f-3b562c09ff12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:37:38.387705 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ca4a6d-76d0-41b5-a34f-3b562c09ff12 None admin] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:37:38.387982 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ca4a6d-76d0-41b5-a34f-3b562c09ff12 None admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:37:38.388258 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ca4a6d-76d0-41b5-a34f-3b562c09ff12 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:37:38.388894 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6ca4a6d-76d0-41b5-a34f-3b562c09ff12 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:37:38.390567 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6ca4a6d-76d0-41b5-a34f-3b562c09ff12 None admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:37:38.400320 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application [None req-e6ca4a6d-76d0-41b5-a34f-3b562c09ff12 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 01 16:37:38.400320 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 16:37:38.400320 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 01 16:37:38.400320 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 01 16:37:38.400320 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 01 16:37:38.400320 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 01 16:37:38.400320 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 01 16:37:38.400320 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 01 16:37:38.400320 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 01 16:37:38.400320 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application Aug 01 16:37:38.400320 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 01 16:37:38.400320 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application Aug 01 16:37:38.400320 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 16:37:38.400320 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 16:37:38.400320 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 16:37:38.400320 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 16:37:38.400320 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 16:37:38.401343 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 16:37:38.401343 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 16:37:38.401343 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 16:37:38.401343 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 16:37:38.401343 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 16:37:38.401343 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 16:37:38.401343 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 01 16:37:38.401343 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 01 16:37:38.401343 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 01 16:37:38.401343 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 01 16:37:38.401343 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 16:37:38.401343 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 16:37:38.401343 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 16:37:38.401343 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 16:37:38.401343 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 16:37:38.401343 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 16:37:38.407801 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 16:37:38.407801 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application with Lock( Aug 01 16:37:38.407801 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 16:37:38.407801 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self._enter() Aug 01 16:37:38.407801 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 16:37:38.407801 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 16:37:38.407801 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 16:37:38.407801 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self.creator() Aug 01 16:37:38.407801 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 16:37:38.407801 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application created_value = creator( Aug 01 16:37:38.407801 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 01 16:37:38.407801 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 01 16:37:38.407801 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 01 16:37:38.407801 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application Aug 01 16:37:38.407801 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 495/1010] 10.222.0.20 () {56 vars in 1126 bytes} [Mon Aug 1 16:37:38 2022] GET /identity/v3/services/object-store => generated 93 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 16:37:38.409027 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7924b30c-397b-4c1c-96bd-2cf7eff927a7 None None] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:37:38.513523 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7924b30c-397b-4c1c-96bd-2cf7eff927a7 None None] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:37:38.531618 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG dbcounter [-] [87058] Writing DB stats keystone:SELECT=100 {{(pid=87058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:37:38.549152 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eff6704-b607-4ba1-95bd-3fa9bb55db09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:37:38.549770 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eff6704-b607-4ba1-95bd-3fa9bb55db09 None admin] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:37:38.550069 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eff6704-b607-4ba1-95bd-3fa9bb55db09 None admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:37:38.550404 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eff6704-b607-4ba1-95bd-3fa9bb55db09 None admin] PATH_INFO: `/v3/services` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:37:38.551094 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eff6704-b607-4ba1-95bd-3fa9bb55db09 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=87058) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:37:38.551397 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eff6704-b607-4ba1-95bd-3fa9bb55db09 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:37:38.552792 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eff6704-b607-4ba1-95bd-3fa9bb55db09 None admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:37:38.558315 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 516/1011] 10.222.0.20 () {56 vars in 1122 bytes} [Mon Aug 1 16:37:38 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:37:38.563794 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ca4a6d-76d0-41b5-a34f-3b562c09ff12 None admin] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:37:38.646301 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG dbcounter [-] [87057] Writing DB stats keystone:SELECT=100 {{(pid=87057) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:37:38.665493 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ca4a6d-76d0-41b5-a34f-3b562c09ff12 None admin] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:37:38.700345 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c46e0db8-b26e-4bd9-b1c0-70bddee282ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:37:38.700957 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c46e0db8-b26e-4bd9-b1c0-70bddee282ed None admin] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:37:38.701235 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c46e0db8-b26e-4bd9-b1c0-70bddee282ed None admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:37:38.701508 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c46e0db8-b26e-4bd9-b1c0-70bddee282ed None admin] PATH_INFO: `/v3/services` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:37:38.702222 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c46e0db8-b26e-4bd9-b1c0-70bddee282ed None admin] RBAC: Adding query filter params (type=object-store) {{(pid=87057) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:37:38.702526 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c46e0db8-b26e-4bd9-b1c0-70bddee282ed None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:37:38.703886 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c46e0db8-b26e-4bd9-b1c0-70bddee282ed None admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:37:38.713382 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 496/1012] 10.222.0.20 () {56 vars in 1122 bytes} [Mon Aug 1 16:37:38 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:37:38.714599 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eff6704-b607-4ba1-95bd-3fa9bb55db09 None admin] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:37:38.818537 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eff6704-b607-4ba1-95bd-3fa9bb55db09 None admin] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:37:38.853321 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-232da316-0f37-46d9-b7fb-3c402b4e2fbc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:37:38.853941 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232da316-0f37-46d9-b7fb-3c402b4e2fbc None admin] REQUEST_METHOD: `POST` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:37:38.854253 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232da316-0f37-46d9-b7fb-3c402b4e2fbc None admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:37:38.854530 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232da316-0f37-46d9-b7fb-3c402b4e2fbc None admin] PATH_INFO: `/v3/endpoints` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:37:38.855250 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-232da316-0f37-46d9-b7fb-3c402b4e2fbc None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:37:38.856613 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-232da316-0f37-46d9-b7fb-3c402b4e2fbc None admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:37:38.962159 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 517/1013] 10.222.0.20 () {60 vars in 1144 bytes} [Mon Aug 1 16:37:38 2022] POST /identity/v3/endpoints => generated 325 bytes in 241 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 16:37:39.313837 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a96c9cd-8251-4e17-9807-00c64803e3f3 None None] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:37:39.314411 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a96c9cd-8251-4e17-9807-00c64803e3f3 None None] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:37:39.314411 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a96c9cd-8251-4e17-9807-00c64803e3f3 None None] PATH_INFO: `/` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:37:39.315782 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 497/1014] 10.222.0.20 () {54 vars in 926 bytes} [Mon Aug 1 16:37:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 16:37:39.321650 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbebd539-6de0-4e70-90bc-2ce01735c643 None None] REQUEST_METHOD: `POST` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:37:39.322151 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbebd539-6de0-4e70-90bc-2ce01735c643 None None] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:37:39.322244 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbebd539-6de0-4e70-90bc-2ce01735c643 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:37:39.339330 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15aa226-d53c-418e-ba7b-dfdab4f3bb39 None None] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:37:39.339626 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15aa226-d53c-418e-ba7b-dfdab4f3bb39 None None] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:37:39.339889 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15aa226-d53c-418e-ba7b-dfdab4f3bb39 None None] PATH_INFO: `/` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:37:39.341146 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.auth.core [None req-cbebd539-6de0-4e70-90bc-2ce01735c643 None None] MFA Rules not processed for user `1007f8e379b74e169b98fb7c3c0ba5b4`. Rule list: `[]` (Enabled: `True`). {{(pid=87058) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:37:39.341255 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 498/1015] 10.222.0.20 () {54 vars in 926 bytes} [Mon Aug 1 16:37:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 01 16:37:39.342376 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.fernet_utils [None req-cbebd539-6de0-4e70-90bc-2ce01735c643 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:37:39.346242 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f898b75d-9876-4662-bbf7-83facfda2b8c None None] REQUEST_METHOD: `POST` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:37:39.346527 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f898b75d-9876-4662-bbf7-83facfda2b8c None None] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:37:39.346798 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f898b75d-9876-4662-bbf7-83facfda2b8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:37:39.364388 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.auth.core [None req-f898b75d-9876-4662-bbf7-83facfda2b8c None None] MFA Rules not processed for user `1007f8e379b74e169b98fb7c3c0ba5b4`. Rule list: `[]` (Enabled: `True`). {{(pid=87057) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:37:39.365943 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.fernet_utils [None req-f898b75d-9876-4662-bbf7-83facfda2b8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87057) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:37:39.386140 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG dbcounter [-] [87058] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:37:39.425371 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 518/1016] 10.222.0.20 () {58 vars in 1023 bytes} [Mon Aug 1 16:37:39 2022] POST /identity/v3/auth/tokens => generated 1863 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 01 16:37:39.440074 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 499/1017] 10.222.0.20 () {58 vars in 1023 bytes} [Mon Aug 1 16:37:39 2022] POST /identity/v3/auth/tokens => generated 1863 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 01 16:37:39.456144 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbebd539-6de0-4e70-90bc-2ce01735c643 None None] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:37:39.464680 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f898b75d-9876-4662-bbf7-83facfda2b8c None None] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:37:39.541705 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG dbcounter [-] [87057] Writing DB stats keystone:SELECT=100 {{(pid=87057) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:37:39.568153 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f898b75d-9876-4662-bbf7-83facfda2b8c None None] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:37:39.569239 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbebd539-6de0-4e70-90bc-2ce01735c643 None None] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:37:39.606151 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-818518e7-2d15-41ed-b241-6c6df0668b41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:37:39.606797 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818518e7-2d15-41ed-b241-6c6df0668b41 None admin] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:37:39.607078 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818518e7-2d15-41ed-b241-6c6df0668b41 None admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:37:39.607351 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818518e7-2d15-41ed-b241-6c6df0668b41 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:37:39.608442 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac3ee5b-a0a7-436f-ba83-82d228165d0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:37:39.609268 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac3ee5b-a0a7-436f-ba83-82d228165d0d None admin] REQUEST_METHOD: `POST` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:37:39.609522 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac3ee5b-a0a7-436f-ba83-82d228165d0d None admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:37:39.610917 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac3ee5b-a0a7-436f-ba83-82d228165d0d None admin] PATH_INFO: `/v3/services` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:37:39.611632 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ac3ee5b-a0a7-436f-ba83-82d228165d0d None admin] RBAC: Authorizing `identity:create_service()` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:37:39.613392 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ac3ee5b-a0a7-436f-ba83-82d228165d0d None admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:37:39.619535 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-818518e7-2d15-41ed-b241-6c6df0668b41 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 16:37:39.624939 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application [None req-818518e7-2d15-41ed-b241-6c6df0668b41 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 01 16:37:39.624939 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 16:37:39.624939 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 16:37:39.624939 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 16:37:39.624939 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 16:37:39.624939 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 16:37:39.624939 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 16:37:39.624939 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 16:37:39.624939 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 16:37:39.624939 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 16:37:39.624939 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 16:37:39.624939 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 16:37:39.624939 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 16:37:39.624939 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 16:37:39.624939 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 16:37:39.624939 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 16:37:39.626665 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 16:37:39.626665 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 16:37:39.626665 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 16:37:39.626665 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 16:37:39.626665 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 16:37:39.626665 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 16:37:39.626665 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 16:37:39.626665 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 16:37:39.626665 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 16:37:39.626665 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 16:37:39.626665 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 16:37:39.626665 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application with Lock( Aug 01 16:37:39.626665 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 16:37:39.626665 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self._enter() Aug 01 16:37:39.626665 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 16:37:39.626665 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 16:37:39.627630 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 16:37:39.627630 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self.creator() Aug 01 16:37:39.627630 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 16:37:39.627630 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application created_value = creator( Aug 01 16:37:39.627630 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 16:37:39.627630 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 16:37:39.627630 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 16:37:39.627630 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 16:37:39.627630 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 16:37:39.627630 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 16:37:39.627630 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 01 16:37:39.627630 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application Aug 01 16:37:39.627630 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 500/1018] 10.222.0.20 () {56 vars in 1120 bytes} [Mon Aug 1 16:37:39 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 01 16:37:39.632248 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-818518e7-2d15-41ed-b241-6c6df0668b41 None admin] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:37:39.710059 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 519/1019] 10.222.0.20 () {60 vars in 1141 bytes} [Mon Aug 1 16:37:39 2022] POST /identity/v3/services => generated 242 bytes in 252 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 16:37:39.741071 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-818518e7-2d15-41ed-b241-6c6df0668b41 None admin] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:37:39.782071 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG dbcounter [-] [87057] Writing DB stats keystone:SELECT=100 {{(pid=87057) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:37:39.789949 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec0cfc0-9399-4660-a0a1-7e7f0ca6f1d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:37:39.790609 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec0cfc0-9399-4660-a0a1-7e7f0ca6f1d0 None admin] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:37:39.790889 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec0cfc0-9399-4660-a0a1-7e7f0ca6f1d0 None admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:37:39.791157 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec0cfc0-9399-4660-a0a1-7e7f0ca6f1d0 None admin] PATH_INFO: `/v3/roles` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:37:39.791943 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ec0cfc0-9399-4660-a0a1-7e7f0ca6f1d0 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=87057) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:37:39.792249 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ec0cfc0-9399-4660-a0a1-7e7f0ca6f1d0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:37:39.793644 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ec0cfc0-9399-4660-a0a1-7e7f0ca6f1d0 None admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:37:39.806236 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 501/1020] 10.222.0.20 () {56 vars in 1115 bytes} [Mon Aug 1 16:37:39 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 16:37:39.809698 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac3ee5b-a0a7-436f-ba83-82d228165d0d None admin] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:37:39.844363 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG dbcounter [-] [87058] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:37:39.925749 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac3ee5b-a0a7-436f-ba83-82d228165d0d None admin] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:37:39.962872 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-979e3209-00c9-4a86-965b-5074ac3ab5c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:37:39.963513 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979e3209-00c9-4a86-965b-5074ac3ab5c4 None admin] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:37:39.963784 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979e3209-00c9-4a86-965b-5074ac3ab5c4 None admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:37:39.964048 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979e3209-00c9-4a86-965b-5074ac3ab5c4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:37:39.974658 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-979e3209-00c9-4a86-965b-5074ac3ab5c4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:37:39.976102 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-979e3209-00c9-4a86-965b-5074ac3ab5c4 None admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:37:39.991696 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application [None req-979e3209-00c9-4a86-965b-5074ac3ab5c4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 16:37:39.991696 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 16:37:39.991696 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 16:37:39.991696 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 16:37:39.991696 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 16:37:39.991696 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 16:37:39.991696 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 16:37:39.991696 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 16:37:39.991696 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 16:37:39.991696 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 16:37:39.991696 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 16:37:39.991696 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 16:37:39.991696 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 16:37:39.991696 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 16:37:39.991696 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 16:37:39.992626 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 16:37:39.992626 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 16:37:39.992626 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 16:37:39.992626 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 16:37:39.992626 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 16:37:39.992626 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 16:37:39.992626 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 16:37:39.992626 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 16:37:39.992626 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application with Lock( Aug 01 16:37:39.992626 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 16:37:39.992626 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self._enter() Aug 01 16:37:39.992626 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 16:37:39.992626 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 16:37:39.992626 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 16:37:39.992626 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self.creator() Aug 01 16:37:39.992626 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 16:37:39.992626 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application created_value = creator( Aug 01 16:37:39.993619 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 16:37:39.993619 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 16:37:39.993619 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 16:37:39.993619 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application Aug 01 16:37:39.994328 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 520/1021] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Aug 1 16:37:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 16:37:39.998871 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec0cfc0-9399-4660-a0a1-7e7f0ca6f1d0 None admin] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:37:40.107060 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec0cfc0-9399-4660-a0a1-7e7f0ca6f1d0 None admin] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:37:40.142374 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f616d9-c920-46eb-b1e6-fba6de438601 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:37:40.142990 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f616d9-c920-46eb-b1e6-fba6de438601 None admin] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:37:40.143267 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f616d9-c920-46eb-b1e6-fba6de438601 None admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:37:40.143535 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f616d9-c920-46eb-b1e6-fba6de438601 None admin] PATH_INFO: `/v3/domains` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:37:40.144283 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6f616d9-c920-46eb-b1e6-fba6de438601 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87057) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:37:40.144584 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6f616d9-c920-46eb-b1e6-fba6de438601 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:37:40.147147 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6f616d9-c920-46eb-b1e6-fba6de438601 None admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:37:40.163403 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 502/1022] 10.222.0.20 () {56 vars in 1109 bytes} [Mon Aug 1 16:37:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:37:40.164671 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-979e3209-00c9-4a86-965b-5074ac3ab5c4 None admin] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:37:40.248340 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG dbcounter [-] [87058] Writing DB stats keystone:SELECT=100 {{(pid=87058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:37:40.267775 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-979e3209-00c9-4a86-965b-5074ac3ab5c4 None admin] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:37:40.302228 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e7baf7-d9f8-4009-b28c-178143cf56b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:37:40.302864 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e7baf7-d9f8-4009-b28c-178143cf56b0 None admin] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:37:40.303139 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e7baf7-d9f8-4009-b28c-178143cf56b0 None admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:37:40.303405 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e7baf7-d9f8-4009-b28c-178143cf56b0 None admin] PATH_INFO: `/v3/users/nova` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:37:40.310333 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7e7baf7-d9f8-4009-b28c-178143cf56b0 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:37:40.311751 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7e7baf7-d9f8-4009-b28c-178143cf56b0 None admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:37:40.323672 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application [None req-c7e7baf7-d9f8-4009-b28c-178143cf56b0 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 01 16:37:40.323672 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 16:37:40.323672 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 16:37:40.323672 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 16:37:40.323672 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 16:37:40.323672 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 16:37:40.323672 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 16:37:40.323672 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 16:37:40.323672 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 16:37:40.323672 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 16:37:40.323672 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 16:37:40.323672 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 16:37:40.323672 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 01 16:37:40.323672 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 01 16:37:40.323672 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 01 16:37:40.323672 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 01 16:37:40.325044 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 16:37:40.325044 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 16:37:40.325044 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 01 16:37:40.325044 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 16:37:40.325044 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 01 16:37:40.325044 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 16:37:40.325044 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 16:37:40.325044 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 16:37:40.325044 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 16:37:40.325044 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 16:37:40.325044 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 16:37:40.325044 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application with Lock( Aug 01 16:37:40.325044 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 16:37:40.325044 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self._enter() Aug 01 16:37:40.325044 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 16:37:40.325044 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 16:37:40.325044 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 16:37:40.325044 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self.creator() Aug 01 16:37:40.326616 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 16:37:40.326616 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application created_value = creator( Aug 01 16:37:40.326616 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 01 16:37:40.326616 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 01 16:37:40.326616 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 01 16:37:40.326616 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 01 16:37:40.326616 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 01 16:37:40.326616 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 01 16:37:40.326616 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 01 16:37:40.326616 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application Aug 01 16:37:40.326616 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 521/1023] 10.222.0.20 () {56 vars in 1093 bytes} [Mon Aug 1 16:37:40 2022] GET /identity/v3/users/nova => generated 82 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 16:37:40.331693 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f616d9-c920-46eb-b1e6-fba6de438601 None admin] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:37:40.387541 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG dbcounter [-] [87057] Writing DB stats keystone:SELECT=100 {{(pid=87057) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:37:40.436281 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f616d9-c920-46eb-b1e6-fba6de438601 None admin] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:37:40.470088 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-426c0453-19c1-4cb9-8cab-9255385f771c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:37:40.470709 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426c0453-19c1-4cb9-8cab-9255385f771c None admin] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:37:40.470988 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426c0453-19c1-4cb9-8cab-9255385f771c None admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:37:40.471260 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426c0453-19c1-4cb9-8cab-9255385f771c None admin] PATH_INFO: `/v3/users` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:37:40.472056 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-426c0453-19c1-4cb9-8cab-9255385f771c None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=87057) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:37:40.472360 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-426c0453-19c1-4cb9-8cab-9255385f771c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:37:40.473817 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-426c0453-19c1-4cb9-8cab-9255385f771c None admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:37:40.496247 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 503/1024] 10.222.0.20 () {56 vars in 1133 bytes} [Mon Aug 1 16:37:40 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:37:40.497863 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e7baf7-d9f8-4009-b28c-178143cf56b0 None admin] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:37:40.602455 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e7baf7-d9f8-4009-b28c-178143cf56b0 None admin] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:37:40.635830 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1333178-9563-40d6-b5d5-27e3822aa6f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:37:40.636449 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1333178-9563-40d6-b5d5-27e3822aa6f7 None admin] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:37:40.636738 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1333178-9563-40d6-b5d5-27e3822aa6f7 None admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:37:40.637007 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1333178-9563-40d6-b5d5-27e3822aa6f7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:37:40.648872 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG dbcounter [-] [87058] Writing DB stats keystone:SELECT=100 {{(pid=87058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:37:40.708037 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1333178-9563-40d6-b5d5-27e3822aa6f7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:37:40.709487 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1333178-9563-40d6-b5d5-27e3822aa6f7 None admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:37:40.811464 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 522/1025] 10.222.0.20 () {58 vars in 1282 bytes} [Mon Aug 1 16:37:40 2022] GET /identity/v3/auth/tokens => generated 1959 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 01 16:37:40.817746 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-426c0453-19c1-4cb9-8cab-9255385f771c None admin] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:37:40.930308 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-426c0453-19c1-4cb9-8cab-9255385f771c None admin] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:37:40.941513 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG dbcounter [-] [87057] Writing DB stats keystone:SELECT=100 {{(pid=87057) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:37:40.964934 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b57cc3f0-b9f1-419c-b20c-89f680f60bc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:37:40.965579 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57cc3f0-b9f1-419c-b20c-89f680f60bc4 None admin] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:37:40.965857 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57cc3f0-b9f1-419c-b20c-89f680f60bc4 None admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:37:40.966159 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57cc3f0-b9f1-419c-b20c-89f680f60bc4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:37:40.976578 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b57cc3f0-b9f1-419c-b20c-89f680f60bc4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:37:40.978026 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b57cc3f0-b9f1-419c-b20c-89f680f60bc4 None admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:37:40.985876 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb21b84-800c-4f93-a156-a6800f773253 None None] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:37:40.986216 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb21b84-800c-4f93-a156-a6800f773253 None None] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:37:40.986498 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb21b84-800c-4f93-a156-a6800f773253 None None] PATH_INFO: `/` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:37:40.988028 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 523/1026] 10.222.0.20 () {54 vars in 926 bytes} [Mon Aug 1 16:37:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 16:37:40.993363 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8545d1a-6cdd-47de-955d-c3eca93c80a5 None None] REQUEST_METHOD: `POST` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:37:40.993652 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8545d1a-6cdd-47de-955d-c3eca93c80a5 None None] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:37:40.993938 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8545d1a-6cdd-47de-955d-c3eca93c80a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:37:40.994176 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application [None req-b57cc3f0-b9f1-419c-b20c-89f680f60bc4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 16:37:40.994176 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 16:37:40.994176 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 16:37:40.994176 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 16:37:40.994176 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 16:37:40.994176 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 16:37:40.994176 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 16:37:40.994176 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 16:37:40.994176 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 16:37:40.994176 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 16:37:40.994176 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 16:37:40.994176 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 16:37:40.994176 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 16:37:40.994176 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 16:37:40.994176 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 16:37:40.995069 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 16:37:40.995069 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 16:37:40.995069 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 16:37:40.995069 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 16:37:40.995069 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 16:37:40.995069 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 16:37:40.995069 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 16:37:40.995069 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 16:37:40.995069 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application with Lock( Aug 01 16:37:40.995069 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 16:37:40.995069 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self._enter() Aug 01 16:37:40.995069 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 16:37:40.995069 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 16:37:40.995069 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 16:37:40.995069 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self.creator() Aug 01 16:37:40.995069 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 16:37:40.995069 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application created_value = creator( Aug 01 16:37:41.000630 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 16:37:41.000630 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 16:37:41.000630 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 16:37:41.000630 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application Aug 01 16:37:41.000630 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 504/1027] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Aug 1 16:37:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 16:37:41.002190 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b57cc3f0-b9f1-419c-b20c-89f680f60bc4 None admin] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:37:41.015450 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.auth.core [None req-a8545d1a-6cdd-47de-955d-c3eca93c80a5 None None] MFA Rules not processed for user `1007f8e379b74e169b98fb7c3c0ba5b4`. Rule list: `[]` (Enabled: `True`). {{(pid=87058) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:37:41.016650 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.fernet_utils [None req-a8545d1a-6cdd-47de-955d-c3eca93c80a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:37:41.055059 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG dbcounter [-] [87058] Writing DB stats keystone:SELECT=100 {{(pid=87058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:37:41.089523 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 524/1028] 10.222.0.20 () {58 vars in 1023 bytes} [Mon Aug 1 16:37:40 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 01 16:37:41.110705 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b57cc3f0-b9f1-419c-b20c-89f680f60bc4 None admin] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:37:41.112091 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8545d1a-6cdd-47de-955d-c3eca93c80a5 None None] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:37:41.149327 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5acbfb34-8309-45a9-9067-5bbd19c1c55d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:37:41.149947 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5acbfb34-8309-45a9-9067-5bbd19c1c55d None admin] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:37:41.150260 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5acbfb34-8309-45a9-9067-5bbd19c1c55d None admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:37:41.150537 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5acbfb34-8309-45a9-9067-5bbd19c1c55d None admin] PATH_INFO: `/v3/domains` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:37:41.151262 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5acbfb34-8309-45a9-9067-5bbd19c1c55d None admin] RBAC: Adding query filter params (name=Default) {{(pid=87057) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:37:41.151659 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5acbfb34-8309-45a9-9067-5bbd19c1c55d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:37:41.153017 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5acbfb34-8309-45a9-9067-5bbd19c1c55d None admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:37:41.170551 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 505/1029] 10.222.0.20 () {56 vars in 1109 bytes} [Mon Aug 1 16:37:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:37:41.171484 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5acbfb34-8309-45a9-9067-5bbd19c1c55d None admin] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:37:41.197632 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG dbcounter [-] [87057] Writing DB stats keystone:SELECT=100 {{(pid=87057) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:37:41.221397 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8545d1a-6cdd-47de-955d-c3eca93c80a5 None None] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:37:41.257159 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6b23c51-72ad-4669-bfaa-159aee242d4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:37:41.257805 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b23c51-72ad-4669-bfaa-159aee242d4e None admin] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:37:41.259042 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b23c51-72ad-4669-bfaa-159aee242d4e None admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:37:41.259042 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b23c51-72ad-4669-bfaa-159aee242d4e None admin] PATH_INFO: `/v3/domains/default` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:37:41.260471 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6b23c51-72ad-4669-bfaa-159aee242d4e None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:37:41.261912 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6b23c51-72ad-4669-bfaa-159aee242d4e None admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:37:41.263704 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 525/1030] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Aug 1 16:37:41 2022] GET /identity/v3/domains/default => generated 199 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 16:37:41.269417 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6b23c51-72ad-4669-bfaa-159aee242d4e None admin] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:37:41.277363 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5acbfb34-8309-45a9-9067-5bbd19c1c55d None admin] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:37:41.311962 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8356791-e163-46a3-82db-9cfbfea02239 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:37:41.312610 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8356791-e163-46a3-82db-9cfbfea02239 None admin] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:37:41.312894 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8356791-e163-46a3-82db-9cfbfea02239 None admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:37:41.313053 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG dbcounter [-] [87058] Writing DB stats keystone:SELECT=100 {{(pid=87058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:37:41.313231 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8356791-e163-46a3-82db-9cfbfea02239 None admin] PATH_INFO: `/v3/projects/service` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:37:41.320545 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8356791-e163-46a3-82db-9cfbfea02239 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:37:41.322004 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8356791-e163-46a3-82db-9cfbfea02239 None admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:37:41.334391 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application [None req-e8356791-e163-46a3-82db-9cfbfea02239 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 16:37:41.334391 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 16:37:41.334391 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 16:37:41.334391 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 16:37:41.334391 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 16:37:41.334391 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 16:37:41.334391 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 16:37:41.334391 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 16:37:41.334391 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 16:37:41.334391 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 16:37:41.334391 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 16:37:41.334391 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 16:37:41.334391 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 01 16:37:41.334391 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 01 16:37:41.334391 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 01 16:37:41.335949 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 01 16:37:41.335949 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 16:37:41.335949 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 16:37:41.335949 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 16:37:41.335949 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 16:37:41.335949 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 16:37:41.335949 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 16:37:41.335949 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 16:37:41.335949 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application with Lock( Aug 01 16:37:41.335949 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 16:37:41.335949 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self._enter() Aug 01 16:37:41.335949 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 16:37:41.335949 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 16:37:41.335949 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 16:37:41.335949 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self.creator() Aug 01 16:37:41.335949 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 16:37:41.335949 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application created_value = creator( Aug 01 16:37:41.337093 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 01 16:37:41.337093 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 01 16:37:41.337093 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 01 16:37:41.337093 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 01 16:37:41.337093 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 01 16:37:41.337093 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 01 16:37:41.337093 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 16:37:41.337093 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application Aug 01 16:37:41.340487 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 506/1031] 10.222.0.20 () {56 vars in 1111 bytes} [Mon Aug 1 16:37:41 2022] GET /identity/v3/projects/service => generated 88 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 16:37:41.341328 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8356791-e163-46a3-82db-9cfbfea02239 None admin] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:37:41.380910 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6b23c51-72ad-4669-bfaa-159aee242d4e None admin] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:37:41.417375 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e39a244a-255c-4bfb-8cc8-2f36ee89dcab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:37:41.418005 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e39a244a-255c-4bfb-8cc8-2f36ee89dcab None admin] REQUEST_METHOD: `POST` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:37:41.418309 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e39a244a-255c-4bfb-8cc8-2f36ee89dcab None admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:37:41.418581 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e39a244a-255c-4bfb-8cc8-2f36ee89dcab None admin] PATH_INFO: `/v3/projects` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:37:41.419274 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e39a244a-255c-4bfb-8cc8-2f36ee89dcab None admin] RBAC: Authorizing `identity:create_project()` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:37:41.420736 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e39a244a-255c-4bfb-8cc8-2f36ee89dcab None admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:37:41.427686 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG dbcounter [-] [87057] Writing DB stats keystone:SELECT=100 {{(pid=87057) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:37:41.446517 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8356791-e163-46a3-82db-9cfbfea02239 None admin] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:37:41.481371 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-698eb61e-b681-48b8-bfdb-38d63c0b169a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:37:41.481990 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698eb61e-b681-48b8-bfdb-38d63c0b169a None admin] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:37:41.482303 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698eb61e-b681-48b8-bfdb-38d63c0b169a None admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:37:41.482575 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698eb61e-b681-48b8-bfdb-38d63c0b169a None admin] PATH_INFO: `/v3/projects` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:37:41.483324 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-698eb61e-b681-48b8-bfdb-38d63c0b169a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87057) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:37:41.483628 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-698eb61e-b681-48b8-bfdb-38d63c0b169a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:37:41.485106 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-698eb61e-b681-48b8-bfdb-38d63c0b169a None admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:37:41.506084 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 526/1032] 10.222.0.20 () {60 vars in 1141 bytes} [Mon Aug 1 16:37:41 2022] POST /identity/v3/projects => generated 311 bytes in 229 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 16:37:41.506293 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 507/1033] 10.222.0.20 () {56 vars in 1148 bytes} [Mon Aug 1 16:37:41 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:37:41.507314 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e39a244a-255c-4bfb-8cc8-2f36ee89dcab None admin] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:37:41.613853 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG dbcounter [-] [87058] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:37:41.624993 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e39a244a-255c-4bfb-8cc8-2f36ee89dcab None admin] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:37:41.665362 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0d9d3d3-a586-4787-a0dc-414a2bc09ff1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:37:41.665988 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d9d3d3-a586-4787-a0dc-414a2bc09ff1 None admin] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:37:41.666304 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d9d3d3-a586-4787-a0dc-414a2bc09ff1 None admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:37:41.666582 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d9d3d3-a586-4787-a0dc-414a2bc09ff1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:37:41.667380 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0d9d3d3-a586-4787-a0dc-414a2bc09ff1 None admin] RBAC: Adding query filter params (role.id=653d2ccc68e8415191e22643427e28bc, scope.project.id=7b992d6a65df4bf9b27c13192d57c5f0, user.id=6533106d03eb4ca185a8839336d2aff9) {{(pid=87058) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:37:41.667699 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0d9d3d3-a586-4787-a0dc-414a2bc09ff1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:37:41.669128 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0d9d3d3-a586-4787-a0dc-414a2bc09ff1 None admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:37:41.674911 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 527/1034] 10.222.0.20 () {56 vars in 1374 bytes} [Mon Aug 1 16:37:41 2022] GET /identity/v3/role_assignments?user.id=6533106d03eb4ca185a8839336d2aff9&scope.project.id=7b992d6a65df4bf9b27c13192d57c5f0&role.id=653d2ccc68e8415191e22643427e28bc => generated 259 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:37:41.760121 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f34e4e7-7bd9-4879-8322-08087190aa34 None None] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:37:41.760410 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f34e4e7-7bd9-4879-8322-08087190aa34 None None] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:37:41.760678 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f34e4e7-7bd9-4879-8322-08087190aa34 None None] PATH_INFO: `/` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:37:41.762208 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 508/1035] 10.222.0.20 () {54 vars in 926 bytes} [Mon Aug 1 16:37:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 16:37:41.767775 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00286a27-b61d-4b50-98f0-fa2b6def197f None None] REQUEST_METHOD: `POST` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:37:41.768065 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00286a27-b61d-4b50-98f0-fa2b6def197f None None] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:37:41.768331 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00286a27-b61d-4b50-98f0-fa2b6def197f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:37:41.787893 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.auth.core [None req-00286a27-b61d-4b50-98f0-fa2b6def197f None None] MFA Rules not processed for user `1007f8e379b74e169b98fb7c3c0ba5b4`. Rule list: `[]` (Enabled: `True`). {{(pid=87058) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:37:41.789114 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.fernet_utils [None req-00286a27-b61d-4b50-98f0-fa2b6def197f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:37:41.861818 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 528/1036] 10.222.0.20 () {58 vars in 1023 bytes} [Mon Aug 1 16:37:41 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 01 16:37:41.884587 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f34e4e7-7bd9-4879-8322-08087190aa34 None None] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:37:41.991673 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f34e4e7-7bd9-4879-8322-08087190aa34 None None] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:37:42.027959 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e3a18f-4d05-4a03-9dc6-586b9c6ddfcb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:37:42.028602 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e3a18f-4d05-4a03-9dc6-586b9c6ddfcb None admin] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:37:42.028875 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e3a18f-4d05-4a03-9dc6-586b9c6ddfcb None admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:37:42.029140 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e3a18f-4d05-4a03-9dc6-586b9c6ddfcb None admin] PATH_INFO: `/v3/services/image` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:37:42.029766 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0e3a18f-4d05-4a03-9dc6-586b9c6ddfcb None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:37:42.031154 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0e3a18f-4d05-4a03-9dc6-586b9c6ddfcb None admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:37:42.041016 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application [None req-e0e3a18f-4d05-4a03-9dc6-586b9c6ddfcb None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Aug 01 16:37:42.041016 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 16:37:42.041016 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 01 16:37:42.041016 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 01 16:37:42.041016 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 01 16:37:42.041016 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 01 16:37:42.041016 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 01 16:37:42.041016 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 01 16:37:42.041016 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 01 16:37:42.041016 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application Aug 01 16:37:42.041016 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 01 16:37:42.041016 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application Aug 01 16:37:42.041016 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 16:37:42.041016 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 16:37:42.041016 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 16:37:42.041016 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 16:37:42.041016 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 16:37:42.042089 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 16:37:42.042089 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 16:37:42.042089 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 16:37:42.042089 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 16:37:42.042089 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 16:37:42.042089 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 16:37:42.042089 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 01 16:37:42.042089 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 01 16:37:42.042089 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 01 16:37:42.042089 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 01 16:37:42.042089 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 16:37:42.042089 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 16:37:42.042089 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 16:37:42.042089 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 16:37:42.042089 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 16:37:42.042089 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 16:37:42.042990 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 16:37:42.042990 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application with Lock( Aug 01 16:37:42.042990 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 16:37:42.042990 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self._enter() Aug 01 16:37:42.042990 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 16:37:42.042990 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 16:37:42.042990 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 16:37:42.042990 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self.creator() Aug 01 16:37:42.042990 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 16:37:42.042990 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application created_value = creator( Aug 01 16:37:42.042990 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 01 16:37:42.042990 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 01 16:37:42.042990 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 01 16:37:42.042990 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application Aug 01 16:37:42.044533 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 509/1037] 10.222.0.20 () {56 vars in 1105 bytes} [Mon Aug 1 16:37:41 2022] GET /identity/v3/services/image => generated 86 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 01 16:37:42.048062 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00286a27-b61d-4b50-98f0-fa2b6def197f None None] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:37:42.131448 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG dbcounter [-] [87058] Writing DB stats keystone:SELECT=100 {{(pid=87058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:37:42.147650 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00286a27-b61d-4b50-98f0-fa2b6def197f None None] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:37:42.181310 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25e8ce72-4a3d-438c-af87-075fe29fdeb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:37:42.181931 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e8ce72-4a3d-438c-af87-075fe29fdeb6 None admin] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:37:42.182242 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e8ce72-4a3d-438c-af87-075fe29fdeb6 None admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:37:42.182510 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e8ce72-4a3d-438c-af87-075fe29fdeb6 None admin] PATH_INFO: `/v3/services` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:37:42.183181 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25e8ce72-4a3d-438c-af87-075fe29fdeb6 None admin] RBAC: Adding query filter params (name=image) {{(pid=87058) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:37:42.183480 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25e8ce72-4a3d-438c-af87-075fe29fdeb6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:37:42.184845 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25e8ce72-4a3d-438c-af87-075fe29fdeb6 None admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:37:42.189851 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 529/1038] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Aug 1 16:37:42 2022] GET /identity/v3/services?name=image => generated 122 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:37:42.195378 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e3a18f-4d05-4a03-9dc6-586b9c6ddfcb None admin] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:37:42.204737 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG dbcounter [-] [87057] Writing DB stats keystone:SELECT=100 {{(pid=87057) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:37:42.296037 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e3a18f-4d05-4a03-9dc6-586b9c6ddfcb None admin] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:37:42.330251 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a0732e2-b70f-4c72-9428-726afd480b0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:37:42.332074 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0732e2-b70f-4c72-9428-726afd480b0b None admin] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:37:42.332074 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0732e2-b70f-4c72-9428-726afd480b0b None admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:37:42.332074 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0732e2-b70f-4c72-9428-726afd480b0b None admin] PATH_INFO: `/v3/services` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:37:42.332074 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a0732e2-b70f-4c72-9428-726afd480b0b None admin] RBAC: Adding query filter params (type=image) {{(pid=87057) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:37:42.332383 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a0732e2-b70f-4c72-9428-726afd480b0b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:37:42.335691 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a0732e2-b70f-4c72-9428-726afd480b0b None admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:37:42.340367 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 510/1039] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Aug 1 16:37:42 2022] GET /identity/v3/services?type=image => generated 350 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:37:42.346058 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25e8ce72-4a3d-438c-af87-075fe29fdeb6 None admin] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:37:42.445733 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25e8ce72-4a3d-438c-af87-075fe29fdeb6 None admin] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:37:42.479529 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e77ebf-85e3-44dc-8971-61d4557b8273 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:37:42.480135 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e77ebf-85e3-44dc-8971-61d4557b8273 None admin] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:37:42.480413 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e77ebf-85e3-44dc-8971-61d4557b8273 None admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:37:42.480681 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e77ebf-85e3-44dc-8971-61d4557b8273 None admin] PATH_INFO: `/v3/endpoints` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:37:42.481418 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63e77ebf-85e3-44dc-8971-61d4557b8273 None admin] RBAC: Adding query filter params (interface=public, service_id=270d624e5b184bd4a4c621a976167a99, region_id=RegionOne) {{(pid=87058) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:37:42.481721 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63e77ebf-85e3-44dc-8971-61d4557b8273 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:37:42.483108 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63e77ebf-85e3-44dc-8971-61d4557b8273 None admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:37:42.488166 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 530/1040] 10.222.0.20 () {56 vars in 1251 bytes} [Mon Aug 1 16:37:42 2022] GET /identity/v3/endpoints?service_id=270d624e5b184bd4a4c621a976167a99&interface=public®ion_id=RegionOne => generated 194 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:37:42.493523 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a0732e2-b70f-4c72-9428-726afd480b0b None admin] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:37:42.565799 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG dbcounter [-] [87057] Writing DB stats keystone:SELECT=100 {{(pid=87057) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:37:42.596432 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a0732e2-b70f-4c72-9428-726afd480b0b None admin] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:37:42.631636 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bccbf87-5352-4de9-976d-b0261c5b9226 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:37:42.632258 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bccbf87-5352-4de9-976d-b0261c5b9226 None admin] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:37:42.632539 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bccbf87-5352-4de9-976d-b0261c5b9226 None admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:37:42.632825 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bccbf87-5352-4de9-976d-b0261c5b9226 None admin] PATH_INFO: `/v3/services` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:37:42.633470 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bccbf87-5352-4de9-976d-b0261c5b9226 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:37:42.634856 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bccbf87-5352-4de9-976d-b0261c5b9226 None admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:37:42.647899 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 511/1041] 10.222.0.20 () {56 vars in 1087 bytes} [Mon Aug 1 16:37:42 2022] GET /identity/v3/services => generated 1957 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 16:37:43.585952 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ee0c75-5f0b-4387-8f5f-5fc5a3ec00d3 None None] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:37:43.586500 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ee0c75-5f0b-4387-8f5f-5fc5a3ec00d3 None None] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:37:43.586580 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ee0c75-5f0b-4387-8f5f-5fc5a3ec00d3 None None] PATH_INFO: `/` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:37:43.587934 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 531/1042] 10.222.0.20 () {54 vars in 926 bytes} [Mon Aug 1 16:37:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 16:37:43.592986 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c5aefc-b353-4109-be62-31fb4d436bfe None None] REQUEST_METHOD: `POST` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:37:43.593384 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c5aefc-b353-4109-be62-31fb4d436bfe None None] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:37:43.593544 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c5aefc-b353-4109-be62-31fb4d436bfe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:37:43.611578 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.auth.core [None req-59c5aefc-b353-4109-be62-31fb4d436bfe None None] MFA Rules not processed for user `1007f8e379b74e169b98fb7c3c0ba5b4`. Rule list: `[]` (Enabled: `True`). {{(pid=87057) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:37:43.612770 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.fernet_utils [None req-59c5aefc-b353-4109-be62-31fb4d436bfe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87057) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:37:43.680909 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 512/1043] 10.222.0.20 () {58 vars in 1023 bytes} [Mon Aug 1 16:37:43 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 01 16:37:43.705228 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7ee0c75-5f0b-4387-8f5f-5fc5a3ec00d3 None None] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:37:43.710137 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e45a5c3a-86b0-4f0a-a095-f5ad6f127e4a None None] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:37:43.710429 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e45a5c3a-86b0-4f0a-a095-f5ad6f127e4a None None] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:37:43.710706 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e45a5c3a-86b0-4f0a-a095-f5ad6f127e4a None None] PATH_INFO: `/` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:37:43.712183 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 513/1044] 10.222.0.20 () {54 vars in 926 bytes} [Mon Aug 1 16:37:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 01 16:37:43.717017 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f86b00d-bf5b-4950-8ee6-407b6341213f None None] REQUEST_METHOD: `POST` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:37:43.717304 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f86b00d-bf5b-4950-8ee6-407b6341213f None None] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:37:43.717580 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f86b00d-bf5b-4950-8ee6-407b6341213f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:37:43.722166 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG dbcounter [-] [87058] Writing DB stats keystone:SELECT=100 {{(pid=87058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:37:43.737297 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.auth.core [None req-6f86b00d-bf5b-4950-8ee6-407b6341213f None None] MFA Rules not processed for user `1007f8e379b74e169b98fb7c3c0ba5b4`. Rule list: `[]` (Enabled: `True`). {{(pid=87057) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:37:43.738549 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.fernet_utils [None req-6f86b00d-bf5b-4950-8ee6-407b6341213f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87057) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:37:43.810076 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG dbcounter [-] [87057] Writing DB stats keystone:SELECT=100 {{(pid=87057) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:37:43.817746 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7ee0c75-5f0b-4387-8f5f-5fc5a3ec00d3 None None] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:37:43.819912 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 514/1045] 10.222.0.20 () {58 vars in 1023 bytes} [Mon Aug 1 16:37:43 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 01 16:37:43.842789 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f86b00d-bf5b-4950-8ee6-407b6341213f None None] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:37:43.854183 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8230e8ad-01c0-4b14-a3e5-531f72109a3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:37:43.854832 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8230e8ad-01c0-4b14-a3e5-531f72109a3a None admin] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:37:43.855109 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8230e8ad-01c0-4b14-a3e5-531f72109a3a None admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:37:43.855383 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8230e8ad-01c0-4b14-a3e5-531f72109a3a None admin] PATH_INFO: `/v3/domains/default` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:37:43.856527 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8230e8ad-01c0-4b14-a3e5-531f72109a3a None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:37:43.858112 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8230e8ad-01c0-4b14-a3e5-531f72109a3a None admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:37:43.859853 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 532/1046] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Aug 1 16:37:43 2022] GET /identity/v3/domains/default => generated 199 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:37:43.865912 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8230e8ad-01c0-4b14-a3e5-531f72109a3a None admin] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:37:43.959282 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f86b00d-bf5b-4950-8ee6-407b6341213f None None] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:37:43.961444 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG dbcounter [-] [87058] Writing DB stats keystone:SELECT=100 {{(pid=87058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:37:43.981414 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8230e8ad-01c0-4b14-a3e5-531f72109a3a None admin] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:37:43.995405 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91f0345b-6246-4cf2-b8f6-d57c18dcc1ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:37:43.996050 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f0345b-6246-4cf2-b8f6-d57c18dcc1ef None admin] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:37:43.996329 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f0345b-6246-4cf2-b8f6-d57c18dcc1ef None admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:37:43.996604 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f0345b-6246-4cf2-b8f6-d57c18dcc1ef None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:37:44.006621 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91f0345b-6246-4cf2-b8f6-d57c18dcc1ef None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 16:37:44.011467 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application [None req-91f0345b-6246-4cf2-b8f6-d57c18dcc1ef None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 01 16:37:44.011467 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 16:37:44.011467 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 16:37:44.011467 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 16:37:44.011467 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 16:37:44.011467 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 16:37:44.011467 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 16:37:44.011467 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 16:37:44.011467 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 16:37:44.011467 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 16:37:44.011467 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 16:37:44.011467 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 16:37:44.011467 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 16:37:44.011467 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 16:37:44.011467 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 16:37:44.011467 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 16:37:44.012391 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 16:37:44.012391 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 16:37:44.012391 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 16:37:44.012391 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 16:37:44.012391 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 16:37:44.012391 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 16:37:44.012391 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 16:37:44.012391 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 16:37:44.012391 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 16:37:44.012391 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 16:37:44.012391 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 16:37:44.012391 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application with Lock( Aug 01 16:37:44.012391 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 16:37:44.012391 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self._enter() Aug 01 16:37:44.012391 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 16:37:44.012391 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 16:37:44.013249 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 16:37:44.013249 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self.creator() Aug 01 16:37:44.013249 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 16:37:44.013249 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application created_value = creator( Aug 01 16:37:44.013249 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 16:37:44.013249 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 16:37:44.013249 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 16:37:44.013249 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 16:37:44.013249 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 16:37:44.013249 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 16:37:44.013249 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 01 16:37:44.013249 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application Aug 01 16:37:44.013249 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 515/1047] 10.222.0.20 () {56 vars in 1120 bytes} [Mon Aug 1 16:37:43 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 01 16:37:44.018865 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91f0345b-6246-4cf2-b8f6-d57c18dcc1ef None admin] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:37:44.021666 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-896ea939-4f92-4020-829c-89aed3457228 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:37:44.022400 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896ea939-4f92-4020-829c-89aed3457228 None admin] REQUEST_METHOD: `POST` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:37:44.022717 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896ea939-4f92-4020-829c-89aed3457228 None admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:37:44.023073 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896ea939-4f92-4020-829c-89aed3457228 None admin] PATH_INFO: `/v3/users` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:37:44.023763 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-896ea939-4f92-4020-829c-89aed3457228 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:37:44.025184 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-896ea939-4f92-4020-829c-89aed3457228 None admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:37:44.075400 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG dbcounter [-] [87057] Writing DB stats keystone:SELECT=100 {{(pid=87057) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:37:44.119554 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91f0345b-6246-4cf2-b8f6-d57c18dcc1ef None admin] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:37:44.146062 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 533/1048] 10.222.0.20 () {60 vars in 1132 bytes} [Mon Aug 1 16:37:43 2022] POST /identity/v3/users => generated 285 bytes in 270 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 16:37:44.154996 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd658767-6099-4dc3-8b7f-edafa01d0330 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:37:44.155690 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd658767-6099-4dc3-8b7f-edafa01d0330 None admin] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:37:44.156458 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd658767-6099-4dc3-8b7f-edafa01d0330 None admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:37:44.157102 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd658767-6099-4dc3-8b7f-edafa01d0330 None admin] PATH_INFO: `/v3/roles` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:37:44.158717 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd658767-6099-4dc3-8b7f-edafa01d0330 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=87057) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:37:44.159029 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd658767-6099-4dc3-8b7f-edafa01d0330 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:37:44.160460 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd658767-6099-4dc3-8b7f-edafa01d0330 None admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:37:44.175520 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 516/1049] 10.222.0.20 () {56 vars in 1115 bytes} [Mon Aug 1 16:37:44 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 16:37:44.177943 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-896ea939-4f92-4020-829c-89aed3457228 None admin] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:37:44.290405 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-896ea939-4f92-4020-829c-89aed3457228 None admin] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:37:44.327395 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05057e1c-05f0-4c11-ad82-19fc238b22f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:37:44.328048 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05057e1c-05f0-4c11-ad82-19fc238b22f5 None admin] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:37:44.328329 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05057e1c-05f0-4c11-ad82-19fc238b22f5 None admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:37:44.328604 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05057e1c-05f0-4c11-ad82-19fc238b22f5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:37:44.331381 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG dbcounter [-] [87058] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:37:44.341070 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05057e1c-05f0-4c11-ad82-19fc238b22f5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:37:44.342555 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05057e1c-05f0-4c11-ad82-19fc238b22f5 None admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:37:44.358498 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application [None req-05057e1c-05f0-4c11-ad82-19fc238b22f5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 16:37:44.358498 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 16:37:44.358498 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 16:37:44.358498 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 16:37:44.358498 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 16:37:44.358498 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 16:37:44.358498 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 16:37:44.358498 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 16:37:44.358498 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 16:37:44.358498 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 16:37:44.358498 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 16:37:44.358498 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 16:37:44.358498 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 16:37:44.358498 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 16:37:44.358498 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 16:37:44.359660 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 16:37:44.359660 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 16:37:44.359660 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 16:37:44.359660 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 16:37:44.359660 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 16:37:44.359660 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 16:37:44.359660 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 16:37:44.359660 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 16:37:44.359660 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application with Lock( Aug 01 16:37:44.359660 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 16:37:44.359660 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self._enter() Aug 01 16:37:44.359660 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 16:37:44.359660 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 16:37:44.359660 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 16:37:44.359660 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self.creator() Aug 01 16:37:44.359660 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 16:37:44.359660 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application created_value = creator( Aug 01 16:37:44.360770 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 16:37:44.360770 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 16:37:44.360770 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 16:37:44.360770 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application Aug 01 16:37:44.362647 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 534/1050] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Aug 1 16:37:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 01 16:37:44.372349 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd658767-6099-4dc3-8b7f-edafa01d0330 None admin] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:37:44.475761 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd658767-6099-4dc3-8b7f-edafa01d0330 None admin] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:37:44.492528 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG dbcounter [-] [87057] Writing DB stats keystone:SELECT=100 {{(pid=87057) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:37:44.510776 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f04adc-f59f-4132-b0db-142412e7688d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:37:44.511397 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f04adc-f59f-4132-b0db-142412e7688d None admin] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:37:44.511678 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f04adc-f59f-4132-b0db-142412e7688d None admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:37:44.511951 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f04adc-f59f-4132-b0db-142412e7688d None admin] PATH_INFO: `/v3/domains` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:37:44.512646 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05f04adc-f59f-4132-b0db-142412e7688d None admin] RBAC: Adding query filter params (name=Default) {{(pid=87057) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:37:44.513138 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05f04adc-f59f-4132-b0db-142412e7688d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:37:44.514537 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05f04adc-f59f-4132-b0db-142412e7688d None admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:37:44.527105 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 517/1051] 10.222.0.20 () {56 vars in 1109 bytes} [Mon Aug 1 16:37:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:37:44.532959 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05057e1c-05f0-4c11-ad82-19fc238b22f5 None admin] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:37:44.636514 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05057e1c-05f0-4c11-ad82-19fc238b22f5 None admin] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:37:44.667923 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42f5304-2878-4305-aee9-ac0309fba9fd None None] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:37:44.668443 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42f5304-2878-4305-aee9-ac0309fba9fd None None] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:37:44.668443 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42f5304-2878-4305-aee9-ac0309fba9fd None None] PATH_INFO: `/` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:37:44.669914 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 518/1052] 10.222.0.20 () {54 vars in 926 bytes} [Mon Aug 1 16:37:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 16:37:44.672174 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c2f7ec-536e-43a7-abf5-11e0d6a0e5b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:37:44.672819 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c2f7ec-536e-43a7-abf5-11e0d6a0e5b5 None admin] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:37:44.673919 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c2f7ec-536e-43a7-abf5-11e0d6a0e5b5 None admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:37:44.673919 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c2f7ec-536e-43a7-abf5-11e0d6a0e5b5 None admin] PATH_INFO: `/v3/users/nova` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:37:44.675358 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c14b86-173d-41fe-8990-11454d495890 None None] REQUEST_METHOD: `POST` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:37:44.675632 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c14b86-173d-41fe-8990-11454d495890 None None] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:37:44.675926 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c14b86-173d-41fe-8990-11454d495890 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:37:44.682643 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74c2f7ec-536e-43a7-abf5-11e0d6a0e5b5 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:37:44.684089 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74c2f7ec-536e-43a7-abf5-11e0d6a0e5b5 None admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:37:44.694799 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.auth.core [None req-71c14b86-173d-41fe-8990-11454d495890 None None] MFA Rules not processed for user `1007f8e379b74e169b98fb7c3c0ba5b4`. Rule list: `[]` (Enabled: `True`). {{(pid=87057) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:37:44.695662 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application [None req-74c2f7ec-536e-43a7-abf5-11e0d6a0e5b5 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 01 16:37:44.695662 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 16:37:44.695662 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 16:37:44.695662 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 16:37:44.695662 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 16:37:44.695662 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 16:37:44.695662 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 16:37:44.695662 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 16:37:44.695662 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 16:37:44.695662 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 16:37:44.695662 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 16:37:44.695662 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 16:37:44.695662 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 01 16:37:44.695662 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 01 16:37:44.695662 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 01 16:37:44.695662 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 01 16:37:44.696584 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 16:37:44.696584 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 16:37:44.696584 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 01 16:37:44.696584 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 16:37:44.696584 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 01 16:37:44.696584 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 16:37:44.696584 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 16:37:44.696584 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 16:37:44.696584 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 16:37:44.696584 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 16:37:44.696584 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 16:37:44.696584 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application with Lock( Aug 01 16:37:44.696584 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 16:37:44.696584 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self._enter() Aug 01 16:37:44.696584 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 16:37:44.696584 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 16:37:44.696584 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 16:37:44.696584 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self.creator() Aug 01 16:37:44.702652 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 16:37:44.702652 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application created_value = creator( Aug 01 16:37:44.702652 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 01 16:37:44.702652 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 01 16:37:44.702652 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 01 16:37:44.702652 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 01 16:37:44.702652 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 01 16:37:44.702652 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 01 16:37:44.702652 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 01 16:37:44.702652 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application Aug 01 16:37:44.703213 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.fernet_utils [None req-71c14b86-173d-41fe-8990-11454d495890 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87057) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:37:44.703298 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 535/1053] 10.222.0.20 () {56 vars in 1093 bytes} [Mon Aug 1 16:37:44 2022] GET /identity/v3/users/nova => generated 82 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 16:37:44.703298 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c2f7ec-536e-43a7-abf5-11e0d6a0e5b5 None admin] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:37:44.755894 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG dbcounter [-] [87058] Writing DB stats keystone:SELECT=100 {{(pid=87058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:37:44.771512 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 519/1054] 10.222.0.20 () {58 vars in 1023 bytes} [Mon Aug 1 16:37:44 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 01 16:37:44.793783 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c14b86-173d-41fe-8990-11454d495890 None None] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:37:44.809194 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c2f7ec-536e-43a7-abf5-11e0d6a0e5b5 None admin] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:37:44.843723 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e0ed06-4e41-4e03-a491-f72eb4455718 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:37:44.844373 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e0ed06-4e41-4e03-a491-f72eb4455718 None admin] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:37:44.844651 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e0ed06-4e41-4e03-a491-f72eb4455718 None admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:37:44.844920 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e0ed06-4e41-4e03-a491-f72eb4455718 None admin] PATH_INFO: `/v3/users` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:37:44.845718 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8e0ed06-4e41-4e03-a491-f72eb4455718 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=87058) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:37:44.846022 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8e0ed06-4e41-4e03-a491-f72eb4455718 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:37:44.847467 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8e0ed06-4e41-4e03-a491-f72eb4455718 None admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:37:44.871093 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 536/1055] 10.222.0.20 () {56 vars in 1133 bytes} [Mon Aug 1 16:37:44 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 16:37:44.872215 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e0ed06-4e41-4e03-a491-f72eb4455718 None admin] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:37:44.897899 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c14b86-173d-41fe-8990-11454d495890 None None] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:37:44.909922 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG dbcounter [-] [87057] Writing DB stats keystone:SELECT=100 {{(pid=87057) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:37:44.939544 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceca3547-c50b-4561-8962-5950484b4d9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:37:44.940196 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceca3547-c50b-4561-8962-5950484b4d9c None admin] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:37:44.940478 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceca3547-c50b-4561-8962-5950484b4d9c None admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:37:44.940749 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceca3547-c50b-4561-8962-5950484b4d9c None admin] PATH_INFO: `/v3/services/image` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:37:44.941382 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceca3547-c50b-4561-8962-5950484b4d9c None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:37:44.942797 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceca3547-c50b-4561-8962-5950484b4d9c None admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:37:44.952352 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application [None req-ceca3547-c50b-4561-8962-5950484b4d9c None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Aug 01 16:37:44.952352 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 16:37:44.952352 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 01 16:37:44.952352 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 01 16:37:44.952352 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 01 16:37:44.952352 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 01 16:37:44.952352 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 01 16:37:44.952352 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 01 16:37:44.952352 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 01 16:37:44.952352 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application Aug 01 16:37:44.952352 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 01 16:37:44.952352 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application Aug 01 16:37:44.952352 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 16:37:44.952352 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 16:37:44.952352 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 16:37:44.952352 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 16:37:44.952352 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 16:37:44.953571 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 16:37:44.953571 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 16:37:44.953571 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 16:37:44.953571 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 16:37:44.953571 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 16:37:44.953571 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 16:37:44.953571 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 01 16:37:44.953571 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 01 16:37:44.953571 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 01 16:37:44.953571 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 01 16:37:44.953571 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 16:37:44.953571 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 16:37:44.953571 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 16:37:44.953571 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 16:37:44.953571 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 16:37:44.953571 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 16:37:44.954663 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 16:37:44.954663 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application with Lock( Aug 01 16:37:44.954663 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 16:37:44.954663 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self._enter() Aug 01 16:37:44.954663 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 16:37:44.954663 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 16:37:44.954663 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 16:37:44.954663 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self.creator() Aug 01 16:37:44.954663 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 16:37:44.954663 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application created_value = creator( Aug 01 16:37:44.954663 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 01 16:37:44.954663 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 01 16:37:44.954663 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 01 16:37:44.954663 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application Aug 01 16:37:44.954663 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 520/1056] 10.222.0.20 () {56 vars in 1105 bytes} [Mon Aug 1 16:37:44 2022] GET /identity/v3/services/image => generated 86 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 01 16:37:44.960188 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceca3547-c50b-4561-8962-5950484b4d9c None admin] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:37:44.994104 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e0ed06-4e41-4e03-a491-f72eb4455718 None admin] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:37:45.002535 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG dbcounter [-] [87058] Writing DB stats keystone:SELECT=100 {{(pid=87058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:37:45.031189 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac29c4ae-5382-40d7-8fb7-3bc11fa8c12d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:37:45.031984 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac29c4ae-5382-40d7-8fb7-3bc11fa8c12d None admin] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:37:45.032259 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac29c4ae-5382-40d7-8fb7-3bc11fa8c12d None admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:37:45.032648 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac29c4ae-5382-40d7-8fb7-3bc11fa8c12d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:37:45.044678 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac29c4ae-5382-40d7-8fb7-3bc11fa8c12d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:37:45.046260 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac29c4ae-5382-40d7-8fb7-3bc11fa8c12d None admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:37:45.063313 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application [None req-ac29c4ae-5382-40d7-8fb7-3bc11fa8c12d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 16:37:45.063313 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 16:37:45.063313 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 16:37:45.063313 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 16:37:45.063313 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 16:37:45.063313 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 16:37:45.063313 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 16:37:45.063313 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 16:37:45.063313 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 16:37:45.063313 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 16:37:45.063313 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 16:37:45.063313 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 16:37:45.063313 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 16:37:45.063313 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 16:37:45.063313 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 16:37:45.064251 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 16:37:45.064251 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 16:37:45.064251 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 16:37:45.064251 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 16:37:45.064251 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 16:37:45.064251 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 16:37:45.064251 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 16:37:45.064251 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 16:37:45.064251 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application with Lock( Aug 01 16:37:45.064251 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 16:37:45.064251 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self._enter() Aug 01 16:37:45.064251 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 16:37:45.064251 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 16:37:45.064251 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 16:37:45.064251 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self.creator() Aug 01 16:37:45.064251 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 16:37:45.064251 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application created_value = creator( Aug 01 16:37:45.065170 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 16:37:45.065170 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 16:37:45.065170 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 16:37:45.065170 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application Aug 01 16:37:45.065170 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 537/1057] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Aug 1 16:37:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 16:37:45.069195 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceca3547-c50b-4561-8962-5950484b4d9c None admin] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:37:45.070467 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac29c4ae-5382-40d7-8fb7-3bc11fa8c12d None admin] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:37:45.109790 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afedcb43-dd3e-4c19-8026-7a7199bdcac2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:37:45.110436 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afedcb43-dd3e-4c19-8026-7a7199bdcac2 None admin] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:37:45.110947 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afedcb43-dd3e-4c19-8026-7a7199bdcac2 None admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:37:45.110947 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afedcb43-dd3e-4c19-8026-7a7199bdcac2 None admin] PATH_INFO: `/v3/services` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:37:45.111644 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afedcb43-dd3e-4c19-8026-7a7199bdcac2 None admin] RBAC: Adding query filter params (name=image) {{(pid=87057) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:37:45.111947 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afedcb43-dd3e-4c19-8026-7a7199bdcac2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:37:45.113306 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afedcb43-dd3e-4c19-8026-7a7199bdcac2 None admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:37:45.122795 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 521/1058] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Aug 1 16:37:44 2022] GET /identity/v3/services?name=image => generated 122 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 16:37:45.123808 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afedcb43-dd3e-4c19-8026-7a7199bdcac2 None admin] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:37:45.162056 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG dbcounter [-] [87057] Writing DB stats keystone:SELECT=100 {{(pid=87057) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:37:45.181005 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac29c4ae-5382-40d7-8fb7-3bc11fa8c12d None admin] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:37:45.215517 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42df6e4c-e950-4aa6-8f50-fed580b8d128 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:37:45.216208 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42df6e4c-e950-4aa6-8f50-fed580b8d128 None admin] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:37:45.216392 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42df6e4c-e950-4aa6-8f50-fed580b8d128 None admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:37:45.216663 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42df6e4c-e950-4aa6-8f50-fed580b8d128 None admin] PATH_INFO: `/v3/domains` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:37:45.217347 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42df6e4c-e950-4aa6-8f50-fed580b8d128 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87058) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:37:45.217643 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42df6e4c-e950-4aa6-8f50-fed580b8d128 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:37:45.219037 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42df6e4c-e950-4aa6-8f50-fed580b8d128 None admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:37:45.235081 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afedcb43-dd3e-4c19-8026-7a7199bdcac2 None admin] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:37:45.236853 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 538/1059] 10.222.0.20 () {56 vars in 1109 bytes} [Mon Aug 1 16:37:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:37:45.237715 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42df6e4c-e950-4aa6-8f50-fed580b8d128 None admin] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:37:45.257942 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG dbcounter [-] [87058] Writing DB stats keystone:SELECT=100 {{(pid=87058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:37:45.275167 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1fcb214-15b7-4dc4-b0f9-d10ebab0659b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:37:45.275790 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1fcb214-15b7-4dc4-b0f9-d10ebab0659b None admin] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:37:45.276070 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1fcb214-15b7-4dc4-b0f9-d10ebab0659b None admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:37:45.276345 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1fcb214-15b7-4dc4-b0f9-d10ebab0659b None admin] PATH_INFO: `/v3/services` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:37:45.277021 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1fcb214-15b7-4dc4-b0f9-d10ebab0659b None admin] RBAC: Adding query filter params (type=image) {{(pid=87057) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:37:45.277326 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1fcb214-15b7-4dc4-b0f9-d10ebab0659b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:37:45.278717 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1fcb214-15b7-4dc4-b0f9-d10ebab0659b None admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:37:45.286077 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 522/1060] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Aug 1 16:37:45 2022] GET /identity/v3/services?type=image => generated 350 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 16:37:45.289784 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1fcb214-15b7-4dc4-b0f9-d10ebab0659b None admin] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:37:45.351100 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42df6e4c-e950-4aa6-8f50-fed580b8d128 None admin] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:37:45.389358 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60dcd89b-f5ef-40dc-8ac3-45b011efefae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:37:45.390297 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60dcd89b-f5ef-40dc-8ac3-45b011efefae None admin] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:37:45.390585 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60dcd89b-f5ef-40dc-8ac3-45b011efefae None admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:37:45.390854 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60dcd89b-f5ef-40dc-8ac3-45b011efefae None admin] PATH_INFO: `/v3/projects/service` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:37:45.394125 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG dbcounter [-] [87057] Writing DB stats keystone:SELECT=100 {{(pid=87057) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:37:45.397501 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1fcb214-15b7-4dc4-b0f9-d10ebab0659b None admin] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:37:45.398770 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60dcd89b-f5ef-40dc-8ac3-45b011efefae None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:37:45.400232 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60dcd89b-f5ef-40dc-8ac3-45b011efefae None admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:37:45.412548 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application [None req-60dcd89b-f5ef-40dc-8ac3-45b011efefae None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 16:37:45.412548 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 16:37:45.412548 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 16:37:45.412548 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 16:37:45.412548 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 16:37:45.412548 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 16:37:45.412548 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 16:37:45.412548 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 16:37:45.412548 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 16:37:45.412548 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 16:37:45.412548 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 16:37:45.412548 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 16:37:45.412548 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 01 16:37:45.412548 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 01 16:37:45.412548 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 01 16:37:45.413482 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 01 16:37:45.413482 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 16:37:45.413482 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 16:37:45.413482 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 16:37:45.413482 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 16:37:45.413482 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 16:37:45.413482 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 16:37:45.413482 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 16:37:45.413482 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application with Lock( Aug 01 16:37:45.413482 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 16:37:45.413482 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self._enter() Aug 01 16:37:45.413482 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 16:37:45.413482 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 16:37:45.413482 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 16:37:45.413482 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self.creator() Aug 01 16:37:45.413482 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 16:37:45.413482 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application created_value = creator( Aug 01 16:37:45.414448 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 01 16:37:45.414448 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 01 16:37:45.414448 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 01 16:37:45.414448 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 01 16:37:45.414448 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 01 16:37:45.414448 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 01 16:37:45.414448 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 16:37:45.414448 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application Aug 01 16:37:45.414448 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 539/1061] 10.222.0.20 () {56 vars in 1111 bytes} [Mon Aug 1 16:37:45 2022] GET /identity/v3/projects/service => generated 88 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 16:37:45.420957 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60dcd89b-f5ef-40dc-8ac3-45b011efefae None admin] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:37:45.444856 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f602c2-3e09-4f68-9d0c-1977f1503dc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:37:45.445481 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f602c2-3e09-4f68-9d0c-1977f1503dc9 None admin] REQUEST_METHOD: `POST` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:37:45.445744 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f602c2-3e09-4f68-9d0c-1977f1503dc9 None admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:37:45.446013 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f602c2-3e09-4f68-9d0c-1977f1503dc9 None admin] PATH_INFO: `/v3/endpoints` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:37:45.446757 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10f602c2-3e09-4f68-9d0c-1977f1503dc9 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:37:45.448132 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10f602c2-3e09-4f68-9d0c-1977f1503dc9 None admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:37:45.501339 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG dbcounter [-] [87058] Writing DB stats keystone:SELECT=100 {{(pid=87058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:37:45.524688 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60dcd89b-f5ef-40dc-8ac3-45b011efefae None admin] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:37:45.529378 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 523/1062] 10.222.0.20 () {60 vars in 1144 bytes} [Mon Aug 1 16:37:45 2022] POST /identity/v3/endpoints => generated 327 bytes in 240 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 16:37:45.569350 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b792db2-27eb-4596-96c2-7af73e147807 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:37:45.569975 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b792db2-27eb-4596-96c2-7af73e147807 None admin] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:37:45.570278 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b792db2-27eb-4596-96c2-7af73e147807 None admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:37:45.570546 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b792db2-27eb-4596-96c2-7af73e147807 None admin] PATH_INFO: `/v3/projects` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:37:45.571289 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b792db2-27eb-4596-96c2-7af73e147807 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87058) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:37:45.571590 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b792db2-27eb-4596-96c2-7af73e147807 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:37:45.573011 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b792db2-27eb-4596-96c2-7af73e147807 None admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:37:45.592426 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 540/1063] 10.222.0.20 () {56 vars in 1148 bytes} [Mon Aug 1 16:37:45 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 16:37:45.594235 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f602c2-3e09-4f68-9d0c-1977f1503dc9 None admin] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:37:45.706212 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f602c2-3e09-4f68-9d0c-1977f1503dc9 None admin] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:37:45.741290 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb1dc3d9-5879-4b3a-80da-0d8996a29594 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:37:45.741978 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1dc3d9-5879-4b3a-80da-0d8996a29594 None admin] REQUEST_METHOD: `PUT` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:37:45.742289 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1dc3d9-5879-4b3a-80da-0d8996a29594 None admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:37:45.742573 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1dc3d9-5879-4b3a-80da-0d8996a29594 None admin] PATH_INFO: `/v3/projects/7b992d6a65df4bf9b27c13192d57c5f0/users/6533106d03eb4ca185a8839336d2aff9/roles/653d2ccc68e8415191e22643427e28bc` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:37:45.744781 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb1dc3d9-5879-4b3a-80da-0d8996a29594 None admin] RBAC: Authorizing `identity:create_grant(project_id=7b992d6a65df4bf9b27c13192d57c5f0, user_id=6533106d03eb4ca185a8839336d2aff9, role_id=653d2ccc68e8415191e22643427e28bc)` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:37:45.746268 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb1dc3d9-5879-4b3a-80da-0d8996a29594 None admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:37:45.837590 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 524/1064] 10.222.0.20 () {58 vars in 1439 bytes} [Mon Aug 1 16:37:45 2022] PUT /identity/v3/projects/7b992d6a65df4bf9b27c13192d57c5f0/users/6533106d03eb4ca185a8839336d2aff9/roles/653d2ccc68e8415191e22643427e28bc => generated 0 bytes in 244 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 16:37:46.245673 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059d57cf-0c75-4bf4-94cf-06ed17bc1a54 None None] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:37:46.246233 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059d57cf-0c75-4bf4-94cf-06ed17bc1a54 None None] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:37:46.246233 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059d57cf-0c75-4bf4-94cf-06ed17bc1a54 None None] PATH_INFO: `/` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:37:46.247781 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 541/1065] 10.222.0.20 () {54 vars in 926 bytes} [Mon Aug 1 16:37:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 16:37:46.252816 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d934698a-dcc9-4b8c-9e89-5692ea2a16b6 None None] REQUEST_METHOD: `POST` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:37:46.253106 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d934698a-dcc9-4b8c-9e89-5692ea2a16b6 None None] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:37:46.253383 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d934698a-dcc9-4b8c-9e89-5692ea2a16b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:37:46.273304 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.auth.core [None req-d934698a-dcc9-4b8c-9e89-5692ea2a16b6 None None] MFA Rules not processed for user `1007f8e379b74e169b98fb7c3c0ba5b4`. Rule list: `[]` (Enabled: `True`). {{(pid=87057) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:37:46.274518 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.fernet_utils [None req-d934698a-dcc9-4b8c-9e89-5692ea2a16b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87057) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:37:46.284090 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG dbcounter [-] [87057] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=87057) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:37:46.345041 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 525/1066] 10.222.0.20 () {58 vars in 1023 bytes} [Mon Aug 1 16:37:46 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 01 16:37:46.368367 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-059d57cf-0c75-4bf4-94cf-06ed17bc1a54 None None] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:37:46.472588 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-059d57cf-0c75-4bf4-94cf-06ed17bc1a54 None None] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:37:46.507759 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd33f0a-51a9-4bd7-ae57-b3ec9eb45d7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:37:46.508330 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd33f0a-51a9-4bd7-ae57-b3ec9eb45d7b None admin] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:37:46.508608 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd33f0a-51a9-4bd7-ae57-b3ec9eb45d7b None admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:37:46.508881 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd33f0a-51a9-4bd7-ae57-b3ec9eb45d7b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:37:46.510811 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG dbcounter [-] [87058] Writing DB stats keystone:SELECT=100 {{(pid=87058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:37:46.520739 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cd33f0a-51a9-4bd7-ae57-b3ec9eb45d7b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 16:37:46.527077 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application [None req-2cd33f0a-51a9-4bd7-ae57-b3ec9eb45d7b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 16:37:46.527077 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 16:37:46.527077 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 16:37:46.527077 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 16:37:46.527077 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 16:37:46.527077 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 16:37:46.527077 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 16:37:46.527077 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 16:37:46.527077 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 16:37:46.527077 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 16:37:46.527077 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 16:37:46.527077 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 16:37:46.527077 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 16:37:46.527077 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 16:37:46.527077 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 16:37:46.527077 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 16:37:46.527991 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 16:37:46.527991 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 16:37:46.527991 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 16:37:46.527991 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 16:37:46.527991 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 16:37:46.527991 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 16:37:46.527991 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 16:37:46.527991 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 16:37:46.527991 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 16:37:46.527991 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 16:37:46.527991 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 16:37:46.527991 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application with Lock( Aug 01 16:37:46.527991 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 16:37:46.527991 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self._enter() Aug 01 16:37:46.527991 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 16:37:46.527991 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 16:37:46.528894 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 16:37:46.528894 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self.creator() Aug 01 16:37:46.528894 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 16:37:46.528894 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application created_value = creator( Aug 01 16:37:46.528894 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 16:37:46.528894 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 16:37:46.528894 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 16:37:46.528894 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 16:37:46.528894 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 16:37:46.528894 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 16:37:46.528894 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 16:37:46.528894 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application Aug 01 16:37:46.530145 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 542/1067] 10.222.0.20 () {56 vars in 1096 bytes} [Mon Aug 1 16:37:46 2022] GET /identity/v3/roles/admin => generated 83 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 16:37:46.533625 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d934698a-dcc9-4b8c-9e89-5692ea2a16b6 None None] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:37:46.633722 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d934698a-dcc9-4b8c-9e89-5692ea2a16b6 None None] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:37:46.669853 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-685fe14a-10a1-4380-9fdd-a17343fa9b32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:37:46.670510 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685fe14a-10a1-4380-9fdd-a17343fa9b32 None admin] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:37:46.670789 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685fe14a-10a1-4380-9fdd-a17343fa9b32 None admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:37:46.671057 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685fe14a-10a1-4380-9fdd-a17343fa9b32 None admin] PATH_INFO: `/v3/roles` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:37:46.671788 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-685fe14a-10a1-4380-9fdd-a17343fa9b32 None admin] RBAC: Adding query filter params (name=admin) {{(pid=87057) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:37:46.672084 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-685fe14a-10a1-4380-9fdd-a17343fa9b32 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:37:46.673491 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-685fe14a-10a1-4380-9fdd-a17343fa9b32 None admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:37:46.687427 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 526/1068] 10.222.0.20 () {56 vars in 1099 bytes} [Mon Aug 1 16:37:46 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 16:37:46.689153 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd33f0a-51a9-4bd7-ae57-b3ec9eb45d7b None admin] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:37:46.793660 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd33f0a-51a9-4bd7-ae57-b3ec9eb45d7b None admin] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:37:46.827963 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b1b3f10-fd7d-413b-ad78-8a7841c7354a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:37:46.828602 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1b3f10-fd7d-413b-ad78-8a7841c7354a None admin] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:37:46.828889 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1b3f10-fd7d-413b-ad78-8a7841c7354a None admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:37:46.829164 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1b3f10-fd7d-413b-ad78-8a7841c7354a None admin] PATH_INFO: `/v3/users/909657aaff6a45ab9e781f13f918c6d4` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:37:46.843085 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b1b3f10-fd7d-413b-ad78-8a7841c7354a None admin] RBAC: Authorizing `identity:get_user(user_id=909657aaff6a45ab9e781f13f918c6d4)` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:37:46.844529 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b1b3f10-fd7d-413b-ad78-8a7841c7354a None admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:37:46.850907 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 543/1069] 10.222.0.20 () {56 vars in 1177 bytes} [Mon Aug 1 16:37:46 2022] GET /identity/v3/users/909657aaff6a45ab9e781f13f918c6d4 => generated 285 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:37:46.852032 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-685fe14a-10a1-4380-9fdd-a17343fa9b32 None admin] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:37:46.863603 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG dbcounter [-] [87057] Writing DB stats keystone:SELECT=100 {{(pid=87057) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:37:46.965065 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-685fe14a-10a1-4380-9fdd-a17343fa9b32 None admin] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:37:46.999418 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d6c980-cd12-4aaa-916f-26dbc30c8270 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:37:47.000032 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d6c980-cd12-4aaa-916f-26dbc30c8270 None admin] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:37:47.000311 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d6c980-cd12-4aaa-916f-26dbc30c8270 None admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:37:47.000641 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d6c980-cd12-4aaa-916f-26dbc30c8270 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:37:47.070107 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6d6c980-cd12-4aaa-916f-26dbc30c8270 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:37:47.071483 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6d6c980-cd12-4aaa-916f-26dbc30c8270 None admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:37:47.079421 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG dbcounter [-] [87057] Writing DB stats keystone:SELECT=100 {{(pid=87057) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:37:47.175599 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 527/1070] 10.222.0.20 () {58 vars in 1282 bytes} [Mon Aug 1 16:37:46 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 01 16:37:47.177209 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b1b3f10-fd7d-413b-ad78-8a7841c7354a None admin] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:37:47.231709 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG dbcounter [-] [87058] Writing DB stats keystone:SELECT=100 {{(pid=87058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:37:47.280461 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b1b3f10-fd7d-413b-ad78-8a7841c7354a None admin] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:37:47.320841 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95247ac8-27e4-43bd-8865-1ddb92777fb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:37:47.321824 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95247ac8-27e4-43bd-8865-1ddb92777fb8 None admin] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:37:47.322343 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95247ac8-27e4-43bd-8865-1ddb92777fb8 None admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:37:47.322818 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95247ac8-27e4-43bd-8865-1ddb92777fb8 None admin] PATH_INFO: `/v3/projects/585c4683da3243eea2b4695c50946bdd` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:37:47.324851 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95247ac8-27e4-43bd-8865-1ddb92777fb8 None admin] RBAC: Authorizing `identity:get_project(project_id=585c4683da3243eea2b4695c50946bdd)` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:37:47.327482 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95247ac8-27e4-43bd-8865-1ddb92777fb8 None admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:37:47.336055 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 544/1071] 10.222.0.20 () {56 vars in 1186 bytes} [Mon Aug 1 16:37:47 2022] GET /identity/v3/projects/585c4683da3243eea2b4695c50946bdd => generated 311 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 16:37:47.337339 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d6c980-cd12-4aaa-916f-26dbc30c8270 None admin] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:37:47.439599 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d6c980-cd12-4aaa-916f-26dbc30c8270 None admin] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:37:47.447272 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG dbcounter [-] [87057] Writing DB stats keystone:SELECT=100 {{(pid=87057) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:37:47.474272 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb09a5e8-b739-4eaf-94f4-7edbcdcc102b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:37:47.474891 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb09a5e8-b739-4eaf-94f4-7edbcdcc102b None admin] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:37:47.475185 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb09a5e8-b739-4eaf-94f4-7edbcdcc102b None admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:37:47.475454 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb09a5e8-b739-4eaf-94f4-7edbcdcc102b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:37:47.476627 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb09a5e8-b739-4eaf-94f4-7edbcdcc102b None admin] RBAC: Adding query filter params (role.id=440bb943dd5b46fea038b903b6eaaf81, scope.project.id=585c4683da3243eea2b4695c50946bdd, user.id=909657aaff6a45ab9e781f13f918c6d4) {{(pid=87057) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:37:47.477200 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb09a5e8-b739-4eaf-94f4-7edbcdcc102b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:37:47.479347 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb09a5e8-b739-4eaf-94f4-7edbcdcc102b None admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:37:47.494067 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 528/1072] 10.222.0.20 () {56 vars in 1374 bytes} [Mon Aug 1 16:37:47 2022] GET /identity/v3/role_assignments?user.id=909657aaff6a45ab9e781f13f918c6d4&scope.project.id=585c4683da3243eea2b4695c50946bdd&role.id=440bb943dd5b46fea038b903b6eaaf81 => generated 259 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:37:47.582848 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-827f9242-e434-4c29-bce6-222a1ecf86eb None None] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:37:47.583030 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-827f9242-e434-4c29-bce6-222a1ecf86eb None None] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:37:47.583328 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-827f9242-e434-4c29-bce6-222a1ecf86eb None None] PATH_INFO: `/` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:37:47.584815 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 545/1073] 10.222.0.20 () {54 vars in 926 bytes} [Mon Aug 1 16:37:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 16:37:47.590173 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46da5cb8-948f-464c-b409-005566a32cfa None None] REQUEST_METHOD: `POST` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:37:47.590458 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46da5cb8-948f-464c-b409-005566a32cfa None None] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:37:47.590731 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46da5cb8-948f-464c-b409-005566a32cfa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:37:47.610558 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.auth.core [None req-46da5cb8-948f-464c-b409-005566a32cfa None None] MFA Rules not processed for user `1007f8e379b74e169b98fb7c3c0ba5b4`. Rule list: `[]` (Enabled: `True`). {{(pid=87057) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:37:47.611787 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.fernet_utils [None req-46da5cb8-948f-464c-b409-005566a32cfa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87057) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:37:47.684386 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 529/1074] 10.222.0.20 () {58 vars in 1023 bytes} [Mon Aug 1 16:37:47 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 01 16:37:47.706579 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-827f9242-e434-4c29-bce6-222a1ecf86eb None None] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:37:47.810494 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-827f9242-e434-4c29-bce6-222a1ecf86eb None None] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:37:47.830942 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG dbcounter [-] [87058] Writing DB stats keystone:SELECT=100 {{(pid=87058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:37:47.845409 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f72fa3f3-72df-4b7c-b1b2-80cdf6ee0330 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:37:47.845962 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72fa3f3-72df-4b7c-b1b2-80cdf6ee0330 None admin] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:37:47.846264 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72fa3f3-72df-4b7c-b1b2-80cdf6ee0330 None admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:37:47.846531 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72fa3f3-72df-4b7c-b1b2-80cdf6ee0330 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:37:47.868176 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ceacd9c-607e-4eaf-be5f-08b6e05d7d35 None None] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:37:47.868682 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ceacd9c-607e-4eaf-be5f-08b6e05d7d35 None None] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:37:47.868773 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ceacd9c-607e-4eaf-be5f-08b6e05d7d35 None None] PATH_INFO: `/` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:37:47.870258 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 530/1075] 10.222.0.20 () {54 vars in 926 bytes} [Mon Aug 1 16:37:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 16:37:47.875233 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5717b231-9834-4648-925d-ebfe86fedda1 None None] REQUEST_METHOD: `POST` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:37:47.875529 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5717b231-9834-4648-925d-ebfe86fedda1 None None] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:37:47.875812 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5717b231-9834-4648-925d-ebfe86fedda1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:37:47.895489 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.auth.core [None req-5717b231-9834-4648-925d-ebfe86fedda1 None None] MFA Rules not processed for user `1007f8e379b74e169b98fb7c3c0ba5b4`. Rule list: `[]` (Enabled: `True`). {{(pid=87057) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:37:47.896696 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.fernet_utils [None req-5717b231-9834-4648-925d-ebfe86fedda1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87057) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:37:47.928501 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f72fa3f3-72df-4b7c-b1b2-80cdf6ee0330 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:37:47.929952 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f72fa3f3-72df-4b7c-b1b2-80cdf6ee0330 None admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:37:47.971240 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 531/1076] 10.222.0.20 () {58 vars in 1023 bytes} [Mon Aug 1 16:37:47 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 01 16:37:47.994369 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5717b231-9834-4648-925d-ebfe86fedda1 None None] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:37:48.027244 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG dbcounter [-] [87057] Writing DB stats keystone:SELECT=100 {{(pid=87057) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:37:48.037720 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 546/1077] 10.222.0.20 () {58 vars in 1282 bytes} [Mon Aug 1 16:37:47 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 332 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 01 16:37:48.043599 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f72fa3f3-72df-4b7c-b1b2-80cdf6ee0330 None admin] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:37:48.067612 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG dbcounter [-] [87058] Writing DB stats keystone:SELECT=100 {{(pid=87058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:37:48.107513 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5717b231-9834-4648-925d-ebfe86fedda1 None None] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:37:48.142312 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-195502cb-09f1-49be-a61b-2164438f2861 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:37:48.143014 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195502cb-09f1-49be-a61b-2164438f2861 None admin] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:37:48.143297 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195502cb-09f1-49be-a61b-2164438f2861 None admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:37:48.143568 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195502cb-09f1-49be-a61b-2164438f2861 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:37:48.149868 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f72fa3f3-72df-4b7c-b1b2-80cdf6ee0330 None admin] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:37:48.154669 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-195502cb-09f1-49be-a61b-2164438f2861 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 16:37:48.159598 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application [None req-195502cb-09f1-49be-a61b-2164438f2861 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 01 16:37:48.159598 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 16:37:48.159598 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 16:37:48.159598 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 16:37:48.159598 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 16:37:48.159598 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 16:37:48.159598 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 16:37:48.159598 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 16:37:48.159598 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 16:37:48.159598 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 16:37:48.159598 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 16:37:48.159598 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 16:37:48.159598 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 16:37:48.159598 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 16:37:48.159598 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 16:37:48.159598 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 16:37:48.161287 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 16:37:48.161287 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 16:37:48.161287 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 16:37:48.161287 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 16:37:48.161287 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 16:37:48.161287 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 16:37:48.161287 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 16:37:48.161287 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 16:37:48.161287 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 16:37:48.161287 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 16:37:48.161287 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 16:37:48.161287 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application with Lock( Aug 01 16:37:48.161287 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 16:37:48.161287 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self._enter() Aug 01 16:37:48.161287 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 16:37:48.161287 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 16:37:48.162276 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 16:37:48.162276 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self.creator() Aug 01 16:37:48.162276 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 16:37:48.162276 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application created_value = creator( Aug 01 16:37:48.162276 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 16:37:48.162276 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 16:37:48.162276 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 16:37:48.162276 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 16:37:48.162276 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 16:37:48.162276 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 16:37:48.162276 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 01 16:37:48.162276 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application Aug 01 16:37:48.162276 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 532/1078] 10.222.0.20 () {56 vars in 1120 bytes} [Mon Aug 1 16:37:47 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 01 16:37:48.167439 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-195502cb-09f1-49be-a61b-2164438f2861 None admin] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:37:48.189496 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc8fe3cc-6177-4a14-86ca-6b4fe230d8d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:37:48.190185 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8fe3cc-6177-4a14-86ca-6b4fe230d8d2 None admin] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:37:48.190464 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8fe3cc-6177-4a14-86ca-6b4fe230d8d2 None admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:37:48.190734 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8fe3cc-6177-4a14-86ca-6b4fe230d8d2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:37:48.202647 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc8fe3cc-6177-4a14-86ca-6b4fe230d8d2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:37:48.204120 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc8fe3cc-6177-4a14-86ca-6b4fe230d8d2 None admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:37:48.219377 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application [None req-fc8fe3cc-6177-4a14-86ca-6b4fe230d8d2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 16:37:48.219377 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 16:37:48.219377 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 16:37:48.219377 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 16:37:48.219377 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 16:37:48.219377 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 16:37:48.219377 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 16:37:48.219377 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 16:37:48.219377 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 16:37:48.219377 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 16:37:48.219377 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 16:37:48.219377 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 16:37:48.219377 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 16:37:48.219377 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 16:37:48.219377 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 16:37:48.220348 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 16:37:48.220348 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 16:37:48.220348 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 16:37:48.220348 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 16:37:48.220348 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 16:37:48.220348 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 16:37:48.220348 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 16:37:48.220348 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 16:37:48.220348 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application with Lock( Aug 01 16:37:48.220348 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 16:37:48.220348 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self._enter() Aug 01 16:37:48.220348 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 16:37:48.220348 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 16:37:48.220348 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 16:37:48.220348 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self.creator() Aug 01 16:37:48.220348 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 16:37:48.220348 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application created_value = creator( Aug 01 16:37:48.222012 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 16:37:48.222012 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 16:37:48.222012 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 16:37:48.222012 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application Aug 01 16:37:48.222012 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 547/1079] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Aug 1 16:37:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 01 16:37:48.226673 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc8fe3cc-6177-4a14-86ca-6b4fe230d8d2 None admin] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:37:48.273872 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG dbcounter [-] [87057] Writing DB stats keystone:SELECT=100 {{(pid=87057) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:37:48.283609 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-195502cb-09f1-49be-a61b-2164438f2861 None admin] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:37:48.301938 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG dbcounter [-] [87058] Writing DB stats keystone:SELECT=100 {{(pid=87058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:37:48.322107 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ed8280e-04db-491f-94ef-35381787c7f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:37:48.322752 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed8280e-04db-491f-94ef-35381787c7f3 None admin] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:37:48.323031 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed8280e-04db-491f-94ef-35381787c7f3 None admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:37:48.323308 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed8280e-04db-491f-94ef-35381787c7f3 None admin] PATH_INFO: `/v3/roles` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:37:48.324021 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ed8280e-04db-491f-94ef-35381787c7f3 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=87057) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:37:48.324381 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ed8280e-04db-491f-94ef-35381787c7f3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:37:48.325761 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ed8280e-04db-491f-94ef-35381787c7f3 None admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:37:48.333704 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc8fe3cc-6177-4a14-86ca-6b4fe230d8d2 None admin] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:37:48.334697 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 533/1080] 10.222.0.20 () {56 vars in 1115 bytes} [Mon Aug 1 16:37:48 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 16:37:48.340053 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ed8280e-04db-491f-94ef-35381787c7f3 None admin] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:37:48.373051 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30dbbe9c-09a8-405c-9bf8-1cb3949e7462 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:37:48.373596 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30dbbe9c-09a8-405c-9bf8-1cb3949e7462 None admin] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:37:48.373877 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30dbbe9c-09a8-405c-9bf8-1cb3949e7462 None admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:37:48.374180 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30dbbe9c-09a8-405c-9bf8-1cb3949e7462 None admin] PATH_INFO: `/v3/domains` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:37:48.374974 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30dbbe9c-09a8-405c-9bf8-1cb3949e7462 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87058) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:37:48.375280 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30dbbe9c-09a8-405c-9bf8-1cb3949e7462 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:37:48.376661 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30dbbe9c-09a8-405c-9bf8-1cb3949e7462 None admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:37:48.391285 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 548/1081] 10.222.0.20 () {56 vars in 1109 bytes} [Mon Aug 1 16:37:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 16:37:48.461031 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ed8280e-04db-491f-94ef-35381787c7f3 None admin] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:37:48.499165 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58d069dc-3357-4f91-992c-72526172dc60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:37:48.500176 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d069dc-3357-4f91-992c-72526172dc60 None admin] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:37:48.500640 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d069dc-3357-4f91-992c-72526172dc60 None admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:37:48.501093 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d069dc-3357-4f91-992c-72526172dc60 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:37:48.513789 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58d069dc-3357-4f91-992c-72526172dc60 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:37:48.515871 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58d069dc-3357-4f91-992c-72526172dc60 None admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:37:48.531999 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application [None req-58d069dc-3357-4f91-992c-72526172dc60 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 16:37:48.531999 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 16:37:48.531999 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 16:37:48.531999 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 16:37:48.531999 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 16:37:48.531999 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 16:37:48.531999 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 16:37:48.531999 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 16:37:48.531999 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 16:37:48.531999 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 16:37:48.531999 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 16:37:48.531999 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 16:37:48.531999 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 16:37:48.531999 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 16:37:48.531999 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 16:37:48.532996 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 16:37:48.532996 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 16:37:48.532996 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 16:37:48.532996 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 16:37:48.532996 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 16:37:48.532996 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 16:37:48.532996 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 16:37:48.532996 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 16:37:48.532996 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application with Lock( Aug 01 16:37:48.532996 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 16:37:48.532996 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self._enter() Aug 01 16:37:48.532996 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 16:37:48.532996 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 16:37:48.532996 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 16:37:48.532996 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self.creator() Aug 01 16:37:48.532996 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 16:37:48.532996 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application created_value = creator( Aug 01 16:37:48.533911 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 16:37:48.533911 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 16:37:48.533911 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 16:37:48.533911 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application Aug 01 16:37:48.536830 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 534/1082] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Aug 1 16:37:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 01 16:37:48.542216 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30dbbe9c-09a8-405c-9bf8-1cb3949e7462 None admin] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:37:48.651602 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30dbbe9c-09a8-405c-9bf8-1cb3949e7462 None admin] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:37:48.683761 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG dbcounter [-] [87058] Writing DB stats keystone:SELECT=100 {{(pid=87058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:37:48.688905 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1294e726-6be6-4e7f-bce7-e712be1e9a00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:37:48.689528 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1294e726-6be6-4e7f-bce7-e712be1e9a00 None admin] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:37:48.689806 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1294e726-6be6-4e7f-bce7-e712be1e9a00 None admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:37:48.690102 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1294e726-6be6-4e7f-bce7-e712be1e9a00 None admin] PATH_INFO: `/v3/domains` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:37:48.690794 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1294e726-6be6-4e7f-bce7-e712be1e9a00 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87058) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:37:48.691097 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1294e726-6be6-4e7f-bce7-e712be1e9a00 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:37:48.692742 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1294e726-6be6-4e7f-bce7-e712be1e9a00 None admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:37:48.703653 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 549/1083] 10.222.0.20 () {56 vars in 1109 bytes} [Mon Aug 1 16:37:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:37:48.709961 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58d069dc-3357-4f91-992c-72526172dc60 None admin] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:37:48.714295 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG dbcounter [-] [87057] Writing DB stats keystone:SELECT=100 {{(pid=87057) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:37:48.807015 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58d069dc-3357-4f91-992c-72526172dc60 None admin] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:37:48.839454 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb828a18-61b8-4a1a-932b-0fc543541ea8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:37:48.840085 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb828a18-61b8-4a1a-932b-0fc543541ea8 None admin] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:37:48.840363 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb828a18-61b8-4a1a-932b-0fc543541ea8 None admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:37:48.840636 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb828a18-61b8-4a1a-932b-0fc543541ea8 None admin] PATH_INFO: `/v3/users/nova` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:37:48.848581 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb828a18-61b8-4a1a-932b-0fc543541ea8 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:37:48.850013 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb828a18-61b8-4a1a-932b-0fc543541ea8 None admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:37:48.860876 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application [None req-eb828a18-61b8-4a1a-932b-0fc543541ea8 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 01 16:37:48.860876 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 16:37:48.860876 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 16:37:48.860876 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 16:37:48.860876 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 16:37:48.860876 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 16:37:48.860876 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 16:37:48.860876 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 16:37:48.860876 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 16:37:48.860876 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 16:37:48.860876 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 16:37:48.860876 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 16:37:48.860876 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 01 16:37:48.860876 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 01 16:37:48.860876 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 01 16:37:48.860876 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 01 16:37:48.862012 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 16:37:48.862012 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 16:37:48.862012 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 01 16:37:48.862012 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 16:37:48.862012 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 01 16:37:48.862012 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 16:37:48.862012 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 16:37:48.862012 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 16:37:48.862012 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 16:37:48.862012 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 16:37:48.862012 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 16:37:48.862012 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application with Lock( Aug 01 16:37:48.862012 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 16:37:48.862012 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self._enter() Aug 01 16:37:48.862012 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 16:37:48.862012 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 16:37:48.862012 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 16:37:48.862012 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self.creator() Aug 01 16:37:48.863045 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 16:37:48.863045 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application created_value = creator( Aug 01 16:37:48.863045 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 01 16:37:48.863045 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 01 16:37:48.863045 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 01 16:37:48.863045 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 01 16:37:48.863045 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 01 16:37:48.863045 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 01 16:37:48.863045 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 01 16:37:48.863045 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application Aug 01 16:37:48.863045 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 535/1084] 10.222.0.20 () {56 vars in 1093 bytes} [Mon Aug 1 16:37:48 2022] GET /identity/v3/users/nova => generated 82 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 16:37:48.868456 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1294e726-6be6-4e7f-bce7-e712be1e9a00 None admin] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:37:48.968124 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1294e726-6be6-4e7f-bce7-e712be1e9a00 None admin] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:37:49.001988 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef4dbbd6-d269-493f-afcf-3a6ad33477c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:37:49.002632 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4dbbd6-d269-493f-afcf-3a6ad33477c4 None admin] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:37:49.002910 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4dbbd6-d269-493f-afcf-3a6ad33477c4 None admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:37:49.003180 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4dbbd6-d269-493f-afcf-3a6ad33477c4 None admin] PATH_INFO: `/v3/users` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:37:49.003969 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef4dbbd6-d269-493f-afcf-3a6ad33477c4 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=87058) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:37:49.004339 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef4dbbd6-d269-493f-afcf-3a6ad33477c4 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:37:49.006467 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef4dbbd6-d269-493f-afcf-3a6ad33477c4 None admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:37:49.024372 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 550/1085] 10.222.0.20 () {56 vars in 1133 bytes} [Mon Aug 1 16:37:48 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:37:49.030167 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb828a18-61b8-4a1a-932b-0fc543541ea8 None admin] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:37:49.092386 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG dbcounter [-] [87057] Writing DB stats keystone:SELECT=100 {{(pid=87057) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:37:49.130949 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb828a18-61b8-4a1a-932b-0fc543541ea8 None admin] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:37:49.164483 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29219cfe-f624-4ccb-aa8b-b506f899b2c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:37:49.165019 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29219cfe-f624-4ccb-aa8b-b506f899b2c4 None admin] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:37:49.165295 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29219cfe-f624-4ccb-aa8b-b506f899b2c4 None admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:37:49.165564 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29219cfe-f624-4ccb-aa8b-b506f899b2c4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:37:49.230609 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29219cfe-f624-4ccb-aa8b-b506f899b2c4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:37:49.232047 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29219cfe-f624-4ccb-aa8b-b506f899b2c4 None admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:37:49.290703 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG dbcounter [-] [87057] Writing DB stats keystone:SELECT=100 {{(pid=87057) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:37:49.327638 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 536/1086] 10.222.0.20 () {58 vars in 1282 bytes} [Mon Aug 1 16:37:49 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 298 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 01 16:37:49.334127 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef4dbbd6-d269-493f-afcf-3a6ad33477c4 None admin] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:37:49.383625 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG dbcounter [-] [87058] Writing DB stats keystone:SELECT=100 {{(pid=87058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:37:49.433305 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef4dbbd6-d269-493f-afcf-3a6ad33477c4 None admin] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:37:49.467259 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fdabe9e-7395-4581-8d56-8d223d2a3066 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:37:49.467894 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fdabe9e-7395-4581-8d56-8d223d2a3066 None admin] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:37:49.468170 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fdabe9e-7395-4581-8d56-8d223d2a3066 None admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:37:49.468438 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fdabe9e-7395-4581-8d56-8d223d2a3066 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:37:49.478501 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fdabe9e-7395-4581-8d56-8d223d2a3066 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:37:49.479928 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fdabe9e-7395-4581-8d56-8d223d2a3066 None admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:37:49.494642 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application [None req-5fdabe9e-7395-4581-8d56-8d223d2a3066 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 16:37:49.494642 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 16:37:49.494642 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 16:37:49.494642 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 16:37:49.494642 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 16:37:49.494642 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 16:37:49.494642 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 16:37:49.494642 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 16:37:49.494642 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 16:37:49.494642 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 16:37:49.494642 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 16:37:49.494642 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 16:37:49.494642 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 16:37:49.494642 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 16:37:49.494642 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 16:37:49.495736 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 16:37:49.495736 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 16:37:49.495736 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 16:37:49.495736 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 16:37:49.495736 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 16:37:49.495736 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 16:37:49.495736 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 16:37:49.495736 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 16:37:49.495736 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application with Lock( Aug 01 16:37:49.495736 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 16:37:49.495736 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self._enter() Aug 01 16:37:49.495736 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 16:37:49.495736 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 16:37:49.495736 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 16:37:49.495736 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self.creator() Aug 01 16:37:49.495736 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 16:37:49.495736 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application created_value = creator( Aug 01 16:37:49.496870 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 16:37:49.496870 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 16:37:49.496870 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 16:37:49.496870 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application Aug 01 16:37:49.498198 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 551/1087] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Aug 1 16:37:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 16:37:49.501381 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29219cfe-f624-4ccb-aa8b-b506f899b2c4 None admin] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:37:49.502024 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d857304-d730-4a87-9662-74e7f788418d None None] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:37:49.502341 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d857304-d730-4a87-9662-74e7f788418d None None] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:37:49.502620 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d857304-d730-4a87-9662-74e7f788418d None None] PATH_INFO: `/` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:37:49.503923 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 552/1088] 10.222.0.20 () {54 vars in 926 bytes} [Mon Aug 1 16:37:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 01 16:37:49.509367 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981edd8c-ddb5-4664-a1f8-48f58aac383a None None] REQUEST_METHOD: `POST` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:37:49.509647 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981edd8c-ddb5-4664-a1f8-48f58aac383a None None] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:37:49.509918 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981edd8c-ddb5-4664-a1f8-48f58aac383a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:37:49.528034 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.auth.core [None req-981edd8c-ddb5-4664-a1f8-48f58aac383a None None] MFA Rules not processed for user `1007f8e379b74e169b98fb7c3c0ba5b4`. Rule list: `[]` (Enabled: `True`). {{(pid=87058) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:37:49.529253 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.fernet_utils [None req-981edd8c-ddb5-4664-a1f8-48f58aac383a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:37:49.601861 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 553/1089] 10.222.0.20 () {58 vars in 1023 bytes} [Mon Aug 1 16:37:49 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 01 16:37:49.602933 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29219cfe-f624-4ccb-aa8b-b506f899b2c4 None admin] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:37:49.624140 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-981edd8c-ddb5-4664-a1f8-48f58aac383a None None] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:37:49.635974 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde4dfcb-6941-43aa-8112-c942853efb42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:37:49.636593 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde4dfcb-6941-43aa-8112-c942853efb42 None admin] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:37:49.636868 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde4dfcb-6941-43aa-8112-c942853efb42 None admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:37:49.637140 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde4dfcb-6941-43aa-8112-c942853efb42 None admin] PATH_INFO: `/v3/domains` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:37:49.637830 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fde4dfcb-6941-43aa-8112-c942853efb42 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87057) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:37:49.638165 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fde4dfcb-6941-43aa-8112-c942853efb42 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:37:49.639526 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fde4dfcb-6941-43aa-8112-c942853efb42 None admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:37:49.651673 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 537/1090] 10.222.0.20 () {56 vars in 1109 bytes} [Mon Aug 1 16:37:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:37:49.654878 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG dbcounter [-] [87058] Writing DB stats keystone:SELECT=100 {{(pid=87058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:37:49.657144 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde4dfcb-6941-43aa-8112-c942853efb42 None admin] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:37:49.682682 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG dbcounter [-] [87057] Writing DB stats keystone:SELECT=100 {{(pid=87057) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:37:49.722456 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-981edd8c-ddb5-4664-a1f8-48f58aac383a None None] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:37:49.756759 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d5c902c-908d-445b-8da5-70d989d13560 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:37:49.757390 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5c902c-908d-445b-8da5-70d989d13560 None admin] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:37:49.757575 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde4dfcb-6941-43aa-8112-c942853efb42 None admin] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:37:49.757677 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5c902c-908d-445b-8da5-70d989d13560 None admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:37:49.757921 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5c902c-908d-445b-8da5-70d989d13560 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:37:49.768252 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d5c902c-908d-445b-8da5-70d989d13560 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 16:37:49.773645 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application [None req-6d5c902c-908d-445b-8da5-70d989d13560 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 16:37:49.773645 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 16:37:49.773645 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 16:37:49.773645 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 16:37:49.773645 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 16:37:49.773645 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 16:37:49.773645 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 16:37:49.773645 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 16:37:49.773645 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 16:37:49.773645 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 16:37:49.773645 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 16:37:49.773645 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 16:37:49.773645 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 16:37:49.773645 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 16:37:49.773645 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 16:37:49.773645 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 16:37:49.774708 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 16:37:49.774708 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 16:37:49.774708 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 16:37:49.774708 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 16:37:49.774708 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 16:37:49.774708 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 16:37:49.774708 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 16:37:49.774708 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 16:37:49.774708 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 16:37:49.774708 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 16:37:49.774708 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 16:37:49.774708 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application with Lock( Aug 01 16:37:49.774708 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 16:37:49.774708 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self._enter() Aug 01 16:37:49.774708 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 16:37:49.774708 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 16:37:49.775575 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 16:37:49.775575 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self.creator() Aug 01 16:37:49.775575 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 16:37:49.775575 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application created_value = creator( Aug 01 16:37:49.775575 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 16:37:49.775575 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 16:37:49.775575 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 16:37:49.775575 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 16:37:49.775575 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 16:37:49.775575 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 16:37:49.775575 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 16:37:49.775575 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application Aug 01 16:37:49.775575 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 554/1091] 10.222.0.20 () {56 vars in 1096 bytes} [Mon Aug 1 16:37:49 2022] GET /identity/v3/roles/admin => generated 83 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 01 16:37:49.780759 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d5c902c-908d-445b-8da5-70d989d13560 None admin] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:37:49.791907 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-244a31e6-1ee1-495d-9cb8-c0edc399db8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:37:49.792550 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244a31e6-1ee1-495d-9cb8-c0edc399db8d None admin] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:37:49.792829 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244a31e6-1ee1-495d-9cb8-c0edc399db8d None admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:37:49.793099 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244a31e6-1ee1-495d-9cb8-c0edc399db8d None admin] PATH_INFO: `/v3/projects/service` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:37:49.799656 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-244a31e6-1ee1-495d-9cb8-c0edc399db8d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:37:49.801095 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-244a31e6-1ee1-495d-9cb8-c0edc399db8d None admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:37:49.812411 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application [None req-244a31e6-1ee1-495d-9cb8-c0edc399db8d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 16:37:49.812411 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 16:37:49.812411 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 16:37:49.812411 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 16:37:49.812411 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 16:37:49.812411 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 16:37:49.812411 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 16:37:49.812411 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 16:37:49.812411 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 16:37:49.812411 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 16:37:49.812411 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 16:37:49.812411 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 16:37:49.812411 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 01 16:37:49.812411 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 01 16:37:49.812411 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 01 16:37:49.813515 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 01 16:37:49.813515 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 16:37:49.813515 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 16:37:49.813515 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 16:37:49.813515 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 16:37:49.813515 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 16:37:49.813515 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 16:37:49.813515 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 16:37:49.813515 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application with Lock( Aug 01 16:37:49.813515 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 16:37:49.813515 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self._enter() Aug 01 16:37:49.813515 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 16:37:49.813515 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 16:37:49.813515 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 16:37:49.813515 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self.creator() Aug 01 16:37:49.813515 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 16:37:49.813515 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application created_value = creator( Aug 01 16:37:49.820497 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 01 16:37:49.820497 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 01 16:37:49.820497 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 01 16:37:49.820497 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 01 16:37:49.820497 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 01 16:37:49.820497 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 01 16:37:49.820497 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 16:37:49.820497 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application Aug 01 16:37:49.820497 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 538/1092] 10.222.0.20 () {56 vars in 1111 bytes} [Mon Aug 1 16:37:49 2022] GET /identity/v3/projects/service => generated 88 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 01 16:37:49.820497 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-244a31e6-1ee1-495d-9cb8-c0edc399db8d None admin] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:37:49.872442 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG dbcounter [-] [87058] Writing DB stats keystone:SELECT=100 {{(pid=87058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:37:49.881422 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d5c902c-908d-445b-8da5-70d989d13560 None admin] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:37:49.911884 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG dbcounter [-] [87057] Writing DB stats keystone:SELECT=100 {{(pid=87057) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:37:49.917981 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a14f4a7-df87-4c28-acfe-d730e21f9775 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:37:49.918721 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a14f4a7-df87-4c28-acfe-d730e21f9775 None admin] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:37:49.919007 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a14f4a7-df87-4c28-acfe-d730e21f9775 None admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:37:49.919276 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a14f4a7-df87-4c28-acfe-d730e21f9775 None admin] PATH_INFO: `/v3/roles` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:37:49.920046 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a14f4a7-df87-4c28-acfe-d730e21f9775 None admin] RBAC: Adding query filter params (name=admin) {{(pid=87058) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:37:49.924014 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a14f4a7-df87-4c28-acfe-d730e21f9775 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:37:49.924014 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a14f4a7-df87-4c28-acfe-d730e21f9775 None admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:37:49.930741 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-244a31e6-1ee1-495d-9cb8-c0edc399db8d None admin] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:37:49.931270 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 555/1093] 10.222.0.20 () {56 vars in 1099 bytes} [Mon Aug 1 16:37:49 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 16:37:49.937286 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a14f4a7-df87-4c28-acfe-d730e21f9775 None admin] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:37:49.967128 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c46a06-7cae-481c-bc3a-9ae72aa89417 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:37:49.967747 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c46a06-7cae-481c-bc3a-9ae72aa89417 None admin] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:37:49.968024 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c46a06-7cae-481c-bc3a-9ae72aa89417 None admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:37:49.968298 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c46a06-7cae-481c-bc3a-9ae72aa89417 None admin] PATH_INFO: `/v3/projects` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:37:49.969054 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70c46a06-7cae-481c-bc3a-9ae72aa89417 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87057) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:37:49.969359 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70c46a06-7cae-481c-bc3a-9ae72aa89417 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:37:49.970829 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70c46a06-7cae-481c-bc3a-9ae72aa89417 None admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:37:49.982349 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 539/1094] 10.222.0.20 () {56 vars in 1148 bytes} [Mon Aug 1 16:37:49 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 16:37:49.988313 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c46a06-7cae-481c-bc3a-9ae72aa89417 None admin] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:37:50.039562 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a14f4a7-df87-4c28-acfe-d730e21f9775 None admin] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:37:50.072927 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b4df59-4d5a-4616-b936-688bc7710159 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:37:50.073586 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b4df59-4d5a-4616-b936-688bc7710159 None admin] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:37:50.073866 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b4df59-4d5a-4616-b936-688bc7710159 None admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:37:50.074176 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b4df59-4d5a-4616-b936-688bc7710159 None admin] PATH_INFO: `/v3/users/909657aaff6a45ab9e781f13f918c6d4` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:37:50.075348 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48b4df59-4d5a-4616-b936-688bc7710159 None admin] RBAC: Authorizing `identity:get_user(user_id=909657aaff6a45ab9e781f13f918c6d4)` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:37:50.076779 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48b4df59-4d5a-4616-b936-688bc7710159 None admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:37:50.078422 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 556/1095] 10.222.0.20 () {56 vars in 1177 bytes} [Mon Aug 1 16:37:49 2022] GET /identity/v3/users/909657aaff6a45ab9e781f13f918c6d4 => generated 285 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 16:37:50.084819 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b4df59-4d5a-4616-b936-688bc7710159 None admin] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:37:50.090765 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c46a06-7cae-481c-bc3a-9ae72aa89417 None admin] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:37:50.108233 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG dbcounter [-] [87058] Writing DB stats keystone:SELECT=100 {{(pid=87058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:37:50.123074 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3471f952-2ddb-4010-b969-81f4f558e465 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:37:50.124151 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3471f952-2ddb-4010-b969-81f4f558e465 None admin] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:37:50.124151 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3471f952-2ddb-4010-b969-81f4f558e465 None admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:37:50.124322 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3471f952-2ddb-4010-b969-81f4f558e465 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:37:50.124979 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3471f952-2ddb-4010-b969-81f4f558e465 None admin] RBAC: Adding query filter params (role.id=653d2ccc68e8415191e22643427e28bc, scope.project.id=7b992d6a65df4bf9b27c13192d57c5f0, user.id=6533106d03eb4ca185a8839336d2aff9) {{(pid=87057) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:37:50.125285 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3471f952-2ddb-4010-b969-81f4f558e465 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:37:50.126747 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3471f952-2ddb-4010-b969-81f4f558e465 None admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:37:50.133188 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 540/1096] 10.222.0.20 () {56 vars in 1374 bytes} [Mon Aug 1 16:37:49 2022] GET /identity/v3/role_assignments?user.id=6533106d03eb4ca185a8839336d2aff9&scope.project.id=7b992d6a65df4bf9b27c13192d57c5f0&role.id=653d2ccc68e8415191e22643427e28bc => generated 608 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:37:50.189773 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b4df59-4d5a-4616-b936-688bc7710159 None admin] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:37:50.226970 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5116699-1f20-49a9-a640-08f780121a3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:37:50.227638 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5116699-1f20-49a9-a640-08f780121a3a None admin] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:37:50.227918 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5116699-1f20-49a9-a640-08f780121a3a None admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:37:50.228195 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5116699-1f20-49a9-a640-08f780121a3a None admin] PATH_INFO: `/v3/projects/585c4683da3243eea2b4695c50946bdd` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:37:50.229403 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5116699-1f20-49a9-a640-08f780121a3a None admin] RBAC: Authorizing `identity:get_project(project_id=585c4683da3243eea2b4695c50946bdd)` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:37:50.230887 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5116699-1f20-49a9-a640-08f780121a3a None admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:37:50.232676 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 557/1097] 10.222.0.20 () {56 vars in 1186 bytes} [Mon Aug 1 16:37:50 2022] GET /identity/v3/projects/585c4683da3243eea2b4695c50946bdd => generated 311 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 16:37:50.238873 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3471f952-2ddb-4010-b969-81f4f558e465 None admin] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:37:50.251141 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG dbcounter [-] [87057] Writing DB stats keystone:SELECT=100 {{(pid=87057) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:37:50.343112 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3471f952-2ddb-4010-b969-81f4f558e465 None admin] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:37:50.377080 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-387feb8d-5f1d-4d5a-8098-436b801c4876 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:37:50.377759 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387feb8d-5f1d-4d5a-8098-436b801c4876 None admin] REQUEST_METHOD: `PUT` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:37:50.378054 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387feb8d-5f1d-4d5a-8098-436b801c4876 None admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:37:50.378340 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387feb8d-5f1d-4d5a-8098-436b801c4876 None admin] PATH_INFO: `/v3/projects/585c4683da3243eea2b4695c50946bdd/users/909657aaff6a45ab9e781f13f918c6d4/roles/440bb943dd5b46fea038b903b6eaaf81` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:37:50.380891 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-387feb8d-5f1d-4d5a-8098-436b801c4876 None admin] RBAC: Authorizing `identity:create_grant(project_id=585c4683da3243eea2b4695c50946bdd, user_id=909657aaff6a45ab9e781f13f918c6d4, role_id=440bb943dd5b46fea038b903b6eaaf81)` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:37:50.382356 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-387feb8d-5f1d-4d5a-8098-436b801c4876 None admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:37:50.454183 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 541/1098] 10.222.0.20 () {58 vars in 1439 bytes} [Mon Aug 1 16:37:50 2022] PUT /identity/v3/projects/585c4683da3243eea2b4695c50946bdd/users/909657aaff6a45ab9e781f13f918c6d4/roles/440bb943dd5b46fea038b903b6eaaf81 => generated 0 bytes in 216 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 16:37:52.421054 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce50ed2-b9ab-4ba3-8cca-d887c5f4cfea None None] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:37:52.421591 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce50ed2-b9ab-4ba3-8cca-d887c5f4cfea None None] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:37:52.421591 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce50ed2-b9ab-4ba3-8cca-d887c5f4cfea None None] PATH_INFO: `/` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:37:52.422986 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 558/1099] 10.222.0.20 () {54 vars in 926 bytes} [Mon Aug 1 16:37:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 16:37:52.428563 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34f3aba-fef6-439f-95f5-5a56ef728a71 None None] REQUEST_METHOD: `POST` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:37:52.429005 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34f3aba-fef6-439f-95f5-5a56ef728a71 None None] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:37:52.429097 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34f3aba-fef6-439f-95f5-5a56ef728a71 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:37:52.447003 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.auth.core [None req-e34f3aba-fef6-439f-95f5-5a56ef728a71 None None] MFA Rules not processed for user `1007f8e379b74e169b98fb7c3c0ba5b4`. Rule list: `[]` (Enabled: `True`). {{(pid=87057) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:37:52.448207 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.fernet_utils [None req-e34f3aba-fef6-439f-95f5-5a56ef728a71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87057) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:37:52.508028 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG dbcounter [-] [87057] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87057) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:37:52.513303 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 542/1100] 10.222.0.20 () {58 vars in 1023 bytes} [Mon Aug 1 16:37:52 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 01 16:37:52.537216 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce50ed2-b9ab-4ba3-8cca-d887c5f4cfea None None] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:37:52.624854 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG dbcounter [-] [87058] Writing DB stats keystone:SELECT=100 {{(pid=87058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:37:52.634842 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce50ed2-b9ab-4ba3-8cca-d887c5f4cfea None None] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:37:52.668966 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7078d416-8d0f-48ad-a949-8f1252fa66db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:37:52.669589 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7078d416-8d0f-48ad-a949-8f1252fa66db None admin] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:37:52.669864 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7078d416-8d0f-48ad-a949-8f1252fa66db None admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:37:52.670160 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7078d416-8d0f-48ad-a949-8f1252fa66db None admin] PATH_INFO: `/v3/roles/admin` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:37:52.679884 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7078d416-8d0f-48ad-a949-8f1252fa66db None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 16:37:52.684613 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application [None req-7078d416-8d0f-48ad-a949-8f1252fa66db None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 16:37:52.684613 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 16:37:52.684613 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 16:37:52.684613 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 16:37:52.684613 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 16:37:52.684613 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 16:37:52.684613 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 16:37:52.684613 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 16:37:52.684613 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 16:37:52.684613 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 16:37:52.684613 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 16:37:52.684613 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 16:37:52.684613 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 16:37:52.684613 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 16:37:52.684613 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 16:37:52.684613 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 16:37:52.685709 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 16:37:52.685709 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 16:37:52.685709 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 16:37:52.685709 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 16:37:52.685709 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 16:37:52.685709 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 16:37:52.685709 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 16:37:52.685709 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 16:37:52.685709 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 16:37:52.685709 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 16:37:52.685709 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 16:37:52.685709 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application with Lock( Aug 01 16:37:52.685709 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 16:37:52.685709 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self._enter() Aug 01 16:37:52.685709 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 16:37:52.685709 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 16:37:52.686814 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 16:37:52.686814 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self.creator() Aug 01 16:37:52.686814 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 16:37:52.686814 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application created_value = creator( Aug 01 16:37:52.686814 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 16:37:52.686814 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 16:37:52.686814 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 16:37:52.686814 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 16:37:52.686814 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 16:37:52.686814 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 16:37:52.686814 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 16:37:52.686814 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application Aug 01 16:37:52.686814 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 559/1101] 10.222.0.20 () {56 vars in 1096 bytes} [Mon Aug 1 16:37:52 2022] GET /identity/v3/roles/admin => generated 83 bytes in 149 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 16:37:52.691887 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e34f3aba-fef6-439f-95f5-5a56ef728a71 None None] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:37:52.787408 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e34f3aba-fef6-439f-95f5-5a56ef728a71 None None] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:37:52.819448 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf7cb74-80ed-4fbf-bdee-e8d41359e87e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:37:52.820051 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf7cb74-80ed-4fbf-bdee-e8d41359e87e None admin] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:37:52.820329 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf7cb74-80ed-4fbf-bdee-e8d41359e87e None admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:37:52.820604 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf7cb74-80ed-4fbf-bdee-e8d41359e87e None admin] PATH_INFO: `/v3/roles` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:37:52.821303 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdf7cb74-80ed-4fbf-bdee-e8d41359e87e None admin] RBAC: Adding query filter params (name=admin) {{(pid=87057) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:37:52.821601 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdf7cb74-80ed-4fbf-bdee-e8d41359e87e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:37:52.822985 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdf7cb74-80ed-4fbf-bdee-e8d41359e87e None admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:37:52.830615 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 543/1102] 10.222.0.20 () {56 vars in 1099 bytes} [Mon Aug 1 16:37:52 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:37:52.836959 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7078d416-8d0f-48ad-a949-8f1252fa66db None admin] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:37:52.935478 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7078d416-8d0f-48ad-a949-8f1252fa66db None admin] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:37:52.969011 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3560fc1d-f427-4f66-90c3-5fc1f34cf6bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:37:52.969641 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3560fc1d-f427-4f66-90c3-5fc1f34cf6bc None admin] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:37:52.969920 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3560fc1d-f427-4f66-90c3-5fc1f34cf6bc None admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:37:52.970223 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3560fc1d-f427-4f66-90c3-5fc1f34cf6bc None admin] PATH_INFO: `/v3/users/909657aaff6a45ab9e781f13f918c6d4` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:37:52.971377 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3560fc1d-f427-4f66-90c3-5fc1f34cf6bc None admin] RBAC: Authorizing `identity:get_user(user_id=909657aaff6a45ab9e781f13f918c6d4)` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:37:52.972861 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3560fc1d-f427-4f66-90c3-5fc1f34cf6bc None admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:37:52.974607 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 560/1103] 10.222.0.20 () {56 vars in 1177 bytes} [Mon Aug 1 16:37:52 2022] GET /identity/v3/users/909657aaff6a45ab9e781f13f918c6d4 => generated 285 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:37:52.980339 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf7cb74-80ed-4fbf-bdee-e8d41359e87e None admin] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:37:53.037140 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG dbcounter [-] [87057] Writing DB stats keystone:SELECT=100 {{(pid=87057) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:37:53.076607 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf7cb74-80ed-4fbf-bdee-e8d41359e87e None admin] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:37:53.109348 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f59d43e-e172-457b-9cef-f87c22c50c72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:37:53.109948 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f59d43e-e172-457b-9cef-f87c22c50c72 None admin] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:37:53.110255 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f59d43e-e172-457b-9cef-f87c22c50c72 None admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:37:53.110528 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f59d43e-e172-457b-9cef-f87c22c50c72 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:37:53.175702 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f59d43e-e172-457b-9cef-f87c22c50c72 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:37:53.177314 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f59d43e-e172-457b-9cef-f87c22c50c72 None admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:37:53.232125 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG dbcounter [-] [87057] Writing DB stats keystone:SELECT=100 {{(pid=87057) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:37:53.272982 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 544/1104] 10.222.0.20 () {58 vars in 1282 bytes} [Mon Aug 1 16:37:52 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 293 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 01 16:37:53.278874 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3560fc1d-f427-4f66-90c3-5fc1f34cf6bc None admin] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:37:53.297785 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG dbcounter [-] [87058] Writing DB stats keystone:SELECT=100 {{(pid=87058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:37:53.377278 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3560fc1d-f427-4f66-90c3-5fc1f34cf6bc None admin] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:37:53.411526 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c3bf53-0fdc-44ad-baae-0d6220664bb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:37:53.412192 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c3bf53-0fdc-44ad-baae-0d6220664bb8 None admin] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:37:53.412474 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c3bf53-0fdc-44ad-baae-0d6220664bb8 None admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:37:53.412748 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c3bf53-0fdc-44ad-baae-0d6220664bb8 None admin] PATH_INFO: `/v3/projects/585c4683da3243eea2b4695c50946bdd` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:37:53.413913 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0c3bf53-0fdc-44ad-baae-0d6220664bb8 None admin] RBAC: Authorizing `identity:get_project(project_id=585c4683da3243eea2b4695c50946bdd)` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:37:53.415358 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0c3bf53-0fdc-44ad-baae-0d6220664bb8 None admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:37:53.417020 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 561/1105] 10.222.0.20 () {56 vars in 1186 bytes} [Mon Aug 1 16:37:53 2022] GET /identity/v3/projects/585c4683da3243eea2b4695c50946bdd => generated 311 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:37:53.423223 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f59d43e-e172-457b-9cef-f87c22c50c72 None admin] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:37:53.519344 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f59d43e-e172-457b-9cef-f87c22c50c72 None admin] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:37:53.552941 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b96627b-ad54-4ff3-8c60-5ce81f065a09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:37:53.553543 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b96627b-ad54-4ff3-8c60-5ce81f065a09 None admin] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:37:53.553821 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b96627b-ad54-4ff3-8c60-5ce81f065a09 None admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:37:53.554115 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b96627b-ad54-4ff3-8c60-5ce81f065a09 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:37:53.554899 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b96627b-ad54-4ff3-8c60-5ce81f065a09 None admin] RBAC: Adding query filter params (role.id=440bb943dd5b46fea038b903b6eaaf81, scope.project.id=585c4683da3243eea2b4695c50946bdd, user.id=909657aaff6a45ab9e781f13f918c6d4) {{(pid=87057) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:37:53.555208 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b96627b-ad54-4ff3-8c60-5ce81f065a09 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:37:53.556739 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b96627b-ad54-4ff3-8c60-5ce81f065a09 None admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:37:53.562392 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 545/1106] 10.222.0.20 () {56 vars in 1374 bytes} [Mon Aug 1 16:37:53 2022] GET /identity/v3/role_assignments?user.id=909657aaff6a45ab9e781f13f918c6d4&scope.project.id=585c4683da3243eea2b4695c50946bdd&role.id=440bb943dd5b46fea038b903b6eaaf81 => generated 608 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:37:55.582665 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9be012-ff63-4c3d-ad19-ba2aa510ab34 None None] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:37:55.583218 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9be012-ff63-4c3d-ad19-ba2aa510ab34 None None] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:37:55.583362 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9be012-ff63-4c3d-ad19-ba2aa510ab34 None None] PATH_INFO: `/` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:37:55.584724 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 562/1107] 10.222.0.20 () {54 vars in 926 bytes} [Mon Aug 1 16:37:55 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 16:37:55.595579 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4b9c5e-a8da-41ad-aecd-f6ca8530c375 None None] REQUEST_METHOD: `POST` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:37:55.596045 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4b9c5e-a8da-41ad-aecd-f6ca8530c375 None None] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:37:55.596186 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4b9c5e-a8da-41ad-aecd-f6ca8530c375 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:37:55.614487 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.auth.core [None req-7e4b9c5e-a8da-41ad-aecd-f6ca8530c375 None None] MFA Rules not processed for user `1007f8e379b74e169b98fb7c3c0ba5b4`. Rule list: `[]` (Enabled: `True`). {{(pid=87057) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:37:55.615683 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.fernet_utils [None req-7e4b9c5e-a8da-41ad-aecd-f6ca8530c375 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87057) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:37:55.632482 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG dbcounter [-] [87057] Writing DB stats keystone:SELECT=100 {{(pid=87057) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:37:55.682238 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 546/1108] 10.222.0.20 () {58 vars in 1023 bytes} [Mon Aug 1 16:37:55 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 01 16:37:55.707192 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c9be012-ff63-4c3d-ad19-ba2aa510ab34 None None] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:37:55.789501 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG dbcounter [-] [87058] Writing DB stats keystone:SELECT=100 {{(pid=87058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:37:55.805419 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c9be012-ff63-4c3d-ad19-ba2aa510ab34 None None] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:37:55.839082 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc23703a-863e-46d8-b2b8-9d4568152372 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:37:55.839661 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc23703a-863e-46d8-b2b8-9d4568152372 None admin] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:37:55.839945 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc23703a-863e-46d8-b2b8-9d4568152372 None admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:37:55.840221 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc23703a-863e-46d8-b2b8-9d4568152372 None admin] PATH_INFO: `/v3/domains/default` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:37:55.841364 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc23703a-863e-46d8-b2b8-9d4568152372 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:37:55.842824 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc23703a-863e-46d8-b2b8-9d4568152372 None admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:37:55.844408 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 563/1109] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Aug 1 16:37:55 2022] GET /identity/v3/domains/default => generated 199 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:37:55.850601 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e4b9c5e-a8da-41ad-aecd-f6ca8530c375 None None] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:37:55.947975 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e4b9c5e-a8da-41ad-aecd-f6ca8530c375 None None] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:37:55.980463 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc5f39ee-7bff-4a8c-9460-36a553ceef01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:37:55.981163 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5f39ee-7bff-4a8c-9460-36a553ceef01 None admin] REQUEST_METHOD: `POST` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:37:55.981451 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5f39ee-7bff-4a8c-9460-36a553ceef01 None admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:37:55.981726 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5f39ee-7bff-4a8c-9460-36a553ceef01 None admin] PATH_INFO: `/v3/users` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:37:55.982443 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc5f39ee-7bff-4a8c-9460-36a553ceef01 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:37:55.983873 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc5f39ee-7bff-4a8c-9460-36a553ceef01 None admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:37:56.064810 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 547/1110] 10.222.0.20 () {60 vars in 1132 bytes} [Mon Aug 1 16:37:55 2022] POST /identity/v3/users => generated 286 bytes in 215 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 16:37:58.083723 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f3dcd0-2418-40c5-99ed-3502427e9a6e None None] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:37:58.084239 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f3dcd0-2418-40c5-99ed-3502427e9a6e None None] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:37:58.084239 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f3dcd0-2418-40c5-99ed-3502427e9a6e None None] PATH_INFO: `/` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:37:58.085606 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 564/1111] 10.222.0.20 () {54 vars in 926 bytes} [Mon Aug 1 16:37:58 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 16:37:58.090678 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73db2cd-3c6e-474c-bc18-37e8f1869649 None None] REQUEST_METHOD: `POST` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:37:58.091092 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73db2cd-3c6e-474c-bc18-37e8f1869649 None None] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:37:58.091236 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73db2cd-3c6e-474c-bc18-37e8f1869649 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:37:58.109403 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.auth.core [None req-d73db2cd-3c6e-474c-bc18-37e8f1869649 None None] MFA Rules not processed for user `1007f8e379b74e169b98fb7c3c0ba5b4`. Rule list: `[]` (Enabled: `True`). {{(pid=87057) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:37:58.110636 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.fernet_utils [None req-d73db2cd-3c6e-474c-bc18-37e8f1869649 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87057) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:37:58.113379 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG dbcounter [-] [87057] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87057) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:37:58.176637 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 548/1112] 10.222.0.20 () {58 vars in 1023 bytes} [Mon Aug 1 16:37:58 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 01 16:37:58.199493 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f3dcd0-2418-40c5-99ed-3502427e9a6e None None] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:37:58.299094 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f3dcd0-2418-40c5-99ed-3502427e9a6e None None] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:37:58.332772 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ba800f-b570-4e52-91a4-fd4cfda66a76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:37:58.333480 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ba800f-b570-4e52-91a4-fd4cfda66a76 None admin] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:37:58.333764 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ba800f-b570-4e52-91a4-fd4cfda66a76 None admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:37:58.334060 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ba800f-b570-4e52-91a4-fd4cfda66a76 None admin] PATH_INFO: `/v3/roles/3fac71a39f3f4ed198a3579064c30268` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:37:58.335513 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43ba800f-b570-4e52-91a4-fd4cfda66a76 None admin] RBAC: Authorizing `identity:get_role(role_id=3fac71a39f3f4ed198a3579064c30268)` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:37:58.336878 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43ba800f-b570-4e52-91a4-fd4cfda66a76 None admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:37:58.338238 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 565/1113] 10.222.0.20 () {56 vars in 1177 bytes} [Mon Aug 1 16:37:58 2022] GET /identity/v3/roles/3fac71a39f3f4ed198a3579064c30268 => generated 223 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:37:58.343869 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d73db2cd-3c6e-474c-bc18-37e8f1869649 None None] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:37:58.440215 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d73db2cd-3c6e-474c-bc18-37e8f1869649 None None] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:37:58.473179 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-470e428f-78f2-4633-af9a-8cf7b3b9d6ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:37:58.473784 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470e428f-78f2-4633-af9a-8cf7b3b9d6ab None admin] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:37:58.474179 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470e428f-78f2-4633-af9a-8cf7b3b9d6ab None admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:37:58.474357 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470e428f-78f2-4633-af9a-8cf7b3b9d6ab None admin] PATH_INFO: `/v3/users/1eff0d9072d54693933df237605f22e8` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:37:58.484158 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG dbcounter [-] [87057] Writing DB stats keystone:SELECT=100 {{(pid=87057) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:37:58.487297 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-470e428f-78f2-4633-af9a-8cf7b3b9d6ab None admin] RBAC: Authorizing `identity:get_user(user_id=1eff0d9072d54693933df237605f22e8)` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:37:58.489790 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-470e428f-78f2-4633-af9a-8cf7b3b9d6ab None admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:37:58.491431 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 549/1114] 10.222.0.20 () {56 vars in 1177 bytes} [Mon Aug 1 16:37:58 2022] GET /identity/v3/users/1eff0d9072d54693933df237605f22e8 => generated 286 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:37:58.497144 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ba800f-b570-4e52-91a4-fd4cfda66a76 None admin] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:37:58.522263 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG dbcounter [-] [87058] Writing DB stats keystone:SELECT=100 {{(pid=87058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:37:58.595939 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ba800f-b570-4e52-91a4-fd4cfda66a76 None admin] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:37:58.629764 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b756ddf7-8cc2-43a4-b9e6-55f941527b86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:37:58.630393 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b756ddf7-8cc2-43a4-b9e6-55f941527b86 None admin] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:37:58.630674 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b756ddf7-8cc2-43a4-b9e6-55f941527b86 None admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:37:58.630948 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b756ddf7-8cc2-43a4-b9e6-55f941527b86 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:37:58.696732 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b756ddf7-8cc2-43a4-b9e6-55f941527b86 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:37:58.698147 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b756ddf7-8cc2-43a4-b9e6-55f941527b86 None admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:37:58.720278 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG dbcounter [-] [87058] Writing DB stats keystone:SELECT=100 {{(pid=87058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:37:58.793668 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 566/1115] 10.222.0.20 () {58 vars in 1282 bytes} [Mon Aug 1 16:37:58 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 297 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 01 16:37:58.799699 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-470e428f-78f2-4633-af9a-8cf7b3b9d6ab None admin] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:37:58.896746 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-470e428f-78f2-4633-af9a-8cf7b3b9d6ab None admin] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:37:58.933100 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcda634c-2cf1-4dfe-950e-1a220fd36881 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:37:58.933728 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcda634c-2cf1-4dfe-950e-1a220fd36881 None admin] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:37:58.934003 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcda634c-2cf1-4dfe-950e-1a220fd36881 None admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:37:58.934301 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcda634c-2cf1-4dfe-950e-1a220fd36881 None admin] PATH_INFO: `/v3/projects/585c4683da3243eea2b4695c50946bdd` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:37:58.935435 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcda634c-2cf1-4dfe-950e-1a220fd36881 None admin] RBAC: Authorizing `identity:get_project(project_id=585c4683da3243eea2b4695c50946bdd)` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:37:58.936856 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcda634c-2cf1-4dfe-950e-1a220fd36881 None admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:37:58.938522 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 550/1116] 10.222.0.20 () {56 vars in 1186 bytes} [Mon Aug 1 16:37:58 2022] GET /identity/v3/projects/585c4683da3243eea2b4695c50946bdd => generated 311 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:37:58.944852 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b756ddf7-8cc2-43a4-b9e6-55f941527b86 None admin] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:37:59.041894 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b756ddf7-8cc2-43a4-b9e6-55f941527b86 None admin] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:37:59.063771 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG dbcounter [-] [87058] Writing DB stats keystone:SELECT=100 {{(pid=87058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:37:59.075124 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76be6fea-d531-46af-a093-ede0afb8f507 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:37:59.075725 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76be6fea-d531-46af-a093-ede0afb8f507 None admin] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:37:59.092197 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76be6fea-d531-46af-a093-ede0afb8f507 None admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:37:59.092197 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76be6fea-d531-46af-a093-ede0afb8f507 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:37:59.092197 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76be6fea-d531-46af-a093-ede0afb8f507 None admin] RBAC: Adding query filter params (role.id=3fac71a39f3f4ed198a3579064c30268, scope.project.id=585c4683da3243eea2b4695c50946bdd, user.id=1eff0d9072d54693933df237605f22e8) {{(pid=87058) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:37:59.092197 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76be6fea-d531-46af-a093-ede0afb8f507 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:37:59.092197 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76be6fea-d531-46af-a093-ede0afb8f507 None admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:37:59.093106 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 567/1117] 10.222.0.20 () {56 vars in 1374 bytes} [Mon Aug 1 16:37:58 2022] GET /identity/v3/role_assignments?user.id=1eff0d9072d54693933df237605f22e8&scope.project.id=585c4683da3243eea2b4695c50946bdd&role.id=3fac71a39f3f4ed198a3579064c30268 => generated 259 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:38:01.035447 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a7a56e-27b9-4db0-b91e-c0c3fbb2c515 None None] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:38:01.036096 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a7a56e-27b9-4db0-b91e-c0c3fbb2c515 None None] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:38:01.036096 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a7a56e-27b9-4db0-b91e-c0c3fbb2c515 None None] PATH_INFO: `/` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:38:01.037337 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 551/1118] 10.222.0.20 () {54 vars in 926 bytes} [Mon Aug 1 16:38:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 01 16:38:01.042350 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b2ad6f-22a1-4fb7-a124-6b39d6d0a072 None None] REQUEST_METHOD: `POST` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:38:01.042781 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b2ad6f-22a1-4fb7-a124-6b39d6d0a072 None None] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:38:01.042909 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b2ad6f-22a1-4fb7-a124-6b39d6d0a072 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:38:01.061292 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.auth.core [None req-96b2ad6f-22a1-4fb7-a124-6b39d6d0a072 None None] MFA Rules not processed for user `1007f8e379b74e169b98fb7c3c0ba5b4`. Rule list: `[]` (Enabled: `True`). {{(pid=87058) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:38:01.062511 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.fernet_utils [None req-96b2ad6f-22a1-4fb7-a124-6b39d6d0a072 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:38:01.128194 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 568/1119] 10.222.0.20 () {58 vars in 1023 bytes} [Mon Aug 1 16:38:01 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 01 16:38:01.150481 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a7a56e-27b9-4db0-b91e-c0c3fbb2c515 None None] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:38:01.215564 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG dbcounter [-] [87057] Writing DB stats keystone:SELECT=100 {{(pid=87057) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:38:01.249864 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a7a56e-27b9-4db0-b91e-c0c3fbb2c515 None None] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:38:01.282457 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a57f1b9-ece2-43ff-ac38-9e0f85c6547f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:38:01.283077 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a57f1b9-ece2-43ff-ac38-9e0f85c6547f None admin] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:38:01.283353 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a57f1b9-ece2-43ff-ac38-9e0f85c6547f None admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:38:01.283623 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a57f1b9-ece2-43ff-ac38-9e0f85c6547f None admin] PATH_INFO: `/v3/roles/3fac71a39f3f4ed198a3579064c30268` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:38:01.285583 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a57f1b9-ece2-43ff-ac38-9e0f85c6547f None admin] RBAC: Authorizing `identity:get_role(role_id=3fac71a39f3f4ed198a3579064c30268)` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:38:01.287037 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a57f1b9-ece2-43ff-ac38-9e0f85c6547f None admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:38:01.288350 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 552/1120] 10.222.0.20 () {56 vars in 1177 bytes} [Mon Aug 1 16:38:01 2022] GET /identity/v3/roles/3fac71a39f3f4ed198a3579064c30268 => generated 223 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:38:01.294043 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96b2ad6f-22a1-4fb7-a124-6b39d6d0a072 None None] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:38:01.390737 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96b2ad6f-22a1-4fb7-a124-6b39d6d0a072 None None] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:38:01.406911 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG dbcounter [-] [87058] Writing DB stats keystone:SELECT=100 {{(pid=87058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:38:01.423778 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4041a413-4fb2-4756-8380-2ccaee38c5b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:38:01.424405 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4041a413-4fb2-4756-8380-2ccaee38c5b8 None admin] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:38:01.424683 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4041a413-4fb2-4756-8380-2ccaee38c5b8 None admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:38:01.424955 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4041a413-4fb2-4756-8380-2ccaee38c5b8 None admin] PATH_INFO: `/v3/users/1eff0d9072d54693933df237605f22e8` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:38:01.426248 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4041a413-4fb2-4756-8380-2ccaee38c5b8 None admin] RBAC: Authorizing `identity:get_user(user_id=1eff0d9072d54693933df237605f22e8)` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:38:01.427767 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4041a413-4fb2-4756-8380-2ccaee38c5b8 None admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:38:01.429363 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 569/1121] 10.222.0.20 () {56 vars in 1177 bytes} [Mon Aug 1 16:38:01 2022] GET /identity/v3/users/1eff0d9072d54693933df237605f22e8 => generated 286 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:38:01.435137 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a57f1b9-ece2-43ff-ac38-9e0f85c6547f None admin] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:38:01.532788 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a57f1b9-ece2-43ff-ac38-9e0f85c6547f None admin] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:38:01.565340 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4b2a3c1-b80d-4fc0-8d22-c01834ad3237 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:38:01.565963 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b2a3c1-b80d-4fc0-8d22-c01834ad3237 None admin] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:38:01.566362 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b2a3c1-b80d-4fc0-8d22-c01834ad3237 None admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:38:01.567125 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b2a3c1-b80d-4fc0-8d22-c01834ad3237 None admin] PATH_INFO: `/v3/projects/585c4683da3243eea2b4695c50946bdd` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:38:01.568274 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4b2a3c1-b80d-4fc0-8d22-c01834ad3237 None admin] RBAC: Authorizing `identity:get_project(project_id=585c4683da3243eea2b4695c50946bdd)` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:38:01.569703 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4b2a3c1-b80d-4fc0-8d22-c01834ad3237 None admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:38:01.571372 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 553/1122] 10.222.0.20 () {56 vars in 1186 bytes} [Mon Aug 1 16:38:01 2022] GET /identity/v3/projects/585c4683da3243eea2b4695c50946bdd => generated 311 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:38:01.577115 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4041a413-4fb2-4756-8380-2ccaee38c5b8 None admin] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:38:01.674013 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4041a413-4fb2-4756-8380-2ccaee38c5b8 None admin] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:38:01.707210 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f90d25a-6b30-4506-a1bc-df952ceaf555 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:38:01.707870 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f90d25a-6b30-4506-a1bc-df952ceaf555 None admin] REQUEST_METHOD: `PUT` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:38:01.708150 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f90d25a-6b30-4506-a1bc-df952ceaf555 None admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:38:01.708423 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f90d25a-6b30-4506-a1bc-df952ceaf555 None admin] PATH_INFO: `/v3/projects/585c4683da3243eea2b4695c50946bdd/users/1eff0d9072d54693933df237605f22e8/roles/3fac71a39f3f4ed198a3579064c30268` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:38:01.710685 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f90d25a-6b30-4506-a1bc-df952ceaf555 None admin] RBAC: Authorizing `identity:create_grant(project_id=585c4683da3243eea2b4695c50946bdd, user_id=1eff0d9072d54693933df237605f22e8, role_id=3fac71a39f3f4ed198a3579064c30268)` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:38:01.712106 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f90d25a-6b30-4506-a1bc-df952ceaf555 None admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:38:01.782491 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 570/1123] 10.222.0.20 () {58 vars in 1439 bytes} [Mon Aug 1 16:38:01 2022] PUT /identity/v3/projects/585c4683da3243eea2b4695c50946bdd/users/1eff0d9072d54693933df237605f22e8/roles/3fac71a39f3f4ed198a3579064c30268 => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 16:38:03.729026 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5cf3480-70ae-428f-9d18-d56af1c72f2f None None] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:38:03.729575 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5cf3480-70ae-428f-9d18-d56af1c72f2f None None] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:38:03.729575 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5cf3480-70ae-428f-9d18-d56af1c72f2f None None] PATH_INFO: `/` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:38:03.730944 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 554/1124] 10.222.0.20 () {54 vars in 926 bytes} [Mon Aug 1 16:38:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 16:38:03.735909 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18819f89-15e7-48ff-8147-f2cc5c0a250d None None] REQUEST_METHOD: `POST` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:38:03.736299 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18819f89-15e7-48ff-8147-f2cc5c0a250d None None] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:38:03.736451 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18819f89-15e7-48ff-8147-f2cc5c0a250d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:38:03.754431 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.auth.core [None req-18819f89-15e7-48ff-8147-f2cc5c0a250d None None] MFA Rules not processed for user `1007f8e379b74e169b98fb7c3c0ba5b4`. Rule list: `[]` (Enabled: `True`). {{(pid=87058) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:38:03.755619 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.fernet_utils [None req-18819f89-15e7-48ff-8147-f2cc5c0a250d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:38:03.795647 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG dbcounter [-] [87058] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:38:03.821143 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 571/1125] 10.222.0.20 () {58 vars in 1023 bytes} [Mon Aug 1 16:38:03 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 01 16:38:03.844273 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5cf3480-70ae-428f-9d18-d56af1c72f2f None None] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:38:03.847175 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG dbcounter [-] [87057] Writing DB stats keystone:SELECT=100 {{(pid=87057) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:38:03.946160 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5cf3480-70ae-428f-9d18-d56af1c72f2f None None] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:38:03.980098 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c800f44-4917-4e19-b2ac-e890c794fbac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:38:03.980669 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c800f44-4917-4e19-b2ac-e890c794fbac None admin] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:38:03.980950 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c800f44-4917-4e19-b2ac-e890c794fbac None admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:38:03.981224 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c800f44-4917-4e19-b2ac-e890c794fbac None admin] PATH_INFO: `/v3/roles/3fac71a39f3f4ed198a3579064c30268` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:38:03.982690 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c800f44-4917-4e19-b2ac-e890c794fbac None admin] RBAC: Authorizing `identity:get_role(role_id=3fac71a39f3f4ed198a3579064c30268)` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:38:03.984069 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c800f44-4917-4e19-b2ac-e890c794fbac None admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:38:03.985402 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 555/1126] 10.222.0.20 () {56 vars in 1177 bytes} [Mon Aug 1 16:38:03 2022] GET /identity/v3/roles/3fac71a39f3f4ed198a3579064c30268 => generated 223 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:38:03.991478 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18819f89-15e7-48ff-8147-f2cc5c0a250d None None] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:38:04.086279 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18819f89-15e7-48ff-8147-f2cc5c0a250d None None] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:38:04.118976 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a09722-3a47-4e72-b375-239e49a37403 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:38:04.119597 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a09722-3a47-4e72-b375-239e49a37403 None admin] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:38:04.119870 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a09722-3a47-4e72-b375-239e49a37403 None admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:38:04.120138 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a09722-3a47-4e72-b375-239e49a37403 None admin] PATH_INFO: `/v3/users/1eff0d9072d54693933df237605f22e8` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:38:04.121276 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74a09722-3a47-4e72-b375-239e49a37403 None admin] RBAC: Authorizing `identity:get_user(user_id=1eff0d9072d54693933df237605f22e8)` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:38:04.122736 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74a09722-3a47-4e72-b375-239e49a37403 None admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:38:04.124298 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 572/1127] 10.222.0.20 () {56 vars in 1177 bytes} [Mon Aug 1 16:38:03 2022] GET /identity/v3/users/1eff0d9072d54693933df237605f22e8 => generated 286 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:38:04.130515 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c800f44-4917-4e19-b2ac-e890c794fbac None admin] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:38:04.198212 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG dbcounter [-] [87057] Writing DB stats keystone:SELECT=100 {{(pid=87057) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:38:04.228332 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c800f44-4917-4e19-b2ac-e890c794fbac None admin] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:38:04.261172 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c6ef620-072e-4627-941a-754341846adb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:38:04.261720 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6ef620-072e-4627-941a-754341846adb None admin] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:38:04.262003 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6ef620-072e-4627-941a-754341846adb None admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:38:04.262310 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6ef620-072e-4627-941a-754341846adb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:38:04.327575 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c6ef620-072e-4627-941a-754341846adb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:38:04.329007 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c6ef620-072e-4627-941a-754341846adb None admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:38:04.393081 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG dbcounter [-] [87057] Writing DB stats keystone:SELECT=100 {{(pid=87057) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:38:04.423046 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 556/1128] 10.222.0.20 () {58 vars in 1282 bytes} [Mon Aug 1 16:38:04 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 293 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 01 16:38:04.429115 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a09722-3a47-4e72-b375-239e49a37403 None admin] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:38:04.471903 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG dbcounter [-] [87058] Writing DB stats keystone:SELECT=100 {{(pid=87058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:38:04.527416 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a09722-3a47-4e72-b375-239e49a37403 None admin] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:38:04.560211 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fddc75c-3de5-48c7-9f35-c14da448aff9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:38:04.560841 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fddc75c-3de5-48c7-9f35-c14da448aff9 None admin] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:38:04.561120 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fddc75c-3de5-48c7-9f35-c14da448aff9 None admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:38:04.561391 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fddc75c-3de5-48c7-9f35-c14da448aff9 None admin] PATH_INFO: `/v3/projects/585c4683da3243eea2b4695c50946bdd` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:38:04.562566 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fddc75c-3de5-48c7-9f35-c14da448aff9 None admin] RBAC: Authorizing `identity:get_project(project_id=585c4683da3243eea2b4695c50946bdd)` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:38:04.563999 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fddc75c-3de5-48c7-9f35-c14da448aff9 None admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:38:04.565652 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 573/1129] 10.222.0.20 () {56 vars in 1186 bytes} [Mon Aug 1 16:38:04 2022] GET /identity/v3/projects/585c4683da3243eea2b4695c50946bdd => generated 311 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:38:04.572460 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c6ef620-072e-4627-941a-754341846adb None admin] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:38:04.668812 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c6ef620-072e-4627-941a-754341846adb None admin] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:38:04.701165 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-881b0042-c2aa-4fed-9682-5fef4c707d23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:38:04.701759 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881b0042-c2aa-4fed-9682-5fef4c707d23 None admin] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:38:04.702062 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881b0042-c2aa-4fed-9682-5fef4c707d23 None admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:38:04.702338 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881b0042-c2aa-4fed-9682-5fef4c707d23 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:38:04.703110 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-881b0042-c2aa-4fed-9682-5fef4c707d23 None admin] RBAC: Adding query filter params (role.id=3fac71a39f3f4ed198a3579064c30268, scope.project.id=585c4683da3243eea2b4695c50946bdd, user.id=1eff0d9072d54693933df237605f22e8) {{(pid=87057) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:38:04.703421 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-881b0042-c2aa-4fed-9682-5fef4c707d23 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:38:04.704826 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-881b0042-c2aa-4fed-9682-5fef4c707d23 None admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:38:04.710971 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 557/1130] 10.222.0.20 () {56 vars in 1374 bytes} [Mon Aug 1 16:38:04 2022] GET /identity/v3/role_assignments?user.id=1eff0d9072d54693933df237605f22e8&scope.project.id=585c4683da3243eea2b4695c50946bdd&role.id=3fac71a39f3f4ed198a3579064c30268 => generated 608 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:38:06.717514 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0017fc95-0038-4491-8eab-19c8e13d08fa None None] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:38:06.718110 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0017fc95-0038-4491-8eab-19c8e13d08fa None None] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:38:06.718110 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0017fc95-0038-4491-8eab-19c8e13d08fa None None] PATH_INFO: `/` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:38:06.719431 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 574/1131] 10.222.0.20 () {54 vars in 926 bytes} [Mon Aug 1 16:38:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 16:38:06.724650 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f06ec46-6a1a-46f1-b0ac-761c48046f79 None None] REQUEST_METHOD: `POST` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:38:06.725069 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f06ec46-6a1a-46f1-b0ac-761c48046f79 None None] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:38:06.725208 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f06ec46-6a1a-46f1-b0ac-761c48046f79 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:38:06.743386 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.auth.core [None req-2f06ec46-6a1a-46f1-b0ac-761c48046f79 None None] MFA Rules not processed for user `1007f8e379b74e169b98fb7c3c0ba5b4`. Rule list: `[]` (Enabled: `True`). {{(pid=87057) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:38:06.744603 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.fernet_utils [None req-2f06ec46-6a1a-46f1-b0ac-761c48046f79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87057) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:38:06.770938 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG dbcounter [-] [87057] Writing DB stats keystone:SELECT=100 {{(pid=87057) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:38:06.810196 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 558/1132] 10.222.0.20 () {58 vars in 1023 bytes} [Mon Aug 1 16:38:06 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 01 16:38:06.833741 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0017fc95-0038-4491-8eab-19c8e13d08fa None None] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:38:06.935080 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0017fc95-0038-4491-8eab-19c8e13d08fa None None] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:38:06.946138 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG dbcounter [-] [87058] Writing DB stats keystone:SELECT=100 {{(pid=87058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:38:06.968465 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a10a6f-c2eb-46d7-959f-00fc930022aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:38:06.969037 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a10a6f-c2eb-46d7-959f-00fc930022aa None admin] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:38:06.969317 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a10a6f-c2eb-46d7-959f-00fc930022aa None admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:38:06.969586 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a10a6f-c2eb-46d7-959f-00fc930022aa None admin] PATH_INFO: `/v3/domains/default` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:38:06.970752 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40a10a6f-c2eb-46d7-959f-00fc930022aa None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:38:06.972227 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40a10a6f-c2eb-46d7-959f-00fc930022aa None admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:38:06.973899 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 575/1133] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Aug 1 16:38:06 2022] GET /identity/v3/domains/default => generated 199 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:38:06.980015 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f06ec46-6a1a-46f1-b0ac-761c48046f79 None None] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:38:07.075687 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f06ec46-6a1a-46f1-b0ac-761c48046f79 None None] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:38:07.108242 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb71fc6-6a30-4801-ba35-d5e52ab8e4a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:38:07.108841 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb71fc6-6a30-4801-ba35-d5e52ab8e4a7 None admin] REQUEST_METHOD: `POST` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:38:07.109118 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb71fc6-6a30-4801-ba35-d5e52ab8e4a7 None admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:38:07.109387 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb71fc6-6a30-4801-ba35-d5e52ab8e4a7 None admin] PATH_INFO: `/v3/projects` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:38:07.110093 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fb71fc6-6a30-4801-ba35-d5e52ab8e4a7 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:38:07.111513 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fb71fc6-6a30-4801-ba35-d5e52ab8e4a7 None admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:38:07.197254 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 559/1134] 10.222.0.20 () {60 vars in 1141 bytes} [Mon Aug 1 16:38:06 2022] POST /identity/v3/projects => generated 311 bytes in 218 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 16:38:09.201040 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8909dd16-e84d-4191-909e-3f7b3cffeb8c None None] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:38:09.201552 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8909dd16-e84d-4191-909e-3f7b3cffeb8c None None] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:38:09.201552 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8909dd16-e84d-4191-909e-3f7b3cffeb8c None None] PATH_INFO: `/` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:38:09.202877 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 576/1135] 10.222.0.20 () {54 vars in 926 bytes} [Mon Aug 1 16:38:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 16:38:09.208141 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d02bbf2-9c2f-47d0-bbaa-4f0ea7291513 None None] REQUEST_METHOD: `POST` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:38:09.208568 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d02bbf2-9c2f-47d0-bbaa-4f0ea7291513 None None] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:38:09.208696 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d02bbf2-9c2f-47d0-bbaa-4f0ea7291513 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:38:09.226625 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.auth.core [None req-7d02bbf2-9c2f-47d0-bbaa-4f0ea7291513 None None] MFA Rules not processed for user `1007f8e379b74e169b98fb7c3c0ba5b4`. Rule list: `[]` (Enabled: `True`). {{(pid=87057) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:38:09.227819 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.fernet_utils [None req-7d02bbf2-9c2f-47d0-bbaa-4f0ea7291513 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87057) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:38:09.238733 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG dbcounter [-] [87057] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87057) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:38:09.295346 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 560/1136] 10.222.0.20 () {58 vars in 1023 bytes} [Mon Aug 1 16:38:09 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 01 16:38:09.319714 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8909dd16-e84d-4191-909e-3f7b3cffeb8c None None] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:38:09.419166 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8909dd16-e84d-4191-909e-3f7b3cffeb8c None None] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:38:09.451976 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e35a530-9874-48d7-8e60-e03dedc89b9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:38:09.452536 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e35a530-9874-48d7-8e60-e03dedc89b9d None admin] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:38:09.452809 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e35a530-9874-48d7-8e60-e03dedc89b9d None admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:38:09.453077 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e35a530-9874-48d7-8e60-e03dedc89b9d None admin] PATH_INFO: `/v3/domains/default` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:38:09.454395 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e35a530-9874-48d7-8e60-e03dedc89b9d None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:38:09.455891 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e35a530-9874-48d7-8e60-e03dedc89b9d None admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:38:09.457694 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 577/1137] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Aug 1 16:38:09 2022] GET /identity/v3/domains/default => generated 199 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:38:09.463490 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d02bbf2-9c2f-47d0-bbaa-4f0ea7291513 None None] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:38:09.562933 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d02bbf2-9c2f-47d0-bbaa-4f0ea7291513 None None] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:38:09.595049 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91677ecc-995a-4f57-8fb4-93919e6f0e01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:38:09.595646 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91677ecc-995a-4f57-8fb4-93919e6f0e01 None admin] REQUEST_METHOD: `POST` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:38:09.595921 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91677ecc-995a-4f57-8fb4-93919e6f0e01 None admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:38:09.596189 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91677ecc-995a-4f57-8fb4-93919e6f0e01 None admin] PATH_INFO: `/v3/users` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:38:09.596860 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91677ecc-995a-4f57-8fb4-93919e6f0e01 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:38:09.598802 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91677ecc-995a-4f57-8fb4-93919e6f0e01 None admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:38:09.682461 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 561/1138] 10.222.0.20 () {60 vars in 1132 bytes} [Mon Aug 1 16:38:09 2022] POST /identity/v3/users => generated 286 bytes in 219 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 16:38:11.758427 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6fd573b-bfa8-4d55-8595-b676a0e1671b None None] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:38:11.758996 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6fd573b-bfa8-4d55-8595-b676a0e1671b None None] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:38:11.758996 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6fd573b-bfa8-4d55-8595-b676a0e1671b None None] PATH_INFO: `/` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:38:11.760316 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 578/1139] 10.222.0.20 () {54 vars in 926 bytes} [Mon Aug 1 16:38:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 16:38:11.765863 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e840bc4f-35e8-4c22-a97d-bbc01602dd52 None None] REQUEST_METHOD: `POST` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:38:11.766328 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e840bc4f-35e8-4c22-a97d-bbc01602dd52 None None] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:38:11.766431 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e840bc4f-35e8-4c22-a97d-bbc01602dd52 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:38:11.772629 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG dbcounter [-] [87057] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87057) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:38:11.784346 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.auth.core [None req-e840bc4f-35e8-4c22-a97d-bbc01602dd52 None None] MFA Rules not processed for user `1007f8e379b74e169b98fb7c3c0ba5b4`. Rule list: `[]` (Enabled: `True`). {{(pid=87057) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:38:11.786191 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.fernet_utils [None req-e840bc4f-35e8-4c22-a97d-bbc01602dd52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87057) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:38:11.851551 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 562/1140] 10.222.0.20 () {58 vars in 1023 bytes} [Mon Aug 1 16:38:11 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 01 16:38:11.874722 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6fd573b-bfa8-4d55-8595-b676a0e1671b None None] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:38:11.923686 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG dbcounter [-] [87058] Writing DB stats keystone:SELECT=100 {{(pid=87058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:38:11.977981 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6fd573b-bfa8-4d55-8595-b676a0e1671b None None] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:38:12.011839 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a539eab-d4b3-42d1-a09e-24c3a39fb2eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:38:12.012478 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a539eab-d4b3-42d1-a09e-24c3a39fb2eb None admin] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:38:12.012758 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a539eab-d4b3-42d1-a09e-24c3a39fb2eb None admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:38:12.013030 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a539eab-d4b3-42d1-a09e-24c3a39fb2eb None admin] PATH_INFO: `/v3/roles/admin` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:38:12.023272 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a539eab-d4b3-42d1-a09e-24c3a39fb2eb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 16:38:12.028094 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application [None req-1a539eab-d4b3-42d1-a09e-24c3a39fb2eb None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 16:38:12.028094 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 16:38:12.028094 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 16:38:12.028094 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 16:38:12.028094 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 16:38:12.028094 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 16:38:12.028094 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 16:38:12.028094 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 16:38:12.028094 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 16:38:12.028094 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 16:38:12.028094 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 16:38:12.028094 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 16:38:12.028094 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 16:38:12.028094 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 16:38:12.028094 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 16:38:12.028094 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 16:38:12.029267 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 16:38:12.029267 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 16:38:12.029267 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 16:38:12.029267 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 16:38:12.029267 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 16:38:12.029267 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 16:38:12.029267 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 16:38:12.029267 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 16:38:12.029267 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 16:38:12.029267 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 16:38:12.029267 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 16:38:12.029267 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application with Lock( Aug 01 16:38:12.029267 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 16:38:12.029267 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self._enter() Aug 01 16:38:12.029267 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 16:38:12.029267 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 16:38:12.030643 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 16:38:12.030643 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self.creator() Aug 01 16:38:12.030643 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 16:38:12.030643 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application created_value = creator( Aug 01 16:38:12.030643 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 16:38:12.030643 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 16:38:12.030643 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 16:38:12.030643 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 16:38:12.030643 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 16:38:12.030643 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 16:38:12.030643 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 16:38:12.030643 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application Aug 01 16:38:12.030643 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 579/1141] 10.222.0.20 () {56 vars in 1096 bytes} [Mon Aug 1 16:38:11 2022] GET /identity/v3/roles/admin => generated 83 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 16:38:12.035393 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e840bc4f-35e8-4c22-a97d-bbc01602dd52 None None] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:38:12.131258 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e840bc4f-35e8-4c22-a97d-bbc01602dd52 None None] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:38:12.160658 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG dbcounter [-] [87057] Writing DB stats keystone:SELECT=100 {{(pid=87057) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:38:12.165579 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a399f9e7-3b7d-40c6-8e4b-eaa533c6552b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:38:12.174840 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a399f9e7-3b7d-40c6-8e4b-eaa533c6552b None admin] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:38:12.174840 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a399f9e7-3b7d-40c6-8e4b-eaa533c6552b None admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:38:12.174840 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a399f9e7-3b7d-40c6-8e4b-eaa533c6552b None admin] PATH_INFO: `/v3/roles` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:38:12.174840 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a399f9e7-3b7d-40c6-8e4b-eaa533c6552b None admin] RBAC: Adding query filter params (name=admin) {{(pid=87057) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:38:12.174840 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a399f9e7-3b7d-40c6-8e4b-eaa533c6552b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:38:12.174840 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a399f9e7-3b7d-40c6-8e4b-eaa533c6552b None admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:38:12.177196 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 563/1142] 10.222.0.20 () {56 vars in 1099 bytes} [Mon Aug 1 16:38:12 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:38:12.183092 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a539eab-d4b3-42d1-a09e-24c3a39fb2eb None admin] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:38:12.280281 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a539eab-d4b3-42d1-a09e-24c3a39fb2eb None admin] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:38:12.287982 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG dbcounter [-] [87058] Writing DB stats keystone:SELECT=100 {{(pid=87058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:38:12.313054 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7725fde8-adb3-467b-8ac6-2fee21313390 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:38:12.313679 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7725fde8-adb3-467b-8ac6-2fee21313390 None admin] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:38:12.313957 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7725fde8-adb3-467b-8ac6-2fee21313390 None admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:38:12.314259 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7725fde8-adb3-467b-8ac6-2fee21313390 None admin] PATH_INFO: `/v3/users/3cbaa1e7d0384a35b8c71f2a71078013` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:38:12.327886 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7725fde8-adb3-467b-8ac6-2fee21313390 None admin] RBAC: Authorizing `identity:get_user(user_id=3cbaa1e7d0384a35b8c71f2a71078013)` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:38:12.329322 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7725fde8-adb3-467b-8ac6-2fee21313390 None admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:38:12.331015 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 580/1143] 10.222.0.20 () {56 vars in 1177 bytes} [Mon Aug 1 16:38:12 2022] GET /identity/v3/users/3cbaa1e7d0384a35b8c71f2a71078013 => generated 286 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 16:38:12.337539 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a399f9e7-3b7d-40c6-8e4b-eaa533c6552b None admin] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:38:12.435097 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a399f9e7-3b7d-40c6-8e4b-eaa533c6552b None admin] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:38:12.467777 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e8d227-57e2-47f6-9635-05d045841594 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:38:12.468383 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e8d227-57e2-47f6-9635-05d045841594 None admin] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:38:12.468659 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e8d227-57e2-47f6-9635-05d045841594 None admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:38:12.468933 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e8d227-57e2-47f6-9635-05d045841594 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:38:12.528157 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG dbcounter [-] [87057] Writing DB stats keystone:SELECT=100 {{(pid=87057) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:38:12.535919 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9e8d227-57e2-47f6-9635-05d045841594 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:38:12.537299 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9e8d227-57e2-47f6-9635-05d045841594 None admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:38:12.631661 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 564/1144] 10.222.0.20 () {58 vars in 1282 bytes} [Mon Aug 1 16:38:12 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 295 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 01 16:38:12.637601 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7725fde8-adb3-467b-8ac6-2fee21313390 None admin] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:38:12.734996 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7725fde8-adb3-467b-8ac6-2fee21313390 None admin] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:38:12.767239 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e30cfb56-39dc-4a88-9d73-32571720afa5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:38:12.767863 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30cfb56-39dc-4a88-9d73-32571720afa5 None admin] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:38:12.768134 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30cfb56-39dc-4a88-9d73-32571720afa5 None admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:38:12.768398 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30cfb56-39dc-4a88-9d73-32571720afa5 None admin] PATH_INFO: `/v3/projects/f20d7f557f874bba8640147cedc5dfce` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:38:12.770019 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e30cfb56-39dc-4a88-9d73-32571720afa5 None admin] RBAC: Authorizing `identity:get_project(project_id=f20d7f557f874bba8640147cedc5dfce)` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:38:12.771457 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e30cfb56-39dc-4a88-9d73-32571720afa5 None admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:38:12.773091 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 581/1145] 10.222.0.20 () {56 vars in 1186 bytes} [Mon Aug 1 16:38:12 2022] GET /identity/v3/projects/f20d7f557f874bba8640147cedc5dfce => generated 311 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:38:12.779083 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e8d227-57e2-47f6-9635-05d045841594 None admin] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:38:12.870743 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG dbcounter [-] [87057] Writing DB stats keystone:SELECT=100 {{(pid=87057) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:38:12.877065 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e8d227-57e2-47f6-9635-05d045841594 None admin] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:38:12.911421 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add009df-b569-4c5d-be9a-032f910adceb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:38:12.912040 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add009df-b569-4c5d-be9a-032f910adceb None admin] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:38:12.912316 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add009df-b569-4c5d-be9a-032f910adceb None admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:38:12.912586 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add009df-b569-4c5d-be9a-032f910adceb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:38:12.913375 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-add009df-b569-4c5d-be9a-032f910adceb None admin] RBAC: Adding query filter params (role.id=440bb943dd5b46fea038b903b6eaaf81, scope.project.id=f20d7f557f874bba8640147cedc5dfce, user.id=3cbaa1e7d0384a35b8c71f2a71078013) {{(pid=87057) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:38:12.913683 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-add009df-b569-4c5d-be9a-032f910adceb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:38:12.915144 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-add009df-b569-4c5d-be9a-032f910adceb None admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:38:12.920419 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 565/1146] 10.222.0.20 () {56 vars in 1374 bytes} [Mon Aug 1 16:38:12 2022] GET /identity/v3/role_assignments?user.id=3cbaa1e7d0384a35b8c71f2a71078013&scope.project.id=f20d7f557f874bba8640147cedc5dfce&role.id=440bb943dd5b46fea038b903b6eaaf81 => generated 259 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:38:14.879915 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54bf486f-7efa-4bed-8777-a16ed43bbe1e None None] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:38:14.880477 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54bf486f-7efa-4bed-8777-a16ed43bbe1e None None] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:38:14.880477 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54bf486f-7efa-4bed-8777-a16ed43bbe1e None None] PATH_INFO: `/` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:38:14.881822 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 582/1147] 10.222.0.20 () {54 vars in 926 bytes} [Mon Aug 1 16:38:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 16:38:14.886854 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80d992c-7ba6-4942-bd2f-3892cf1b0ce2 None None] REQUEST_METHOD: `POST` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:38:14.887264 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80d992c-7ba6-4942-bd2f-3892cf1b0ce2 None None] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:38:14.887411 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80d992c-7ba6-4942-bd2f-3892cf1b0ce2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:38:14.905724 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.auth.core [None req-c80d992c-7ba6-4942-bd2f-3892cf1b0ce2 None None] MFA Rules not processed for user `1007f8e379b74e169b98fb7c3c0ba5b4`. Rule list: `[]` (Enabled: `True`). {{(pid=87057) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:38:14.906957 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.fernet_utils [None req-c80d992c-7ba6-4942-bd2f-3892cf1b0ce2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87057) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:38:14.972867 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 566/1148] 10.222.0.20 () {58 vars in 1023 bytes} [Mon Aug 1 16:38:14 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 01 16:38:14.995170 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54bf486f-7efa-4bed-8777-a16ed43bbe1e None None] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:38:15.026703 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG dbcounter [-] [87058] Writing DB stats keystone:SELECT=100 {{(pid=87058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:38:15.095224 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54bf486f-7efa-4bed-8777-a16ed43bbe1e None None] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:38:15.127705 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17673c5e-2174-473f-841c-41e0eb279871 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:38:15.128395 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17673c5e-2174-473f-841c-41e0eb279871 None admin] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:38:15.128810 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17673c5e-2174-473f-841c-41e0eb279871 None admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:38:15.129075 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17673c5e-2174-473f-841c-41e0eb279871 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:38:15.138779 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17673c5e-2174-473f-841c-41e0eb279871 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 16:38:15.143730 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application [None req-17673c5e-2174-473f-841c-41e0eb279871 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 16:38:15.143730 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 16:38:15.143730 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 16:38:15.143730 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 16:38:15.143730 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 16:38:15.143730 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 16:38:15.143730 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 16:38:15.143730 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 16:38:15.143730 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 16:38:15.143730 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 16:38:15.143730 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 16:38:15.143730 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 16:38:15.143730 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 16:38:15.143730 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 16:38:15.143730 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 16:38:15.143730 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 16:38:15.145535 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 16:38:15.145535 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 16:38:15.145535 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 16:38:15.145535 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 16:38:15.145535 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 16:38:15.145535 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 16:38:15.145535 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 16:38:15.145535 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 16:38:15.145535 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 16:38:15.145535 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 16:38:15.145535 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 16:38:15.145535 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application with Lock( Aug 01 16:38:15.145535 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 16:38:15.145535 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self._enter() Aug 01 16:38:15.145535 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 16:38:15.145535 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 16:38:15.146802 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 16:38:15.146802 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self.creator() Aug 01 16:38:15.146802 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 16:38:15.146802 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application created_value = creator( Aug 01 16:38:15.146802 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 16:38:15.146802 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 16:38:15.146802 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 16:38:15.146802 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 16:38:15.146802 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 16:38:15.146802 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 16:38:15.146802 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 16:38:15.146802 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application Aug 01 16:38:15.146802 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 583/1149] 10.222.0.20 () {56 vars in 1096 bytes} [Mon Aug 1 16:38:14 2022] GET /identity/v3/roles/admin => generated 83 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 16:38:15.151432 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c80d992c-7ba6-4942-bd2f-3892cf1b0ce2 None None] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:38:15.235417 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG dbcounter [-] [87057] Writing DB stats keystone:SELECT=100 {{(pid=87057) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:38:15.247117 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c80d992c-7ba6-4942-bd2f-3892cf1b0ce2 None None] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:38:15.280525 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a36e45-6cad-4b0a-b45c-d3a778682ea8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:38:15.281134 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a36e45-6cad-4b0a-b45c-d3a778682ea8 None admin] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:38:15.281436 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a36e45-6cad-4b0a-b45c-d3a778682ea8 None admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:38:15.281705 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a36e45-6cad-4b0a-b45c-d3a778682ea8 None admin] PATH_INFO: `/v3/roles` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:38:15.282448 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92a36e45-6cad-4b0a-b45c-d3a778682ea8 None admin] RBAC: Adding query filter params (name=admin) {{(pid=87057) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:38:15.282752 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92a36e45-6cad-4b0a-b45c-d3a778682ea8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:38:15.284110 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92a36e45-6cad-4b0a-b45c-d3a778682ea8 None admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:38:15.292674 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 567/1150] 10.222.0.20 () {56 vars in 1099 bytes} [Mon Aug 1 16:38:15 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:38:15.298469 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17673c5e-2174-473f-841c-41e0eb279871 None admin] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:38:15.387193 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG dbcounter [-] [87058] Writing DB stats keystone:SELECT=100 {{(pid=87058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:38:15.399109 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17673c5e-2174-473f-841c-41e0eb279871 None admin] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:38:15.428955 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f74791e-09b3-4674-9cec-f8ade088d422 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:38:15.429582 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f74791e-09b3-4674-9cec-f8ade088d422 None admin] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:38:15.429870 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f74791e-09b3-4674-9cec-f8ade088d422 None admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:38:15.430168 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f74791e-09b3-4674-9cec-f8ade088d422 None admin] PATH_INFO: `/v3/users/3cbaa1e7d0384a35b8c71f2a71078013` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:38:15.431331 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f74791e-09b3-4674-9cec-f8ade088d422 None admin] RBAC: Authorizing `identity:get_user(user_id=3cbaa1e7d0384a35b8c71f2a71078013)` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:38:15.432747 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f74791e-09b3-4674-9cec-f8ade088d422 None admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:38:15.434377 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 584/1151] 10.222.0.20 () {56 vars in 1177 bytes} [Mon Aug 1 16:38:15 2022] GET /identity/v3/users/3cbaa1e7d0384a35b8c71f2a71078013 => generated 286 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:38:15.439975 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a36e45-6cad-4b0a-b45c-d3a778682ea8 None admin] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:38:15.536295 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a36e45-6cad-4b0a-b45c-d3a778682ea8 None admin] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:38:15.568066 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a4a627f-07fb-41fb-96d4-c5ad551f29f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:38:15.568687 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4a627f-07fb-41fb-96d4-c5ad551f29f2 None admin] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:38:15.568965 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4a627f-07fb-41fb-96d4-c5ad551f29f2 None admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:38:15.569236 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4a627f-07fb-41fb-96d4-c5ad551f29f2 None admin] PATH_INFO: `/v3/projects/f20d7f557f874bba8640147cedc5dfce` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:38:15.570396 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a4a627f-07fb-41fb-96d4-c5ad551f29f2 None admin] RBAC: Authorizing `identity:get_project(project_id=f20d7f557f874bba8640147cedc5dfce)` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:38:15.571810 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a4a627f-07fb-41fb-96d4-c5ad551f29f2 None admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:38:15.573421 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 568/1152] 10.222.0.20 () {56 vars in 1186 bytes} [Mon Aug 1 16:38:15 2022] GET /identity/v3/projects/f20d7f557f874bba8640147cedc5dfce => generated 311 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:38:15.579221 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f74791e-09b3-4674-9cec-f8ade088d422 None admin] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:38:15.685821 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f74791e-09b3-4674-9cec-f8ade088d422 None admin] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:38:15.719321 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76129de6-02b9-4f27-95fb-f773b514af7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:38:15.719975 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76129de6-02b9-4f27-95fb-f773b514af7b None admin] REQUEST_METHOD: `PUT` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:38:15.720250 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76129de6-02b9-4f27-95fb-f773b514af7b None admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:38:15.720519 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76129de6-02b9-4f27-95fb-f773b514af7b None admin] PATH_INFO: `/v3/projects/f20d7f557f874bba8640147cedc5dfce/users/3cbaa1e7d0384a35b8c71f2a71078013/roles/440bb943dd5b46fea038b903b6eaaf81` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:38:15.723032 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76129de6-02b9-4f27-95fb-f773b514af7b None admin] RBAC: Authorizing `identity:create_grant(project_id=f20d7f557f874bba8640147cedc5dfce, user_id=3cbaa1e7d0384a35b8c71f2a71078013, role_id=440bb943dd5b46fea038b903b6eaaf81)` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:38:15.724444 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76129de6-02b9-4f27-95fb-f773b514af7b None admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:38:15.806565 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 585/1153] 10.222.0.20 () {58 vars in 1439 bytes} [Mon Aug 1 16:38:15 2022] PUT /identity/v3/projects/f20d7f557f874bba8640147cedc5dfce/users/3cbaa1e7d0384a35b8c71f2a71078013/roles/440bb943dd5b46fea038b903b6eaaf81 => generated 0 bytes in 228 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 16:38:17.775553 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a6d7be5-c843-4721-845a-e30571628af0 None None] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:38:17.776080 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a6d7be5-c843-4721-845a-e30571628af0 None None] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:38:17.776080 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a6d7be5-c843-4721-845a-e30571628af0 None None] PATH_INFO: `/` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:38:17.777430 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 569/1154] 10.222.0.20 () {54 vars in 926 bytes} [Mon Aug 1 16:38:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 16:38:17.782593 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8fd8c2d-0a9c-437b-8ff8-202407d3f0ce None None] REQUEST_METHOD: `POST` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:38:17.783028 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8fd8c2d-0a9c-437b-8ff8-202407d3f0ce None None] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:38:17.783135 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8fd8c2d-0a9c-437b-8ff8-202407d3f0ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:38:17.800784 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.auth.core [None req-a8fd8c2d-0a9c-437b-8ff8-202407d3f0ce None None] MFA Rules not processed for user `1007f8e379b74e169b98fb7c3c0ba5b4`. Rule list: `[]` (Enabled: `True`). {{(pid=87058) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:38:17.801964 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.fernet_utils [None req-a8fd8c2d-0a9c-437b-8ff8-202407d3f0ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:38:17.815613 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG dbcounter [-] [87058] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:38:17.868201 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 586/1155] 10.222.0.20 () {58 vars in 1023 bytes} [Mon Aug 1 16:38:17 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 01 16:38:17.892187 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a6d7be5-c843-4721-845a-e30571628af0 None None] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:38:17.912475 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG dbcounter [-] [87057] Writing DB stats keystone:SELECT=100 {{(pid=87057) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:38:17.992891 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a6d7be5-c843-4721-845a-e30571628af0 None None] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:38:18.025663 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43312139-44c4-4689-a9eb-6efcc8f7192b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:38:18.026253 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43312139-44c4-4689-a9eb-6efcc8f7192b None admin] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:38:18.026532 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43312139-44c4-4689-a9eb-6efcc8f7192b None admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:38:18.026800 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43312139-44c4-4689-a9eb-6efcc8f7192b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:38:18.036370 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43312139-44c4-4689-a9eb-6efcc8f7192b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 16:38:18.041141 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application [None req-43312139-44c4-4689-a9eb-6efcc8f7192b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 16:38:18.041141 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 16:38:18.041141 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 16:38:18.041141 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 16:38:18.041141 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 16:38:18.041141 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 16:38:18.041141 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 16:38:18.041141 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 16:38:18.041141 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 16:38:18.041141 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 16:38:18.041141 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 16:38:18.041141 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 16:38:18.041141 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 16:38:18.041141 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 16:38:18.041141 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 16:38:18.041141 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 16:38:18.042105 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 16:38:18.042105 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 16:38:18.042105 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 16:38:18.042105 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 16:38:18.042105 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 16:38:18.042105 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 16:38:18.042105 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 16:38:18.042105 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 16:38:18.042105 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 16:38:18.042105 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 16:38:18.042105 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 16:38:18.042105 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application with Lock( Aug 01 16:38:18.042105 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 16:38:18.042105 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self._enter() Aug 01 16:38:18.042105 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 16:38:18.042105 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 16:38:18.043128 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 16:38:18.043128 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self.creator() Aug 01 16:38:18.043128 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 16:38:18.043128 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application created_value = creator( Aug 01 16:38:18.043128 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 16:38:18.043128 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 16:38:18.043128 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 16:38:18.043128 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 16:38:18.043128 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 16:38:18.043128 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 16:38:18.043128 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 16:38:18.043128 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application Aug 01 16:38:18.043128 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 570/1156] 10.222.0.20 () {56 vars in 1096 bytes} [Mon Aug 1 16:38:17 2022] GET /identity/v3/roles/admin => generated 83 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 16:38:18.048392 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8fd8c2d-0a9c-437b-8ff8-202407d3f0ce None None] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:38:18.146699 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8fd8c2d-0a9c-437b-8ff8-202407d3f0ce None None] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:38:18.179796 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e0c04f7-c5ba-4f6f-8e0c-055da8bc65cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:38:18.180403 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0c04f7-c5ba-4f6f-8e0c-055da8bc65cf None admin] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:38:18.180682 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0c04f7-c5ba-4f6f-8e0c-055da8bc65cf None admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:38:18.180955 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0c04f7-c5ba-4f6f-8e0c-055da8bc65cf None admin] PATH_INFO: `/v3/roles` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:38:18.181671 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e0c04f7-c5ba-4f6f-8e0c-055da8bc65cf None admin] RBAC: Adding query filter params (name=admin) {{(pid=87058) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:38:18.181974 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e0c04f7-c5ba-4f6f-8e0c-055da8bc65cf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:38:18.183365 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e0c04f7-c5ba-4f6f-8e0c-055da8bc65cf None admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:38:18.191807 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 587/1157] 10.222.0.20 () {56 vars in 1099 bytes} [Mon Aug 1 16:38:18 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:38:18.197536 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43312139-44c4-4689-a9eb-6efcc8f7192b None admin] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:38:18.274914 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG dbcounter [-] [87057] Writing DB stats keystone:SELECT=100 {{(pid=87057) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:38:18.296636 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43312139-44c4-4689-a9eb-6efcc8f7192b None admin] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:38:18.328852 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab2463c-6ac8-467e-a0e8-11dc925bdbc3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:38:18.329479 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab2463c-6ac8-467e-a0e8-11dc925bdbc3 None admin] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:38:18.329761 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab2463c-6ac8-467e-a0e8-11dc925bdbc3 None admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:38:18.330052 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab2463c-6ac8-467e-a0e8-11dc925bdbc3 None admin] PATH_INFO: `/v3/users/3cbaa1e7d0384a35b8c71f2a71078013` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:38:18.331231 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bab2463c-6ac8-467e-a0e8-11dc925bdbc3 None admin] RBAC: Authorizing `identity:get_user(user_id=3cbaa1e7d0384a35b8c71f2a71078013)` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:38:18.332693 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bab2463c-6ac8-467e-a0e8-11dc925bdbc3 None admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:38:18.334310 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 571/1158] 10.222.0.20 () {56 vars in 1177 bytes} [Mon Aug 1 16:38:18 2022] GET /identity/v3/users/3cbaa1e7d0384a35b8c71f2a71078013 => generated 286 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:38:18.340016 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e0c04f7-c5ba-4f6f-8e0c-055da8bc65cf None admin] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:38:18.355586 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG dbcounter [-] [87058] Writing DB stats keystone:SELECT=100 {{(pid=87058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:38:18.440251 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e0c04f7-c5ba-4f6f-8e0c-055da8bc65cf None admin] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:38:18.473806 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-963def68-409d-4517-8046-fb2af63486cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:38:18.474439 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-963def68-409d-4517-8046-fb2af63486cc None admin] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:38:18.474716 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-963def68-409d-4517-8046-fb2af63486cc None admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:38:18.474985 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-963def68-409d-4517-8046-fb2af63486cc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:38:18.540684 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-963def68-409d-4517-8046-fb2af63486cc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:38:18.542152 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-963def68-409d-4517-8046-fb2af63486cc None admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:38:18.555617 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG dbcounter [-] [87058] Writing DB stats keystone:SELECT=100 {{(pid=87058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:38:18.636800 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 588/1159] 10.222.0.20 () {58 vars in 1282 bytes} [Mon Aug 1 16:38:18 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 297 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 01 16:38:18.643089 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab2463c-6ac8-467e-a0e8-11dc925bdbc3 None admin] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:38:18.740184 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab2463c-6ac8-467e-a0e8-11dc925bdbc3 None admin] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:38:18.773489 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec6de593-fdbd-4f65-847f-b8c45ced5059 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:38:18.774139 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6de593-fdbd-4f65-847f-b8c45ced5059 None admin] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:38:18.774422 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6de593-fdbd-4f65-847f-b8c45ced5059 None admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:38:18.774698 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6de593-fdbd-4f65-847f-b8c45ced5059 None admin] PATH_INFO: `/v3/projects/f20d7f557f874bba8640147cedc5dfce` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:38:18.775855 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec6de593-fdbd-4f65-847f-b8c45ced5059 None admin] RBAC: Authorizing `identity:get_project(project_id=f20d7f557f874bba8640147cedc5dfce)` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:38:18.777284 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec6de593-fdbd-4f65-847f-b8c45ced5059 None admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:38:18.778942 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 572/1160] 10.222.0.20 () {56 vars in 1186 bytes} [Mon Aug 1 16:38:18 2022] GET /identity/v3/projects/f20d7f557f874bba8640147cedc5dfce => generated 311 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 16:38:18.785496 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-963def68-409d-4517-8046-fb2af63486cc None admin] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:38:18.879813 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-963def68-409d-4517-8046-fb2af63486cc None admin] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:38:18.892166 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG dbcounter [-] [87058] Writing DB stats keystone:SELECT=100 {{(pid=87058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:38:18.913544 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c343f79-342d-4a94-a011-2f92fd604b1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:38:18.914179 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c343f79-342d-4a94-a011-2f92fd604b1b None admin] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:38:18.914458 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c343f79-342d-4a94-a011-2f92fd604b1b None admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:38:18.914726 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c343f79-342d-4a94-a011-2f92fd604b1b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:38:18.915502 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c343f79-342d-4a94-a011-2f92fd604b1b None admin] RBAC: Adding query filter params (role.id=440bb943dd5b46fea038b903b6eaaf81, scope.project.id=f20d7f557f874bba8640147cedc5dfce, user.id=3cbaa1e7d0384a35b8c71f2a71078013) {{(pid=87058) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:38:18.915806 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c343f79-342d-4a94-a011-2f92fd604b1b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:38:18.917218 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c343f79-342d-4a94-a011-2f92fd604b1b None admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:38:18.922909 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 589/1161] 10.222.0.20 () {56 vars in 1374 bytes} [Mon Aug 1 16:38:18 2022] GET /identity/v3/role_assignments?user.id=3cbaa1e7d0384a35b8c71f2a71078013&scope.project.id=f20d7f557f874bba8640147cedc5dfce&role.id=440bb943dd5b46fea038b903b6eaaf81 => generated 608 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:38:20.914277 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87874c82-f0d8-4cf0-9f8d-b5dcb898401a None None] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:38:20.914739 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87874c82-f0d8-4cf0-9f8d-b5dcb898401a None None] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:38:20.914855 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87874c82-f0d8-4cf0-9f8d-b5dcb898401a None None] PATH_INFO: `/` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:38:20.916180 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 573/1162] 10.222.0.20 () {54 vars in 926 bytes} [Mon Aug 1 16:38:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 16:38:20.921162 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38443e5e-f979-4729-94b1-3b522d48775e None None] REQUEST_METHOD: `POST` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:38:20.921586 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38443e5e-f979-4729-94b1-3b522d48775e None None] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:38:20.921719 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38443e5e-f979-4729-94b1-3b522d48775e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:38:20.940331 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.auth.core [None req-38443e5e-f979-4729-94b1-3b522d48775e None None] MFA Rules not processed for user `1007f8e379b74e169b98fb7c3c0ba5b4`. Rule list: `[]` (Enabled: `True`). {{(pid=87058) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:38:20.941702 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.fernet_utils [None req-38443e5e-f979-4729-94b1-3b522d48775e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:38:21.008021 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 590/1163] 10.222.0.20 () {58 vars in 1023 bytes} [Mon Aug 1 16:38:20 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 01 16:38:21.030445 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87874c82-f0d8-4cf0-9f8d-b5dcb898401a None None] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:38:21.045812 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG dbcounter [-] [87057] Writing DB stats keystone:SELECT=100 {{(pid=87057) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:38:21.130762 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87874c82-f0d8-4cf0-9f8d-b5dcb898401a None None] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:38:21.163195 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-391d1698-b540-45b8-a362-582e73413cab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:38:21.163729 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391d1698-b540-45b8-a362-582e73413cab None admin] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:38:21.164011 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391d1698-b540-45b8-a362-582e73413cab None admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:38:21.164285 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391d1698-b540-45b8-a362-582e73413cab None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:38:21.229001 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-391d1698-b540-45b8-a362-582e73413cab None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:38:21.230472 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-391d1698-b540-45b8-a362-582e73413cab None admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:38:21.244716 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG dbcounter [-] [87057] Writing DB stats keystone:SELECT=100 {{(pid=87057) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:38:21.325491 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 574/1164] 10.222.0.20 () {58 vars in 1282 bytes} [Mon Aug 1 16:38:21 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 296 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 01 16:38:21.331823 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38443e5e-f979-4729-94b1-3b522d48775e None None] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:38:21.429180 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38443e5e-f979-4729-94b1-3b522d48775e None None] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:38:21.436170 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG dbcounter [-] [87058] Writing DB stats keystone:SELECT=100 {{(pid=87058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:38:21.463177 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc7eeda8-e60f-4159-91e0-02959e9cdced None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:38:21.463814 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7eeda8-e60f-4159-91e0-02959e9cdced None admin] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:38:21.464092 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7eeda8-e60f-4159-91e0-02959e9cdced None admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:38:21.464364 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7eeda8-e60f-4159-91e0-02959e9cdced None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:38:21.471411 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc7eeda8-e60f-4159-91e0-02959e9cdced None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:38:21.472910 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc7eeda8-e60f-4159-91e0-02959e9cdced None admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:38:21.484154 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application [None req-dc7eeda8-e60f-4159-91e0-02959e9cdced None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. Aug 01 16:38:21.484154 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 16:38:21.484154 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Aug 01 16:38:21.484154 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Aug 01 16:38:21.484154 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 01 16:38:21.484154 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 01 16:38:21.484154 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 01 16:38:21.484154 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 01 16:38:21.484154 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. Aug 01 16:38:21.484154 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application Aug 01 16:38:21.484154 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 01 16:38:21.484154 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application Aug 01 16:38:21.484154 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 16:38:21.484154 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 16:38:21.484154 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 16:38:21.484154 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 16:38:21.484154 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 16:38:21.485397 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 16:38:21.485397 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 16:38:21.485397 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 16:38:21.485397 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 16:38:21.485397 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 16:38:21.485397 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 16:38:21.485397 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 16:38:21.485397 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 16:38:21.485397 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 16:38:21.485397 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 16:38:21.485397 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 16:38:21.485397 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 16:38:21.485397 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 16:38:21.485397 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 16:38:21.485397 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 16:38:21.485397 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 16:38:21.486333 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 16:38:21.486333 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application with Lock( Aug 01 16:38:21.486333 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 16:38:21.486333 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self._enter() Aug 01 16:38:21.486333 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 16:38:21.486333 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 16:38:21.486333 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 16:38:21.486333 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self.creator() Aug 01 16:38:21.486333 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 16:38:21.486333 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application created_value = creator( Aug 01 16:38:21.486333 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Aug 01 16:38:21.486333 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 16:38:21.486333 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. Aug 01 16:38:21.486333 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application Aug 01 16:38:21.486333 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 591/1165] 10.222.0.20 () {56 vars in 1117 bytes} [Mon Aug 1 16:38:21 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 16:38:21.491383 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-391d1698-b540-45b8-a362-582e73413cab None admin] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:38:21.586941 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-391d1698-b540-45b8-a362-582e73413cab None admin] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:38:21.600198 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG dbcounter [-] [87057] Writing DB stats keystone:SELECT=100 {{(pid=87057) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:38:21.619676 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac4fb56-600c-44e1-9348-fd3f08fb227e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:38:21.620275 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac4fb56-600c-44e1-9348-fd3f08fb227e None admin] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:38:21.620550 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac4fb56-600c-44e1-9348-fd3f08fb227e None admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:38:21.620820 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac4fb56-600c-44e1-9348-fd3f08fb227e None admin] PATH_INFO: `/v3/domains` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:38:21.621494 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aac4fb56-600c-44e1-9348-fd3f08fb227e None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=87057) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:38:21.621792 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aac4fb56-600c-44e1-9348-fd3f08fb227e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:38:21.623187 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aac4fb56-600c-44e1-9348-fd3f08fb227e None admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:38:21.630424 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 575/1166] 10.222.0.20 () {56 vars in 1115 bytes} [Mon Aug 1 16:38:21 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:38:23.591151 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3079f1c0-1c4b-4c2c-86eb-0beab09d3347 None None] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:38:23.591717 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3079f1c0-1c4b-4c2c-86eb-0beab09d3347 None None] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:38:23.591717 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3079f1c0-1c4b-4c2c-86eb-0beab09d3347 None None] PATH_INFO: `/` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:38:23.593068 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 592/1167] 10.222.0.20 () {54 vars in 926 bytes} [Mon Aug 1 16:38:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 16:38:23.598639 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542ebc5d-13aa-4d65-a9a7-7e081e3c9ad5 None None] REQUEST_METHOD: `POST` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:38:23.599066 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542ebc5d-13aa-4d65-a9a7-7e081e3c9ad5 None None] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:38:23.600162 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542ebc5d-13aa-4d65-a9a7-7e081e3c9ad5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:38:23.618736 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.auth.core [None req-542ebc5d-13aa-4d65-a9a7-7e081e3c9ad5 None None] MFA Rules not processed for user `1007f8e379b74e169b98fb7c3c0ba5b4`. Rule list: `[]` (Enabled: `True`). {{(pid=87057) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:38:23.619931 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.fernet_utils [None req-542ebc5d-13aa-4d65-a9a7-7e081e3c9ad5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87057) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:38:23.684873 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 576/1168] 10.222.0.20 () {58 vars in 1023 bytes} [Mon Aug 1 16:38:23 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 01 16:38:23.707943 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3079f1c0-1c4b-4c2c-86eb-0beab09d3347 None None] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:38:23.805896 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3079f1c0-1c4b-4c2c-86eb-0beab09d3347 None None] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:38:23.839148 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe60b69-baf0-4f2b-91e0-c93ce8a20102 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:38:23.839749 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe60b69-baf0-4f2b-91e0-c93ce8a20102 None admin] REQUEST_METHOD: `POST` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:38:23.840026 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe60b69-baf0-4f2b-91e0-c93ce8a20102 None admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:38:23.840297 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe60b69-baf0-4f2b-91e0-c93ce8a20102 None admin] PATH_INFO: `/v3/domains` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:38:23.841005 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfe60b69-baf0-4f2b-91e0-c93ce8a20102 None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:38:23.842437 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfe60b69-baf0-4f2b-91e0-c93ce8a20102 None admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:38:23.932223 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 593/1169] 10.222.0.20 () {60 vars in 1138 bytes} [Mon Aug 1 16:38:23 2022] POST /identity/v3/domains => generated 267 bytes in 225 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 16:38:25.950048 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f92ac2-42b2-4be8-8b39-a8474ddda6e5 None None] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:38:25.950501 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f92ac2-42b2-4be8-8b39-a8474ddda6e5 None None] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:38:25.950613 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f92ac2-42b2-4be8-8b39-a8474ddda6e5 None None] PATH_INFO: `/` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:38:25.951945 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 577/1170] 10.222.0.20 () {54 vars in 926 bytes} [Mon Aug 1 16:38:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 16:38:25.957259 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f566395e-59ef-41d7-b7eb-3134ba3524d6 None None] REQUEST_METHOD: `POST` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:38:25.957690 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f566395e-59ef-41d7-b7eb-3134ba3524d6 None None] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:38:25.957809 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f566395e-59ef-41d7-b7eb-3134ba3524d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:38:25.976267 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.auth.core [None req-f566395e-59ef-41d7-b7eb-3134ba3524d6 None None] MFA Rules not processed for user `1007f8e379b74e169b98fb7c3c0ba5b4`. Rule list: `[]` (Enabled: `True`). {{(pid=87058) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:38:25.977452 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.fernet_utils [None req-f566395e-59ef-41d7-b7eb-3134ba3524d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:38:25.997608 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG dbcounter [-] [87058] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:38:26.042724 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 594/1171] 10.222.0.20 () {58 vars in 1023 bytes} [Mon Aug 1 16:38:25 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 01 16:38:26.066259 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f92ac2-42b2-4be8-8b39-a8474ddda6e5 None None] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:38:26.162532 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f92ac2-42b2-4be8-8b39-a8474ddda6e5 None None] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:38:26.171146 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG dbcounter [-] [87057] Writing DB stats keystone:SELECT=100 {{(pid=87057) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:38:26.196207 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc8ae61-bec2-4ee9-b76e-ab798b574e90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:38:26.196837 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc8ae61-bec2-4ee9-b76e-ab798b574e90 None admin] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:38:26.197116 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc8ae61-bec2-4ee9-b76e-ab798b574e90 None admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:38:26.197388 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc8ae61-bec2-4ee9-b76e-ab798b574e90 None admin] PATH_INFO: `/v3/domains/aff24a4da7344f5f8001517f0a064d35` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:38:26.208315 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edc8ae61-bec2-4ee9-b76e-ab798b574e90 None admin] RBAC: Authorizing `identity:get_domain(domain_id=aff24a4da7344f5f8001517f0a064d35)` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:38:26.209751 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edc8ae61-bec2-4ee9-b76e-ab798b574e90 None admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:38:26.211374 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 578/1172] 10.222.0.20 () {56 vars in 1183 bytes} [Mon Aug 1 16:38:26 2022] GET /identity/v3/domains/aff24a4da7344f5f8001517f0a064d35 => generated 267 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:38:26.217273 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f566395e-59ef-41d7-b7eb-3134ba3524d6 None None] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:38:26.312262 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f566395e-59ef-41d7-b7eb-3134ba3524d6 None None] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:38:26.344518 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c15d213-d895-4879-9122-24e18d007342 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:38:26.345123 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c15d213-d895-4879-9122-24e18d007342 None admin] REQUEST_METHOD: `POST` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:38:26.345396 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c15d213-d895-4879-9122-24e18d007342 None admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:38:26.345661 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c15d213-d895-4879-9122-24e18d007342 None admin] PATH_INFO: `/v3/projects` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:38:26.346370 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c15d213-d895-4879-9122-24e18d007342 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:38:26.347828 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c15d213-d895-4879-9122-24e18d007342 None admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:38:26.418575 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 595/1173] 10.222.0.20 () {60 vars in 1141 bytes} [Mon Aug 1 16:38:26 2022] POST /identity/v3/projects => generated 361 bytes in 202 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 16:38:28.426494 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad31944-cf6f-4887-a077-fa48c7caf78f None None] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:38:28.427024 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad31944-cf6f-4887-a077-fa48c7caf78f None None] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:38:28.427024 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad31944-cf6f-4887-a077-fa48c7caf78f None None] PATH_INFO: `/` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:38:28.428394 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 579/1174] 10.222.0.20 () {54 vars in 926 bytes} [Mon Aug 1 16:38:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 16:38:28.433613 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2638ec4a-ae82-47cf-bf8a-2583cbdaaf5f None None] REQUEST_METHOD: `POST` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:38:28.433991 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2638ec4a-ae82-47cf-bf8a-2583cbdaaf5f None None] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:38:28.434204 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2638ec4a-ae82-47cf-bf8a-2583cbdaaf5f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:38:28.452926 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.auth.core [None req-2638ec4a-ae82-47cf-bf8a-2583cbdaaf5f None None] MFA Rules not processed for user `1007f8e379b74e169b98fb7c3c0ba5b4`. Rule list: `[]` (Enabled: `True`). {{(pid=87058) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:38:28.454160 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.fernet_utils [None req-2638ec4a-ae82-47cf-bf8a-2583cbdaaf5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:38:28.456644 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG dbcounter [-] [87058] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:38:28.521573 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 596/1175] 10.222.0.20 () {58 vars in 1023 bytes} [Mon Aug 1 16:38:28 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 01 16:38:28.545645 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad31944-cf6f-4887-a077-fa48c7caf78f None None] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:38:28.644831 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad31944-cf6f-4887-a077-fa48c7caf78f None None] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:38:28.677328 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-711d9c65-62c4-4994-ace2-5441e93e0a20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:38:28.677894 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711d9c65-62c4-4994-ace2-5441e93e0a20 None admin] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:38:28.678204 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711d9c65-62c4-4994-ace2-5441e93e0a20 None admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:38:28.678482 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711d9c65-62c4-4994-ace2-5441e93e0a20 None admin] PATH_INFO: `/v3/domains/aff24a4da7344f5f8001517f0a064d35` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:38:28.679631 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-711d9c65-62c4-4994-ace2-5441e93e0a20 None admin] RBAC: Authorizing `identity:get_domain(domain_id=aff24a4da7344f5f8001517f0a064d35)` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:38:28.681059 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-711d9c65-62c4-4994-ace2-5441e93e0a20 None admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:38:28.682652 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 580/1176] 10.222.0.20 () {56 vars in 1183 bytes} [Mon Aug 1 16:38:28 2022] GET /identity/v3/domains/aff24a4da7344f5f8001517f0a064d35 => generated 267 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:38:28.688194 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2638ec4a-ae82-47cf-bf8a-2583cbdaaf5f None None] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:38:28.785140 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2638ec4a-ae82-47cf-bf8a-2583cbdaaf5f None None] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:38:28.819548 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eafab52-5805-4c16-85e8-164162d12735 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:38:28.820149 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eafab52-5805-4c16-85e8-164162d12735 None admin] REQUEST_METHOD: `POST` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:38:28.820431 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eafab52-5805-4c16-85e8-164162d12735 None admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:38:28.820703 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eafab52-5805-4c16-85e8-164162d12735 None admin] PATH_INFO: `/v3/users` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:38:28.821384 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eafab52-5805-4c16-85e8-164162d12735 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:38:28.822837 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eafab52-5805-4c16-85e8-164162d12735 None admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:38:28.832519 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG dbcounter [-] [87058] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=87058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:38:28.910973 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 597/1177] 10.222.0.20 () {60 vars in 1132 bytes} [Mon Aug 1 16:38:28 2022] POST /identity/v3/users => generated 311 bytes in 223 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 16:38:30.944248 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56cb0d4-4ff1-4ae8-ba99-6d7e0f968cc2 None None] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:38:30.944742 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56cb0d4-4ff1-4ae8-ba99-6d7e0f968cc2 None None] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:38:30.944841 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56cb0d4-4ff1-4ae8-ba99-6d7e0f968cc2 None None] PATH_INFO: `/` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:38:30.946152 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 581/1178] 10.222.0.20 () {54 vars in 926 bytes} [Mon Aug 1 16:38:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 16:38:30.951358 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65507fc-5d04-421e-9921-4452493a7c98 None None] REQUEST_METHOD: `POST` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:38:30.951757 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65507fc-5d04-421e-9921-4452493a7c98 None None] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:38:30.951898 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65507fc-5d04-421e-9921-4452493a7c98 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:38:30.969999 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.auth.core [None req-e65507fc-5d04-421e-9921-4452493a7c98 None None] MFA Rules not processed for user `1007f8e379b74e169b98fb7c3c0ba5b4`. Rule list: `[]` (Enabled: `True`). {{(pid=87058) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:38:30.971197 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.fernet_utils [None req-e65507fc-5d04-421e-9921-4452493a7c98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:38:31.035528 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 598/1179] 10.222.0.20 () {58 vars in 1023 bytes} [Mon Aug 1 16:38:30 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 01 16:38:31.058102 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a56cb0d4-4ff1-4ae8-ba99-6d7e0f968cc2 None None] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:38:31.094116 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG dbcounter [-] [87057] Writing DB stats keystone:SELECT=100 {{(pid=87057) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:38:31.156290 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a56cb0d4-4ff1-4ae8-ba99-6d7e0f968cc2 None None] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:38:31.187906 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b18916-3dd2-445a-8447-ce7ddc3bd671 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:38:31.188526 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b18916-3dd2-445a-8447-ce7ddc3bd671 None admin] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:38:31.188802 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b18916-3dd2-445a-8447-ce7ddc3bd671 None admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:38:31.189072 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b18916-3dd2-445a-8447-ce7ddc3bd671 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:38:31.199175 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92b18916-3dd2-445a-8447-ce7ddc3bd671 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 16:38:31.203920 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application [None req-92b18916-3dd2-445a-8447-ce7ddc3bd671 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 16:38:31.203920 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 16:38:31.203920 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 16:38:31.203920 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 16:38:31.203920 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 16:38:31.203920 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 16:38:31.203920 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 16:38:31.203920 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 16:38:31.203920 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 16:38:31.203920 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 16:38:31.203920 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 16:38:31.203920 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 16:38:31.203920 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 16:38:31.203920 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 16:38:31.203920 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 16:38:31.203920 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 16:38:31.205064 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 16:38:31.205064 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 16:38:31.205064 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 16:38:31.205064 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 16:38:31.205064 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 16:38:31.205064 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 16:38:31.205064 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 16:38:31.205064 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 16:38:31.205064 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 16:38:31.205064 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 16:38:31.205064 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 16:38:31.205064 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application with Lock( Aug 01 16:38:31.205064 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 16:38:31.205064 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self._enter() Aug 01 16:38:31.205064 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 16:38:31.205064 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 16:38:31.206840 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 16:38:31.206840 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self.creator() Aug 01 16:38:31.206840 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 16:38:31.206840 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application created_value = creator( Aug 01 16:38:31.206840 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 16:38:31.206840 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 16:38:31.206840 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 16:38:31.206840 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 16:38:31.206840 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 16:38:31.206840 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 16:38:31.206840 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 16:38:31.206840 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application Aug 01 16:38:31.210193 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 582/1180] 10.222.0.20 () {56 vars in 1096 bytes} [Mon Aug 1 16:38:31 2022] GET /identity/v3/roles/admin => generated 83 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 16:38:31.211644 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e65507fc-5d04-421e-9921-4452493a7c98 None None] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:38:31.305926 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e65507fc-5d04-421e-9921-4452493a7c98 None None] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:38:31.328616 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG dbcounter [-] [87058] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=87058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:38:31.339409 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84d31f0e-4e20-43d2-8556-bef3fa06f0ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:38:31.340009 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d31f0e-4e20-43d2-8556-bef3fa06f0ce None admin] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:38:31.341254 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d31f0e-4e20-43d2-8556-bef3fa06f0ce None admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:38:31.341254 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d31f0e-4e20-43d2-8556-bef3fa06f0ce None admin] PATH_INFO: `/v3/roles` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:38:31.341254 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84d31f0e-4e20-43d2-8556-bef3fa06f0ce None admin] RBAC: Adding query filter params (name=admin) {{(pid=87058) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:38:31.341675 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84d31f0e-4e20-43d2-8556-bef3fa06f0ce None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:38:31.342871 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84d31f0e-4e20-43d2-8556-bef3fa06f0ce None admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:38:31.350729 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 599/1181] 10.222.0.20 () {56 vars in 1099 bytes} [Mon Aug 1 16:38:31 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:38:31.356556 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b18916-3dd2-445a-8447-ce7ddc3bd671 None admin] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:38:31.448440 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG dbcounter [-] [87057] Writing DB stats keystone:SELECT=100 {{(pid=87057) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:38:31.453874 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b18916-3dd2-445a-8447-ce7ddc3bd671 None admin] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:38:31.486502 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c096f76f-d837-42a3-b9c8-74d0964d7661 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:38:31.488112 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c096f76f-d837-42a3-b9c8-74d0964d7661 None admin] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:38:31.488388 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c096f76f-d837-42a3-b9c8-74d0964d7661 None admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:38:31.488656 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c096f76f-d837-42a3-b9c8-74d0964d7661 None admin] PATH_INFO: `/v3/users/b9be3c7f89c44300919797b929136edb` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:38:31.501095 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c096f76f-d837-42a3-b9c8-74d0964d7661 None admin] RBAC: Authorizing `identity:get_user(user_id=b9be3c7f89c44300919797b929136edb)` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:38:31.502552 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c096f76f-d837-42a3-b9c8-74d0964d7661 None admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:38:31.504184 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 583/1182] 10.222.0.20 () {56 vars in 1177 bytes} [Mon Aug 1 16:38:31 2022] GET /identity/v3/users/b9be3c7f89c44300919797b929136edb => generated 311 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:38:31.509991 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84d31f0e-4e20-43d2-8556-bef3fa06f0ce None admin] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:38:31.605832 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84d31f0e-4e20-43d2-8556-bef3fa06f0ce None admin] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:38:31.637305 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e946506-6584-4562-b186-eb6ebd260e8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:38:31.637900 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e946506-6584-4562-b186-eb6ebd260e8a None admin] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:38:31.638203 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e946506-6584-4562-b186-eb6ebd260e8a None admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:38:31.638473 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e946506-6584-4562-b186-eb6ebd260e8a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:38:31.691392 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG dbcounter [-] [87058] Writing DB stats keystone:SELECT=100 {{(pid=87058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:38:31.703830 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e946506-6584-4562-b186-eb6ebd260e8a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:38:31.705196 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e946506-6584-4562-b186-eb6ebd260e8a None admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:38:31.809852 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 600/1183] 10.222.0.20 () {58 vars in 1282 bytes} [Mon Aug 1 16:38:31 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 300 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 01 16:38:31.816379 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c096f76f-d837-42a3-b9c8-74d0964d7661 None admin] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:38:31.915064 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c096f76f-d837-42a3-b9c8-74d0964d7661 None admin] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:38:31.948279 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0997cd8e-48f1-4f27-a849-74ca39ed094b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:38:31.948914 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0997cd8e-48f1-4f27-a849-74ca39ed094b None admin] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:38:31.949194 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0997cd8e-48f1-4f27-a849-74ca39ed094b None admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:38:31.949467 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0997cd8e-48f1-4f27-a849-74ca39ed094b None admin] PATH_INFO: `/v3/projects/a2949d74e95b44d7a400f4af9e798072` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:38:31.950639 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0997cd8e-48f1-4f27-a849-74ca39ed094b None admin] RBAC: Authorizing `identity:get_project(project_id=a2949d74e95b44d7a400f4af9e798072)` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:38:31.952068 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0997cd8e-48f1-4f27-a849-74ca39ed094b None admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:38:31.953711 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 584/1184] 10.222.0.20 () {56 vars in 1186 bytes} [Mon Aug 1 16:38:31 2022] GET /identity/v3/projects/a2949d74e95b44d7a400f4af9e798072 => generated 361 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:38:31.959904 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e946506-6584-4562-b186-eb6ebd260e8a None admin] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:38:32.041346 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG dbcounter [-] [87058] Writing DB stats keystone:SELECT=100 {{(pid=87058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:38:32.055349 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e946506-6584-4562-b186-eb6ebd260e8a None admin] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:38:32.088729 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba0caf3-d2c7-436a-a199-0b8ac4dcb49c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:38:32.089332 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba0caf3-d2c7-436a-a199-0b8ac4dcb49c None admin] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:38:32.089608 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba0caf3-d2c7-436a-a199-0b8ac4dcb49c None admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:38:32.089878 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba0caf3-d2c7-436a-a199-0b8ac4dcb49c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:38:32.090677 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eba0caf3-d2c7-436a-a199-0b8ac4dcb49c None admin] RBAC: Adding query filter params (role.id=440bb943dd5b46fea038b903b6eaaf81, scope.project.id=a2949d74e95b44d7a400f4af9e798072, user.id=b9be3c7f89c44300919797b929136edb) {{(pid=87058) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:38:32.090980 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eba0caf3-d2c7-436a-a199-0b8ac4dcb49c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:38:32.092371 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eba0caf3-d2c7-436a-a199-0b8ac4dcb49c None admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:38:32.097623 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 601/1185] 10.222.0.20 () {56 vars in 1374 bytes} [Mon Aug 1 16:38:31 2022] GET /identity/v3/role_assignments?user.id=b9be3c7f89c44300919797b929136edb&scope.project.id=a2949d74e95b44d7a400f4af9e798072&role.id=440bb943dd5b46fea038b903b6eaaf81 => generated 259 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:38:34.053364 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7b3662-64a9-410f-8a55-33b47aca22b7 None None] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:38:34.053903 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7b3662-64a9-410f-8a55-33b47aca22b7 None None] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:38:34.053903 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7b3662-64a9-410f-8a55-33b47aca22b7 None None] PATH_INFO: `/` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:38:34.055275 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 585/1186] 10.222.0.20 () {54 vars in 926 bytes} [Mon Aug 1 16:38:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 16:38:34.060428 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5595549a-d366-42c9-b1a1-59257d3aaa6c None None] REQUEST_METHOD: `POST` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:38:34.060874 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5595549a-d366-42c9-b1a1-59257d3aaa6c None None] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:38:34.060981 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5595549a-d366-42c9-b1a1-59257d3aaa6c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:38:34.080107 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.auth.core [None req-5595549a-d366-42c9-b1a1-59257d3aaa6c None None] MFA Rules not processed for user `1007f8e379b74e169b98fb7c3c0ba5b4`. Rule list: `[]` (Enabled: `True`). {{(pid=87058) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:38:34.082260 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.fernet_utils [None req-5595549a-d366-42c9-b1a1-59257d3aaa6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:38:34.147490 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 602/1187] 10.222.0.20 () {58 vars in 1023 bytes} [Mon Aug 1 16:38:34 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 01 16:38:34.169567 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f7b3662-64a9-410f-8a55-33b47aca22b7 None None] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:38:34.193632 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG dbcounter [-] [87057] Writing DB stats keystone:SELECT=100 {{(pid=87057) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:38:34.269996 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f7b3662-64a9-410f-8a55-33b47aca22b7 None None] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:38:34.302486 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c9ca38-32b3-4b30-afb7-59b9f5fc974f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:38:34.303106 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c9ca38-32b3-4b30-afb7-59b9f5fc974f None admin] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:38:34.303390 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c9ca38-32b3-4b30-afb7-59b9f5fc974f None admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:38:34.303664 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c9ca38-32b3-4b30-afb7-59b9f5fc974f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:38:34.313437 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4c9ca38-32b3-4b30-afb7-59b9f5fc974f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 16:38:34.318959 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application [None req-b4c9ca38-32b3-4b30-afb7-59b9f5fc974f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 16:38:34.318959 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 16:38:34.318959 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 16:38:34.318959 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 16:38:34.318959 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 16:38:34.318959 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 16:38:34.318959 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 16:38:34.318959 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 16:38:34.318959 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 16:38:34.318959 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 16:38:34.318959 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 16:38:34.318959 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 16:38:34.318959 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 16:38:34.318959 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 16:38:34.318959 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 16:38:34.318959 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 16:38:34.320723 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 16:38:34.320723 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 16:38:34.320723 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 16:38:34.320723 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 16:38:34.320723 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 16:38:34.320723 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 16:38:34.320723 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 16:38:34.320723 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 16:38:34.320723 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 16:38:34.320723 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 16:38:34.320723 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 16:38:34.320723 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application with Lock( Aug 01 16:38:34.320723 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 16:38:34.320723 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self._enter() Aug 01 16:38:34.320723 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 16:38:34.320723 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 16:38:34.322341 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 16:38:34.322341 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self.creator() Aug 01 16:38:34.322341 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 16:38:34.322341 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application created_value = creator( Aug 01 16:38:34.322341 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 16:38:34.322341 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 16:38:34.322341 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 16:38:34.322341 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 16:38:34.322341 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 16:38:34.322341 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 16:38:34.322341 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 16:38:34.322341 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application Aug 01 16:38:34.322341 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 586/1188] 10.222.0.20 () {56 vars in 1096 bytes} [Mon Aug 1 16:38:34 2022] GET /identity/v3/roles/admin => generated 83 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 16:38:34.325980 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5595549a-d366-42c9-b1a1-59257d3aaa6c None None] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:38:34.403575 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG dbcounter [-] [87058] Writing DB stats keystone:SELECT=100 {{(pid=87058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:38:34.422058 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5595549a-d366-42c9-b1a1-59257d3aaa6c None None] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:38:34.455324 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56889edf-1edf-4676-bdf5-cd28eb83d507 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:38:34.456475 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56889edf-1edf-4676-bdf5-cd28eb83d507 None admin] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:38:34.456749 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56889edf-1edf-4676-bdf5-cd28eb83d507 None admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:38:34.457018 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56889edf-1edf-4676-bdf5-cd28eb83d507 None admin] PATH_INFO: `/v3/roles` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:38:34.457733 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56889edf-1edf-4676-bdf5-cd28eb83d507 None admin] RBAC: Adding query filter params (name=admin) {{(pid=87058) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:38:34.458061 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56889edf-1edf-4676-bdf5-cd28eb83d507 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:38:34.459411 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56889edf-1edf-4676-bdf5-cd28eb83d507 None admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:38:34.467175 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 603/1189] 10.222.0.20 () {56 vars in 1099 bytes} [Mon Aug 1 16:38:34 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:38:34.472969 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c9ca38-32b3-4b30-afb7-59b9f5fc974f None admin] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:38:34.551994 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG dbcounter [-] [87057] Writing DB stats keystone:SELECT=100 {{(pid=87057) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:38:34.569788 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c9ca38-32b3-4b30-afb7-59b9f5fc974f None admin] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:38:34.603681 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ef9169c-8197-472d-9aa1-f6c9097e845b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:38:34.604309 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef9169c-8197-472d-9aa1-f6c9097e845b None admin] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:38:34.604581 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef9169c-8197-472d-9aa1-f6c9097e845b None admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:38:34.604846 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef9169c-8197-472d-9aa1-f6c9097e845b None admin] PATH_INFO: `/v3/users/b9be3c7f89c44300919797b929136edb` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:38:34.605979 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ef9169c-8197-472d-9aa1-f6c9097e845b None admin] RBAC: Authorizing `identity:get_user(user_id=b9be3c7f89c44300919797b929136edb)` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:38:34.607418 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ef9169c-8197-472d-9aa1-f6c9097e845b None admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:38:34.608980 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 587/1190] 10.222.0.20 () {56 vars in 1177 bytes} [Mon Aug 1 16:38:34 2022] GET /identity/v3/users/b9be3c7f89c44300919797b929136edb => generated 311 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:38:34.614733 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56889edf-1edf-4676-bdf5-cd28eb83d507 None admin] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:38:34.711962 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56889edf-1edf-4676-bdf5-cd28eb83d507 None admin] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:38:34.744234 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-683e6bc9-c7eb-4561-9246-8faec5e73f5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:38:34.744857 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-683e6bc9-c7eb-4561-9246-8faec5e73f5d None admin] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:38:34.745134 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-683e6bc9-c7eb-4561-9246-8faec5e73f5d None admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:38:34.745405 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-683e6bc9-c7eb-4561-9246-8faec5e73f5d None admin] PATH_INFO: `/v3/projects/a2949d74e95b44d7a400f4af9e798072` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:38:34.746560 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-683e6bc9-c7eb-4561-9246-8faec5e73f5d None admin] RBAC: Authorizing `identity:get_project(project_id=a2949d74e95b44d7a400f4af9e798072)` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:38:34.747975 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-683e6bc9-c7eb-4561-9246-8faec5e73f5d None admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:38:34.749597 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 604/1191] 10.222.0.20 () {56 vars in 1186 bytes} [Mon Aug 1 16:38:34 2022] GET /identity/v3/projects/a2949d74e95b44d7a400f4af9e798072 => generated 361 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:38:34.755594 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ef9169c-8197-472d-9aa1-f6c9097e845b None admin] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:38:34.852380 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ef9169c-8197-472d-9aa1-f6c9097e845b None admin] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:38:34.884637 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcb43f17-3829-4804-8239-382e3deca0d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:38:34.885282 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb43f17-3829-4804-8239-382e3deca0d3 None admin] REQUEST_METHOD: `PUT` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:38:34.885562 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb43f17-3829-4804-8239-382e3deca0d3 None admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:38:34.885831 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb43f17-3829-4804-8239-382e3deca0d3 None admin] PATH_INFO: `/v3/projects/a2949d74e95b44d7a400f4af9e798072/users/b9be3c7f89c44300919797b929136edb/roles/440bb943dd5b46fea038b903b6eaaf81` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:38:34.887779 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcb43f17-3829-4804-8239-382e3deca0d3 None admin] RBAC: Authorizing `identity:create_grant(project_id=a2949d74e95b44d7a400f4af9e798072, user_id=b9be3c7f89c44300919797b929136edb, role_id=440bb943dd5b46fea038b903b6eaaf81)` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:38:34.889458 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcb43f17-3829-4804-8239-382e3deca0d3 None admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:38:35.004656 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 588/1192] 10.222.0.20 () {58 vars in 1439 bytes} [Mon Aug 1 16:38:34 2022] PUT /identity/v3/projects/a2949d74e95b44d7a400f4af9e798072/users/b9be3c7f89c44300919797b929136edb/roles/440bb943dd5b46fea038b903b6eaaf81 => generated 0 bytes in 249 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 16:38:36.974489 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4cfab1-78a5-4189-81ee-2650c9a88665 None None] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:38:36.975007 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4cfab1-78a5-4189-81ee-2650c9a88665 None None] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:38:36.975109 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4cfab1-78a5-4189-81ee-2650c9a88665 None None] PATH_INFO: `/` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:38:36.976396 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 605/1193] 10.222.0.20 () {54 vars in 926 bytes} [Mon Aug 1 16:38:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 16:38:36.981444 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d11a486-0a62-40c5-9b14-b39223327a62 None None] REQUEST_METHOD: `POST` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:38:36.981868 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d11a486-0a62-40c5-9b14-b39223327a62 None None] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:38:36.981976 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d11a486-0a62-40c5-9b14-b39223327a62 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:38:36.999625 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.auth.core [None req-8d11a486-0a62-40c5-9b14-b39223327a62 None None] MFA Rules not processed for user `1007f8e379b74e169b98fb7c3c0ba5b4`. Rule list: `[]` (Enabled: `True`). {{(pid=87057) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:38:37.000799 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.fernet_utils [None req-8d11a486-0a62-40c5-9b14-b39223327a62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87057) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:38:37.003703 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG dbcounter [-] [87057] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87057) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:38:37.064644 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 589/1194] 10.222.0.20 () {58 vars in 1023 bytes} [Mon Aug 1 16:38:36 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 01 16:38:37.089041 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba4cfab1-78a5-4189-81ee-2650c9a88665 None None] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:38:37.101811 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG dbcounter [-] [87058] Writing DB stats keystone:SELECT=100 {{(pid=87058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:38:37.189010 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba4cfab1-78a5-4189-81ee-2650c9a88665 None None] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:38:37.221885 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f22a5972-9c2a-469f-a220-b7234d9392c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:38:37.222491 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22a5972-9c2a-469f-a220-b7234d9392c2 None admin] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:38:37.222765 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22a5972-9c2a-469f-a220-b7234d9392c2 None admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:38:37.223029 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22a5972-9c2a-469f-a220-b7234d9392c2 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:38:37.232937 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f22a5972-9c2a-469f-a220-b7234d9392c2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 16:38:37.238534 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application [None req-f22a5972-9c2a-469f-a220-b7234d9392c2 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 16:38:37.238534 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 16:38:37.238534 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 16:38:37.238534 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 16:38:37.238534 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 16:38:37.238534 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 16:38:37.238534 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 16:38:37.238534 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 16:38:37.238534 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 16:38:37.238534 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 16:38:37.238534 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 16:38:37.238534 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 16:38:37.238534 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 16:38:37.238534 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 16:38:37.238534 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 16:38:37.238534 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 16:38:37.239640 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 16:38:37.239640 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 16:38:37.239640 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 16:38:37.239640 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 16:38:37.239640 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 16:38:37.239640 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 16:38:37.239640 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 16:38:37.239640 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 16:38:37.239640 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 16:38:37.239640 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 16:38:37.239640 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 16:38:37.239640 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application with Lock( Aug 01 16:38:37.239640 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 16:38:37.239640 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self._enter() Aug 01 16:38:37.239640 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 16:38:37.239640 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 16:38:37.240616 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 16:38:37.240616 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self.creator() Aug 01 16:38:37.240616 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 16:38:37.240616 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application created_value = creator( Aug 01 16:38:37.240616 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 16:38:37.240616 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 16:38:37.240616 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 16:38:37.240616 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 16:38:37.240616 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 16:38:37.240616 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 16:38:37.240616 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 16:38:37.240616 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application Aug 01 16:38:37.240616 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 606/1195] 10.222.0.20 () {56 vars in 1096 bytes} [Mon Aug 1 16:38:37 2022] GET /identity/v3/roles/admin => generated 83 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 16:38:37.245601 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d11a486-0a62-40c5-9b14-b39223327a62 None None] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:38:37.339519 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d11a486-0a62-40c5-9b14-b39223327a62 None None] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:38:37.371872 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9463d59-d626-401a-85ff-0536b553f7b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:38:37.372473 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9463d59-d626-401a-85ff-0536b553f7b4 None admin] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:38:37.372747 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9463d59-d626-401a-85ff-0536b553f7b4 None admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:38:37.373013 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9463d59-d626-401a-85ff-0536b553f7b4 None admin] PATH_INFO: `/v3/roles` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:38:37.373713 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9463d59-d626-401a-85ff-0536b553f7b4 None admin] RBAC: Adding query filter params (name=admin) {{(pid=87057) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:38:37.374009 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9463d59-d626-401a-85ff-0536b553f7b4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:38:37.375395 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9463d59-d626-401a-85ff-0536b553f7b4 None admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:38:37.383695 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 590/1196] 10.222.0.20 () {56 vars in 1099 bytes} [Mon Aug 1 16:38:37 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 16:38:37.389636 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f22a5972-9c2a-469f-a220-b7234d9392c2 None admin] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:38:37.458848 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG dbcounter [-] [87058] Writing DB stats keystone:SELECT=100 {{(pid=87058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:38:37.488010 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f22a5972-9c2a-469f-a220-b7234d9392c2 None admin] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:38:37.522193 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1afe543a-a57a-4b60-b08d-11ab0a8ba7bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:38:37.522828 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1afe543a-a57a-4b60-b08d-11ab0a8ba7bc None admin] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:38:37.523109 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1afe543a-a57a-4b60-b08d-11ab0a8ba7bc None admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:38:37.523384 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1afe543a-a57a-4b60-b08d-11ab0a8ba7bc None admin] PATH_INFO: `/v3/users/b9be3c7f89c44300919797b929136edb` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:38:37.524574 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1afe543a-a57a-4b60-b08d-11ab0a8ba7bc None admin] RBAC: Authorizing `identity:get_user(user_id=b9be3c7f89c44300919797b929136edb)` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:38:37.526000 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1afe543a-a57a-4b60-b08d-11ab0a8ba7bc None admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:38:37.527625 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 607/1197] 10.222.0.20 () {56 vars in 1177 bytes} [Mon Aug 1 16:38:37 2022] GET /identity/v3/users/b9be3c7f89c44300919797b929136edb => generated 311 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:38:37.533305 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9463d59-d626-401a-85ff-0536b553f7b4 None admin] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:38:37.536252 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG dbcounter [-] [87057] Writing DB stats keystone:SELECT=100 {{(pid=87057) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:38:37.629642 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9463d59-d626-401a-85ff-0536b553f7b4 None admin] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:38:37.661813 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d52d6fbe-8b25-4a44-879c-42b12df46609 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:38:37.662445 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52d6fbe-8b25-4a44-879c-42b12df46609 None admin] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:38:37.662726 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52d6fbe-8b25-4a44-879c-42b12df46609 None admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:38:37.663002 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52d6fbe-8b25-4a44-879c-42b12df46609 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:38:37.729017 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d52d6fbe-8b25-4a44-879c-42b12df46609 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:38:37.730421 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d52d6fbe-8b25-4a44-879c-42b12df46609 None admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:38:37.732150 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG dbcounter [-] [87057] Writing DB stats keystone:SELECT=100 {{(pid=87057) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:38:37.825807 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 591/1198] 10.222.0.20 () {58 vars in 1282 bytes} [Mon Aug 1 16:38:37 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 293 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 01 16:38:37.831713 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1afe543a-a57a-4b60-b08d-11ab0a8ba7bc None admin] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:38:37.930631 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1afe543a-a57a-4b60-b08d-11ab0a8ba7bc None admin] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:38:37.962748 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11e29c76-9fff-4348-bb20-5c766a98f615 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:38:37.963368 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e29c76-9fff-4348-bb20-5c766a98f615 None admin] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:38:37.963638 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e29c76-9fff-4348-bb20-5c766a98f615 None admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:38:37.963904 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e29c76-9fff-4348-bb20-5c766a98f615 None admin] PATH_INFO: `/v3/projects/a2949d74e95b44d7a400f4af9e798072` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:38:37.965025 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11e29c76-9fff-4348-bb20-5c766a98f615 None admin] RBAC: Authorizing `identity:get_project(project_id=a2949d74e95b44d7a400f4af9e798072)` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:38:37.966458 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11e29c76-9fff-4348-bb20-5c766a98f615 None admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:38:37.968093 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 608/1199] 10.222.0.20 () {56 vars in 1186 bytes} [Mon Aug 1 16:38:37 2022] GET /identity/v3/projects/a2949d74e95b44d7a400f4af9e798072 => generated 361 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:38:37.974671 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d52d6fbe-8b25-4a44-879c-42b12df46609 None admin] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:38:38.071173 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d52d6fbe-8b25-4a44-879c-42b12df46609 None admin] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:38:38.072674 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG dbcounter [-] [87057] Writing DB stats keystone:SELECT=100 {{(pid=87057) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:38:38.104262 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94f04ba3-ce73-4e60-be9b-394d18e1bc57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:38:38.104863 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f04ba3-ce73-4e60-be9b-394d18e1bc57 None admin] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:38:38.105142 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f04ba3-ce73-4e60-be9b-394d18e1bc57 None admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:38:38.105415 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f04ba3-ce73-4e60-be9b-394d18e1bc57 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:38:38.106347 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94f04ba3-ce73-4e60-be9b-394d18e1bc57 None admin] RBAC: Adding query filter params (role.id=440bb943dd5b46fea038b903b6eaaf81, scope.project.id=a2949d74e95b44d7a400f4af9e798072, user.id=b9be3c7f89c44300919797b929136edb) {{(pid=87057) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:38:38.106657 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94f04ba3-ce73-4e60-be9b-394d18e1bc57 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:38:38.108081 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94f04ba3-ce73-4e60-be9b-394d18e1bc57 None admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:38:38.114615 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 592/1200] 10.222.0.20 () {56 vars in 1374 bytes} [Mon Aug 1 16:38:37 2022] GET /identity/v3/role_assignments?user.id=b9be3c7f89c44300919797b929136edb&scope.project.id=a2949d74e95b44d7a400f4af9e798072&role.id=440bb943dd5b46fea038b903b6eaaf81 => generated 608 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:38:47.958724 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG dbcounter [-] [87058] Writing DB stats keystone:SELECT=97 {{(pid=87058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:38:48.117058 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG dbcounter [-] [87057] Writing DB stats keystone:SELECT=17 {{(pid=87057) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:39:49.923590 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29fcf594-6dd9-4443-a1a4-a5f5eb711654 None None] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:39:49.924062 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29fcf594-6dd9-4443-a1a4-a5f5eb711654 None None] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:39:49.924156 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29fcf594-6dd9-4443-a1a4-a5f5eb711654 None None] PATH_INFO: `/` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:39:49.925480 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 609/1201] 10.222.0.20 () {54 vars in 926 bytes} [Mon Aug 1 16:39:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 16:39:49.931269 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c020fa9-a540-45fe-8561-c4a86392cf4f None None] REQUEST_METHOD: `POST` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:39:49.931739 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c020fa9-a540-45fe-8561-c4a86392cf4f None None] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:39:49.931840 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c020fa9-a540-45fe-8561-c4a86392cf4f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:39:49.950399 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.auth.core [None req-6c020fa9-a540-45fe-8561-c4a86392cf4f None None] MFA Rules not processed for user `1007f8e379b74e169b98fb7c3c0ba5b4`. Rule list: `[]` (Enabled: `True`). {{(pid=87057) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:39:49.951602 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.fernet_utils [None req-6c020fa9-a540-45fe-8561-c4a86392cf4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87057) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:39:50.017300 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 593/1202] 10.222.0.20 () {58 vars in 1023 bytes} [Mon Aug 1 16:39:49 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 01 16:39:50.041911 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29fcf594-6dd9-4443-a1a4-a5f5eb711654 None None] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:39:50.144783 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29fcf594-6dd9-4443-a1a4-a5f5eb711654 None None] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:39:50.178700 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61cc24ec-f68f-4c7c-82b6-f522b542dcb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:39:50.179360 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61cc24ec-f68f-4c7c-82b6-f522b542dcb3 None admin] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:39:50.179641 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61cc24ec-f68f-4c7c-82b6-f522b542dcb3 None admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:39:50.179911 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61cc24ec-f68f-4c7c-82b6-f522b542dcb3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:39:50.189953 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61cc24ec-f68f-4c7c-82b6-f522b542dcb3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:39:50.191417 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61cc24ec-f68f-4c7c-82b6-f522b542dcb3 None admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:39:50.206365 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application [None req-61cc24ec-f68f-4c7c-82b6-f522b542dcb3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 16:39:50.206365 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 16:39:50.206365 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 16:39:50.206365 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 16:39:50.206365 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 16:39:50.206365 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 16:39:50.206365 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 16:39:50.206365 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 16:39:50.206365 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 16:39:50.206365 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 16:39:50.206365 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 16:39:50.206365 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 16:39:50.206365 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 16:39:50.206365 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 16:39:50.206365 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 16:39:50.207488 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 16:39:50.207488 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 16:39:50.207488 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 16:39:50.207488 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 16:39:50.207488 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 16:39:50.207488 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 16:39:50.207488 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 16:39:50.207488 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 16:39:50.207488 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application with Lock( Aug 01 16:39:50.207488 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 16:39:50.207488 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self._enter() Aug 01 16:39:50.207488 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 16:39:50.207488 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 16:39:50.207488 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 16:39:50.207488 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self.creator() Aug 01 16:39:50.207488 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 16:39:50.207488 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application created_value = creator( Aug 01 16:39:50.208569 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 16:39:50.208569 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 16:39:50.208569 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 16:39:50.208569 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application Aug 01 16:39:50.208569 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 610/1203] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Aug 1 16:39:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 16:39:50.214270 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c020fa9-a540-45fe-8561-c4a86392cf4f None None] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:39:50.310477 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c020fa9-a540-45fe-8561-c4a86392cf4f None None] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:39:50.338597 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG dbcounter [-] [87057] Writing DB stats keystone:SELECT=100 {{(pid=87057) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:39:50.344226 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2ebf748-1227-48df-bc83-607d1b368193 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:39:50.344834 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ebf748-1227-48df-bc83-607d1b368193 None admin] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:39:50.345216 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ebf748-1227-48df-bc83-607d1b368193 None admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:39:50.345386 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ebf748-1227-48df-bc83-607d1b368193 None admin] PATH_INFO: `/v3/domains` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:39:50.346161 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2ebf748-1227-48df-bc83-607d1b368193 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87057) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:39:50.346467 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2ebf748-1227-48df-bc83-607d1b368193 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:39:50.347839 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2ebf748-1227-48df-bc83-607d1b368193 None admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:39:50.359069 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 594/1204] 10.222.0.20 () {56 vars in 1109 bytes} [Mon Aug 1 16:39:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:39:50.364929 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61cc24ec-f68f-4c7c-82b6-f522b542dcb3 None admin] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:39:50.417949 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG dbcounter [-] [87058] Writing DB stats keystone:SELECT=100 {{(pid=87058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:39:50.464064 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61cc24ec-f68f-4c7c-82b6-f522b542dcb3 None admin] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:39:50.497777 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-383b8170-2e5d-42bb-9a3a-30cbbc507454 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:39:50.498461 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383b8170-2e5d-42bb-9a3a-30cbbc507454 None admin] REQUEST_METHOD: `POST` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:39:50.498851 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383b8170-2e5d-42bb-9a3a-30cbbc507454 None admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:39:50.499127 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383b8170-2e5d-42bb-9a3a-30cbbc507454 None admin] PATH_INFO: `/v3/users` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:39:50.499805 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-383b8170-2e5d-42bb-9a3a-30cbbc507454 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:39:50.501217 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-383b8170-2e5d-42bb-9a3a-30cbbc507454 None admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:39:50.580594 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 611/1205] 10.222.0.20 () {60 vars in 1132 bytes} [Mon Aug 1 16:39:50 2022] POST /identity/v3/users => generated 251 bytes in 216 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 16:39:52.596330 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac806eb-ceef-415d-9909-1bef0a795c22 None None] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:39:52.596902 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac806eb-ceef-415d-9909-1bef0a795c22 None None] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:39:52.596902 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac806eb-ceef-415d-9909-1bef0a795c22 None None] PATH_INFO: `/` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:39:52.598258 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 595/1206] 10.222.0.20 () {54 vars in 926 bytes} [Mon Aug 1 16:39:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 01 16:39:52.603480 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d105939-ddfb-4310-aa47-29204c2fe6de None None] REQUEST_METHOD: `POST` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:39:52.603864 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d105939-ddfb-4310-aa47-29204c2fe6de None None] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:39:52.604039 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d105939-ddfb-4310-aa47-29204c2fe6de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:39:52.622125 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.auth.core [None req-1d105939-ddfb-4310-aa47-29204c2fe6de None None] MFA Rules not processed for user `1007f8e379b74e169b98fb7c3c0ba5b4`. Rule list: `[]` (Enabled: `True`). {{(pid=87058) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:39:52.623329 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.fernet_utils [None req-1d105939-ddfb-4310-aa47-29204c2fe6de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:39:52.689983 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 612/1207] 10.222.0.20 () {58 vars in 1023 bytes} [Mon Aug 1 16:39:52 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 01 16:39:52.713049 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac806eb-ceef-415d-9909-1bef0a795c22 None None] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:39:52.813536 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac806eb-ceef-415d-9909-1bef0a795c22 None None] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:39:52.847416 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a60f3868-f79f-4c08-b55a-8f3de1480f9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:39:52.847883 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60f3868-f79f-4c08-b55a-8f3de1480f9d None admin] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:39:52.848168 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60f3868-f79f-4c08-b55a-8f3de1480f9d None admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:39:52.848440 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60f3868-f79f-4c08-b55a-8f3de1480f9d None admin] PATH_INFO: `/v3/roles/service` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:39:52.859113 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a60f3868-f79f-4c08-b55a-8f3de1480f9d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 16:39:52.863993 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application [None req-a60f3868-f79f-4c08-b55a-8f3de1480f9d None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 01 16:39:52.863993 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 16:39:52.863993 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 16:39:52.863993 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 16:39:52.863993 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 16:39:52.863993 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 16:39:52.863993 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 16:39:52.863993 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 16:39:52.863993 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 16:39:52.863993 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 16:39:52.863993 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 16:39:52.863993 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 16:39:52.863993 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 16:39:52.863993 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 16:39:52.863993 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 16:39:52.863993 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 16:39:52.865187 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 16:39:52.865187 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 16:39:52.865187 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 16:39:52.865187 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 16:39:52.865187 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 16:39:52.865187 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 16:39:52.865187 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 16:39:52.865187 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 16:39:52.865187 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 16:39:52.865187 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 16:39:52.865187 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 16:39:52.865187 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application with Lock( Aug 01 16:39:52.865187 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 16:39:52.865187 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self._enter() Aug 01 16:39:52.865187 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 16:39:52.865187 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 16:39:52.866502 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 16:39:52.866502 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self.creator() Aug 01 16:39:52.866502 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 16:39:52.866502 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application created_value = creator( Aug 01 16:39:52.866502 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 16:39:52.866502 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 16:39:52.866502 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 16:39:52.866502 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 16:39:52.866502 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 16:39:52.866502 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 16:39:52.866502 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 01 16:39:52.866502 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application Aug 01 16:39:52.866502 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 596/1208] 10.222.0.20 () {56 vars in 1102 bytes} [Mon Aug 1 16:39:52 2022] GET /identity/v3/roles/service => generated 85 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 16:39:52.871165 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d105939-ddfb-4310-aa47-29204c2fe6de None None] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:39:52.910877 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG dbcounter [-] [87058] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:39:52.970157 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d105939-ddfb-4310-aa47-29204c2fe6de None None] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:39:53.004653 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d93abfd2-b73f-4cf8-8750-e1ea46af41cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:39:53.005282 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93abfd2-b73f-4cf8-8750-e1ea46af41cc None admin] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:39:53.005549 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93abfd2-b73f-4cf8-8750-e1ea46af41cc None admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:39:53.005821 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93abfd2-b73f-4cf8-8750-e1ea46af41cc None admin] PATH_INFO: `/v3/roles` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:39:53.006608 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d93abfd2-b73f-4cf8-8750-e1ea46af41cc None admin] RBAC: Adding query filter params (name=service) {{(pid=87058) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:39:53.006917 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d93abfd2-b73f-4cf8-8750-e1ea46af41cc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:39:53.008323 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d93abfd2-b73f-4cf8-8750-e1ea46af41cc None admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:39:53.016382 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 613/1209] 10.222.0.20 () {56 vars in 1103 bytes} [Mon Aug 1 16:39:52 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:39:53.022896 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a60f3868-f79f-4c08-b55a-8f3de1480f9d None admin] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:39:53.073834 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG dbcounter [-] [87057] Writing DB stats keystone:SELECT=100 {{(pid=87057) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:39:53.122914 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a60f3868-f79f-4c08-b55a-8f3de1480f9d None admin] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:39:53.156527 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd82ebc2-bd8c-4d02-963c-726138a3bdcc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:39:53.157154 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd82ebc2-bd8c-4d02-963c-726138a3bdcc None admin] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:39:53.157429 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd82ebc2-bd8c-4d02-963c-726138a3bdcc None admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:39:53.157698 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd82ebc2-bd8c-4d02-963c-726138a3bdcc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:39:53.167889 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd82ebc2-bd8c-4d02-963c-726138a3bdcc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:39:53.169329 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd82ebc2-bd8c-4d02-963c-726138a3bdcc None admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:39:53.183273 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application [None req-dd82ebc2-bd8c-4d02-963c-726138a3bdcc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 16:39:53.183273 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 16:39:53.183273 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 16:39:53.183273 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 16:39:53.183273 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 16:39:53.183273 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 16:39:53.183273 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 16:39:53.183273 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 16:39:53.183273 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 16:39:53.183273 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 16:39:53.183273 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 16:39:53.183273 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 16:39:53.183273 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 16:39:53.183273 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 16:39:53.183273 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 16:39:53.184421 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 16:39:53.184421 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 16:39:53.184421 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 16:39:53.184421 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 16:39:53.184421 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 16:39:53.184421 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 16:39:53.184421 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 16:39:53.184421 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 16:39:53.184421 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application with Lock( Aug 01 16:39:53.184421 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 16:39:53.184421 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self._enter() Aug 01 16:39:53.184421 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 16:39:53.184421 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 16:39:53.184421 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 16:39:53.184421 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self.creator() Aug 01 16:39:53.184421 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 16:39:53.184421 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application created_value = creator( Aug 01 16:39:53.185378 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 16:39:53.185378 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 16:39:53.185378 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 16:39:53.185378 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application Aug 01 16:39:53.185378 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 597/1210] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Aug 1 16:39:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 16:39:53.190553 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d93abfd2-b73f-4cf8-8750-e1ea46af41cc None admin] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:39:53.286910 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG dbcounter [-] [87058] Writing DB stats keystone:SELECT=100 {{(pid=87058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:39:53.289930 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d93abfd2-b73f-4cf8-8750-e1ea46af41cc None admin] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:39:53.322630 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a7b4c1e-4a16-4302-9a71-3c95439dab13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:39:53.323249 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7b4c1e-4a16-4302-9a71-3c95439dab13 None admin] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:39:53.323528 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7b4c1e-4a16-4302-9a71-3c95439dab13 None admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:39:53.323802 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7b4c1e-4a16-4302-9a71-3c95439dab13 None admin] PATH_INFO: `/v3/domains` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:39:53.324498 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a7b4c1e-4a16-4302-9a71-3c95439dab13 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87058) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:39:53.324802 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a7b4c1e-4a16-4302-9a71-3c95439dab13 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:39:53.326210 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a7b4c1e-4a16-4302-9a71-3c95439dab13 None admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:39:53.338383 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 614/1211] 10.222.0.20 () {56 vars in 1109 bytes} [Mon Aug 1 16:39:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:39:53.343981 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd82ebc2-bd8c-4d02-963c-726138a3bdcc None admin] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:39:53.442720 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd82ebc2-bd8c-4d02-963c-726138a3bdcc None admin] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:39:53.444791 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG dbcounter [-] [87057] Writing DB stats keystone:SELECT=100 {{(pid=87057) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:39:53.476600 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc40893d-f19b-4f8d-9cc2-8ac7c443ba65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:39:53.477229 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc40893d-f19b-4f8d-9cc2-8ac7c443ba65 None admin] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:39:53.477509 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc40893d-f19b-4f8d-9cc2-8ac7c443ba65 None admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:39:53.477781 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc40893d-f19b-4f8d-9cc2-8ac7c443ba65 None admin] PATH_INFO: `/v3/users/placement` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:39:53.484553 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc40893d-f19b-4f8d-9cc2-8ac7c443ba65 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:39:53.485976 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc40893d-f19b-4f8d-9cc2-8ac7c443ba65 None admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:39:53.497967 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application [None req-dc40893d-f19b-4f8d-9cc2-8ac7c443ba65 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 01 16:39:53.497967 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 16:39:53.497967 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 16:39:53.497967 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 16:39:53.497967 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 16:39:53.497967 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 16:39:53.497967 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 16:39:53.497967 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 16:39:53.497967 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 16:39:53.497967 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 16:39:53.497967 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 16:39:53.497967 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 16:39:53.497967 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 01 16:39:53.497967 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 01 16:39:53.497967 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 01 16:39:53.497967 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 01 16:39:53.499250 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 16:39:53.499250 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 16:39:53.499250 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 01 16:39:53.499250 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 16:39:53.499250 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 01 16:39:53.499250 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 16:39:53.499250 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 16:39:53.499250 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 16:39:53.499250 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 16:39:53.499250 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 16:39:53.499250 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 16:39:53.499250 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application with Lock( Aug 01 16:39:53.499250 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 16:39:53.499250 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self._enter() Aug 01 16:39:53.499250 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 16:39:53.499250 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 16:39:53.499250 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 16:39:53.499250 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self.creator() Aug 01 16:39:53.500229 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 16:39:53.500229 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application created_value = creator( Aug 01 16:39:53.500229 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 01 16:39:53.500229 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 01 16:39:53.500229 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 01 16:39:53.500229 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 01 16:39:53.500229 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 01 16:39:53.500229 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 01 16:39:53.500229 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 01 16:39:53.500229 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application Aug 01 16:39:53.500229 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 598/1212] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Aug 1 16:39:53 2022] GET /identity/v3/users/placement => generated 87 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 16:39:53.504944 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a7b4c1e-4a16-4302-9a71-3c95439dab13 None admin] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:39:53.604480 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a7b4c1e-4a16-4302-9a71-3c95439dab13 None admin] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:39:53.638318 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43fc8995-68bf-48d8-99af-b149322a6f97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:39:53.638929 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43fc8995-68bf-48d8-99af-b149322a6f97 None admin] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:39:53.639204 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43fc8995-68bf-48d8-99af-b149322a6f97 None admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:39:53.639472 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43fc8995-68bf-48d8-99af-b149322a6f97 None admin] PATH_INFO: `/v3/users` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:39:53.640256 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43fc8995-68bf-48d8-99af-b149322a6f97 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=87058) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:39:53.640571 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43fc8995-68bf-48d8-99af-b149322a6f97 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:39:53.641974 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43fc8995-68bf-48d8-99af-b149322a6f97 None admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:39:53.659367 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 615/1213] 10.222.0.20 () {56 vars in 1143 bytes} [Mon Aug 1 16:39:53 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:39:53.665193 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc40893d-f19b-4f8d-9cc2-8ac7c443ba65 None admin] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:39:53.764964 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc40893d-f19b-4f8d-9cc2-8ac7c443ba65 None admin] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:39:53.797874 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f065ca-bbc1-456b-b0c5-1be2d86048b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:39:53.798512 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f065ca-bbc1-456b-b0c5-1be2d86048b0 None admin] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:39:53.798789 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f065ca-bbc1-456b-b0c5-1be2d86048b0 None admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:39:53.799061 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f065ca-bbc1-456b-b0c5-1be2d86048b0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:39:53.828555 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG dbcounter [-] [87057] Writing DB stats keystone:SELECT=100 {{(pid=87057) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:39:53.865449 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6f065ca-bbc1-456b-b0c5-1be2d86048b0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:39:53.868119 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6f065ca-bbc1-456b-b0c5-1be2d86048b0 None admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:39:53.966469 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 599/1214] 10.222.0.20 () {58 vars in 1282 bytes} [Mon Aug 1 16:39:53 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 302 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 01 16:39:53.972511 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43fc8995-68bf-48d8-99af-b149322a6f97 None admin] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:39:53.992747 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG dbcounter [-] [87058] Writing DB stats keystone:SELECT=100 {{(pid=87058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:39:54.072144 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43fc8995-68bf-48d8-99af-b149322a6f97 None admin] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:39:54.106792 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-236983b5-1b09-407b-b8d8-77f3fa715d2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:39:54.107463 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236983b5-1b09-407b-b8d8-77f3fa715d2c None admin] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:39:54.107742 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236983b5-1b09-407b-b8d8-77f3fa715d2c None admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:39:54.108014 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236983b5-1b09-407b-b8d8-77f3fa715d2c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:39:54.118661 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-236983b5-1b09-407b-b8d8-77f3fa715d2c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:39:54.120098 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-236983b5-1b09-407b-b8d8-77f3fa715d2c None admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:39:54.134474 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application [None req-236983b5-1b09-407b-b8d8-77f3fa715d2c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 16:39:54.134474 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 16:39:54.134474 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 16:39:54.134474 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 16:39:54.134474 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 16:39:54.134474 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 16:39:54.134474 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 16:39:54.134474 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 16:39:54.134474 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 16:39:54.134474 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 16:39:54.134474 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 16:39:54.134474 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 16:39:54.134474 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 16:39:54.134474 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 16:39:54.134474 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 16:39:54.139000 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 16:39:54.139000 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 16:39:54.139000 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 16:39:54.139000 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 16:39:54.139000 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 16:39:54.139000 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 16:39:54.139000 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 16:39:54.139000 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 16:39:54.139000 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application with Lock( Aug 01 16:39:54.139000 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 16:39:54.139000 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self._enter() Aug 01 16:39:54.139000 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 16:39:54.139000 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 16:39:54.139000 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 16:39:54.139000 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self.creator() Aug 01 16:39:54.139000 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 16:39:54.139000 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application created_value = creator( Aug 01 16:39:54.140163 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 16:39:54.140163 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 16:39:54.140163 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 16:39:54.140163 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application Aug 01 16:39:54.140163 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 616/1215] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Aug 1 16:39:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 16:39:54.141836 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f065ca-bbc1-456b-b0c5-1be2d86048b0 None admin] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:39:54.202661 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG dbcounter [-] [87057] Writing DB stats keystone:SELECT=100 {{(pid=87057) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:39:54.240207 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f065ca-bbc1-456b-b0c5-1be2d86048b0 None admin] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:39:54.273288 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1074c68e-82ec-46a5-8252-ec4d5b78da81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:39:54.273895 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1074c68e-82ec-46a5-8252-ec4d5b78da81 None admin] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:39:54.274207 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1074c68e-82ec-46a5-8252-ec4d5b78da81 None admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:39:54.274483 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1074c68e-82ec-46a5-8252-ec4d5b78da81 None admin] PATH_INFO: `/v3/domains` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:39:54.275171 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1074c68e-82ec-46a5-8252-ec4d5b78da81 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87057) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:39:54.275476 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1074c68e-82ec-46a5-8252-ec4d5b78da81 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:39:54.276907 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1074c68e-82ec-46a5-8252-ec4d5b78da81 None admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:39:54.289341 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 600/1216] 10.222.0.20 () {56 vars in 1109 bytes} [Mon Aug 1 16:39:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:39:54.295017 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-236983b5-1b09-407b-b8d8-77f3fa715d2c None admin] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:39:54.367767 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG dbcounter [-] [87058] Writing DB stats keystone:SELECT=100 {{(pid=87058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:39:54.398515 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-236983b5-1b09-407b-b8d8-77f3fa715d2c None admin] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:39:54.431488 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6c1c784-cd3b-4ac1-9f94-8d320e712482 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:39:54.432124 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c1c784-cd3b-4ac1-9f94-8d320e712482 None admin] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:39:54.432408 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c1c784-cd3b-4ac1-9f94-8d320e712482 None admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:39:54.432682 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c1c784-cd3b-4ac1-9f94-8d320e712482 None admin] PATH_INFO: `/v3/projects/service` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:39:54.439425 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6c1c784-cd3b-4ac1-9f94-8d320e712482 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:39:54.441370 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6c1c784-cd3b-4ac1-9f94-8d320e712482 None admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:39:54.452817 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application [None req-c6c1c784-cd3b-4ac1-9f94-8d320e712482 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 16:39:54.452817 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 16:39:54.452817 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 16:39:54.452817 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 16:39:54.452817 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 16:39:54.452817 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 16:39:54.452817 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 16:39:54.452817 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 16:39:54.452817 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 16:39:54.452817 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 16:39:54.452817 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 16:39:54.452817 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 16:39:54.452817 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 01 16:39:54.452817 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 01 16:39:54.452817 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 01 16:39:54.453885 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 01 16:39:54.453885 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 16:39:54.453885 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 16:39:54.453885 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 16:39:54.453885 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 16:39:54.453885 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 16:39:54.453885 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 16:39:54.453885 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 16:39:54.453885 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application with Lock( Aug 01 16:39:54.453885 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 16:39:54.453885 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self._enter() Aug 01 16:39:54.453885 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 16:39:54.453885 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 16:39:54.453885 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 16:39:54.453885 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self.creator() Aug 01 16:39:54.453885 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 16:39:54.453885 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application created_value = creator( Aug 01 16:39:54.455019 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 01 16:39:54.455019 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 01 16:39:54.455019 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 01 16:39:54.455019 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 01 16:39:54.455019 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 01 16:39:54.455019 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 01 16:39:54.455019 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 16:39:54.455019 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application Aug 01 16:39:54.455019 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 617/1217] 10.222.0.20 () {56 vars in 1111 bytes} [Mon Aug 1 16:39:54 2022] GET /identity/v3/projects/service => generated 88 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 16:39:54.460795 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1074c68e-82ec-46a5-8252-ec4d5b78da81 None admin] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:39:54.557706 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1074c68e-82ec-46a5-8252-ec4d5b78da81 None admin] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:39:54.579776 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG dbcounter [-] [87057] Writing DB stats keystone:SELECT=100 {{(pid=87057) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:39:54.591364 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1038c20d-c7f5-4bb8-aac5-91eb16ca8709 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:39:54.591979 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1038c20d-c7f5-4bb8-aac5-91eb16ca8709 None admin] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:39:54.592276 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1038c20d-c7f5-4bb8-aac5-91eb16ca8709 None admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:39:54.592550 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1038c20d-c7f5-4bb8-aac5-91eb16ca8709 None admin] PATH_INFO: `/v3/projects` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:39:54.593297 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1038c20d-c7f5-4bb8-aac5-91eb16ca8709 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87057) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:39:54.593601 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1038c20d-c7f5-4bb8-aac5-91eb16ca8709 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:39:54.595101 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1038c20d-c7f5-4bb8-aac5-91eb16ca8709 None admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:39:54.606522 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 601/1218] 10.222.0.20 () {56 vars in 1148 bytes} [Mon Aug 1 16:39:54 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:39:54.612765 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6c1c784-cd3b-4ac1-9f94-8d320e712482 None admin] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:39:54.716948 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6c1c784-cd3b-4ac1-9f94-8d320e712482 None admin] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:39:54.744291 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG dbcounter [-] [87058] Writing DB stats keystone:SELECT=100 {{(pid=87058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:39:54.750048 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4e0f884-54a9-403a-9b2b-7ded93abbd87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:39:54.750668 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e0f884-54a9-403a-9b2b-7ded93abbd87 None admin] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:39:54.750946 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e0f884-54a9-403a-9b2b-7ded93abbd87 None admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:39:54.751213 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e0f884-54a9-403a-9b2b-7ded93abbd87 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:39:54.751996 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4e0f884-54a9-403a-9b2b-7ded93abbd87 None admin] RBAC: Adding query filter params (role.id=51bb93e277c446b49bb80a32afcd3af7, scope.project.id=7b992d6a65df4bf9b27c13192d57c5f0, user.id=8362293261614623bcdabc4120a97a9d) {{(pid=87058) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:39:54.752306 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4e0f884-54a9-403a-9b2b-7ded93abbd87 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:39:54.753726 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4e0f884-54a9-403a-9b2b-7ded93abbd87 None admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:39:54.759874 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 618/1219] 10.222.0.20 () {56 vars in 1374 bytes} [Mon Aug 1 16:39:54 2022] GET /identity/v3/role_assignments?user.id=8362293261614623bcdabc4120a97a9d&scope.project.id=7b992d6a65df4bf9b27c13192d57c5f0&role.id=51bb93e277c446b49bb80a32afcd3af7 => generated 259 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:39:56.729112 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be686c32-90ca-450f-8c24-409ee7c01f2f None None] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:39:56.729657 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be686c32-90ca-450f-8c24-409ee7c01f2f None None] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:39:56.729657 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be686c32-90ca-450f-8c24-409ee7c01f2f None None] PATH_INFO: `/` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:39:56.731047 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 602/1220] 10.222.0.20 () {54 vars in 926 bytes} [Mon Aug 1 16:39:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 16:39:56.736559 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29d9740-54c0-40fa-ba5c-61b2b0ffdd27 None None] REQUEST_METHOD: `POST` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:39:56.737085 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29d9740-54c0-40fa-ba5c-61b2b0ffdd27 None None] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:39:56.737167 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29d9740-54c0-40fa-ba5c-61b2b0ffdd27 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:39:56.755833 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.auth.core [None req-a29d9740-54c0-40fa-ba5c-61b2b0ffdd27 None None] MFA Rules not processed for user `1007f8e379b74e169b98fb7c3c0ba5b4`. Rule list: `[]` (Enabled: `True`). {{(pid=87058) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:39:56.757716 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.fernet_utils [None req-a29d9740-54c0-40fa-ba5c-61b2b0ffdd27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:39:56.827698 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 619/1221] 10.222.0.20 () {58 vars in 1023 bytes} [Mon Aug 1 16:39:56 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 01 16:39:56.850614 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be686c32-90ca-450f-8c24-409ee7c01f2f None None] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:39:56.955631 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be686c32-90ca-450f-8c24-409ee7c01f2f None None] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:39:56.992534 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6351cb2-739e-4ad7-a660-93833792968f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:39:56.993083 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6351cb2-739e-4ad7-a660-93833792968f None admin] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:39:56.993359 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6351cb2-739e-4ad7-a660-93833792968f None admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:39:56.993627 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6351cb2-739e-4ad7-a660-93833792968f None admin] PATH_INFO: `/v3/roles/service` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:39:57.003372 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6351cb2-739e-4ad7-a660-93833792968f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 16:39:57.008668 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application [None req-b6351cb2-739e-4ad7-a660-93833792968f None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 01 16:39:57.008668 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 16:39:57.008668 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 16:39:57.008668 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 16:39:57.008668 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 16:39:57.008668 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 16:39:57.008668 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 16:39:57.008668 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 16:39:57.008668 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 16:39:57.008668 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 16:39:57.008668 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 16:39:57.008668 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 16:39:57.008668 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 16:39:57.008668 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 16:39:57.008668 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 16:39:57.008668 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 16:39:57.009789 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 16:39:57.009789 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 16:39:57.009789 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 16:39:57.009789 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 16:39:57.009789 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 16:39:57.009789 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 16:39:57.009789 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 16:39:57.009789 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 16:39:57.009789 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 16:39:57.009789 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 16:39:57.009789 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 16:39:57.009789 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application with Lock( Aug 01 16:39:57.009789 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 16:39:57.009789 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self._enter() Aug 01 16:39:57.009789 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 16:39:57.009789 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 16:39:57.010898 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 16:39:57.010898 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self.creator() Aug 01 16:39:57.010898 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 16:39:57.010898 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application created_value = creator( Aug 01 16:39:57.010898 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 16:39:57.010898 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 16:39:57.010898 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 16:39:57.010898 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 16:39:57.010898 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 16:39:57.010898 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 16:39:57.010898 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 01 16:39:57.010898 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application Aug 01 16:39:57.010898 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 603/1222] 10.222.0.20 () {56 vars in 1102 bytes} [Mon Aug 1 16:39:56 2022] GET /identity/v3/roles/service => generated 85 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 01 16:39:57.018302 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a29d9740-54c0-40fa-ba5c-61b2b0ffdd27 None None] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:39:57.116528 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a29d9740-54c0-40fa-ba5c-61b2b0ffdd27 None None] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:39:57.140377 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG dbcounter [-] [87058] Writing DB stats keystone:SELECT=100 {{(pid=87058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:39:57.151501 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d13a1f5a-2975-41de-8c5c-f2b45a046df8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:39:57.152116 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13a1f5a-2975-41de-8c5c-f2b45a046df8 None admin] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:39:57.152446 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13a1f5a-2975-41de-8c5c-f2b45a046df8 None admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:39:57.152720 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13a1f5a-2975-41de-8c5c-f2b45a046df8 None admin] PATH_INFO: `/v3/roles` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:39:57.153431 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d13a1f5a-2975-41de-8c5c-f2b45a046df8 None admin] RBAC: Adding query filter params (name=service) {{(pid=87058) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:39:57.153732 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d13a1f5a-2975-41de-8c5c-f2b45a046df8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:39:57.155154 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d13a1f5a-2975-41de-8c5c-f2b45a046df8 None admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:39:57.163218 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 620/1223] 10.222.0.20 () {56 vars in 1103 bytes} [Mon Aug 1 16:39:57 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:39:57.169005 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6351cb2-739e-4ad7-a660-93833792968f None admin] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:39:57.213289 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG dbcounter [-] [87057] Writing DB stats keystone:SELECT=100 {{(pid=87057) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:39:57.268489 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6351cb2-739e-4ad7-a660-93833792968f None admin] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:39:57.301675 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c2fa3bb-34f8-486b-ae5f-bf65c3d2d278 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:39:57.302351 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2fa3bb-34f8-486b-ae5f-bf65c3d2d278 None admin] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:39:57.302632 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2fa3bb-34f8-486b-ae5f-bf65c3d2d278 None admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:39:57.302904 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2fa3bb-34f8-486b-ae5f-bf65c3d2d278 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:39:57.313316 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c2fa3bb-34f8-486b-ae5f-bf65c3d2d278 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:39:57.314817 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c2fa3bb-34f8-486b-ae5f-bf65c3d2d278 None admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:39:57.329999 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application [None req-2c2fa3bb-34f8-486b-ae5f-bf65c3d2d278 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 16:39:57.329999 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 16:39:57.329999 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 16:39:57.329999 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 16:39:57.329999 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 16:39:57.329999 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 16:39:57.329999 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 16:39:57.329999 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 16:39:57.329999 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 16:39:57.329999 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 16:39:57.329999 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 16:39:57.329999 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 16:39:57.329999 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 16:39:57.329999 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 16:39:57.329999 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 16:39:57.331125 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 16:39:57.331125 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 16:39:57.331125 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 16:39:57.331125 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 16:39:57.331125 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 16:39:57.331125 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 16:39:57.331125 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 16:39:57.331125 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 16:39:57.331125 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application with Lock( Aug 01 16:39:57.331125 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 16:39:57.331125 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self._enter() Aug 01 16:39:57.331125 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 16:39:57.331125 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 16:39:57.331125 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 16:39:57.331125 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self.creator() Aug 01 16:39:57.331125 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 16:39:57.331125 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application created_value = creator( Aug 01 16:39:57.332094 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 16:39:57.332094 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 16:39:57.332094 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 16:39:57.332094 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application Aug 01 16:39:57.332094 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 604/1224] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Aug 1 16:39:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 16:39:57.337295 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d13a1f5a-2975-41de-8c5c-f2b45a046df8 None admin] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:39:57.437715 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d13a1f5a-2975-41de-8c5c-f2b45a046df8 None admin] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:39:57.471095 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1cc8805-e4f4-4029-ad9a-c1b4b1047259 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:39:57.471710 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1cc8805-e4f4-4029-ad9a-c1b4b1047259 None admin] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:39:57.471991 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1cc8805-e4f4-4029-ad9a-c1b4b1047259 None admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:39:57.472263 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1cc8805-e4f4-4029-ad9a-c1b4b1047259 None admin] PATH_INFO: `/v3/domains` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:39:57.472949 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1cc8805-e4f4-4029-ad9a-c1b4b1047259 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87058) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:39:57.473251 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1cc8805-e4f4-4029-ad9a-c1b4b1047259 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:39:57.474636 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1cc8805-e4f4-4029-ad9a-c1b4b1047259 None admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:39:57.486258 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 621/1225] 10.222.0.20 () {56 vars in 1109 bytes} [Mon Aug 1 16:39:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:39:57.492659 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c2fa3bb-34f8-486b-ae5f-bf65c3d2d278 None admin] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:39:57.584029 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG dbcounter [-] [87057] Writing DB stats keystone:SELECT=100 {{(pid=87057) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:39:57.590724 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c2fa3bb-34f8-486b-ae5f-bf65c3d2d278 None admin] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:39:57.623960 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a46d5cd-9a5c-4902-8fec-f0831822d8e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:39:57.624645 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a46d5cd-9a5c-4902-8fec-f0831822d8e2 None admin] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:39:57.624933 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a46d5cd-9a5c-4902-8fec-f0831822d8e2 None admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:39:57.625210 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a46d5cd-9a5c-4902-8fec-f0831822d8e2 None admin] PATH_INFO: `/v3/users/placement` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:39:57.632415 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a46d5cd-9a5c-4902-8fec-f0831822d8e2 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:39:57.633867 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a46d5cd-9a5c-4902-8fec-f0831822d8e2 None admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:39:57.645821 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application [None req-4a46d5cd-9a5c-4902-8fec-f0831822d8e2 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 01 16:39:57.645821 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 16:39:57.645821 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 16:39:57.645821 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 16:39:57.645821 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 16:39:57.645821 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 16:39:57.645821 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 16:39:57.645821 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 16:39:57.645821 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 16:39:57.645821 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 16:39:57.645821 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 16:39:57.645821 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 16:39:57.645821 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 01 16:39:57.645821 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 01 16:39:57.645821 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 01 16:39:57.645821 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 01 16:39:57.648429 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 16:39:57.648429 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 16:39:57.648429 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 01 16:39:57.648429 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 16:39:57.648429 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 01 16:39:57.648429 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 16:39:57.648429 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 16:39:57.648429 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 16:39:57.648429 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 16:39:57.648429 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 16:39:57.648429 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 16:39:57.648429 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application with Lock( Aug 01 16:39:57.648429 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 16:39:57.648429 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self._enter() Aug 01 16:39:57.648429 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 16:39:57.648429 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 16:39:57.648429 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 16:39:57.648429 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self.creator() Aug 01 16:39:57.649415 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 16:39:57.649415 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application created_value = creator( Aug 01 16:39:57.649415 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 01 16:39:57.649415 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 01 16:39:57.649415 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 01 16:39:57.649415 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 01 16:39:57.649415 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 01 16:39:57.649415 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 01 16:39:57.649415 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 01 16:39:57.649415 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application Aug 01 16:39:57.649415 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 605/1226] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Aug 1 16:39:57 2022] GET /identity/v3/users/placement => generated 87 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 16:39:57.652925 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1cc8805-e4f4-4029-ad9a-c1b4b1047259 None admin] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:39:57.699652 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG dbcounter [-] [87058] Writing DB stats keystone:SELECT=100 {{(pid=87058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:39:57.751650 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1cc8805-e4f4-4029-ad9a-c1b4b1047259 None admin] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:39:57.784767 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced3f456-61ab-46a0-93be-43d7b4eb8df1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:39:57.785377 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced3f456-61ab-46a0-93be-43d7b4eb8df1 None admin] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:39:57.785658 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced3f456-61ab-46a0-93be-43d7b4eb8df1 None admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:39:57.785930 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced3f456-61ab-46a0-93be-43d7b4eb8df1 None admin] PATH_INFO: `/v3/users` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:39:57.786751 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ced3f456-61ab-46a0-93be-43d7b4eb8df1 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=87058) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:39:57.787052 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ced3f456-61ab-46a0-93be-43d7b4eb8df1 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:39:57.789265 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ced3f456-61ab-46a0-93be-43d7b4eb8df1 None admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:39:57.807995 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 622/1227] 10.222.0.20 () {56 vars in 1143 bytes} [Mon Aug 1 16:39:57 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:39:57.813547 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a46d5cd-9a5c-4902-8fec-f0831822d8e2 None admin] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:39:57.914104 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a46d5cd-9a5c-4902-8fec-f0831822d8e2 None admin] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:39:57.947745 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a46369d8-b548-446e-8dcb-4aa8f412b38e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:39:57.948380 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46369d8-b548-446e-8dcb-4aa8f412b38e None admin] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:39:57.948659 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46369d8-b548-446e-8dcb-4aa8f412b38e None admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:39:57.948931 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46369d8-b548-446e-8dcb-4aa8f412b38e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:39:57.959987 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a46369d8-b548-446e-8dcb-4aa8f412b38e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:39:57.961429 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a46369d8-b548-446e-8dcb-4aa8f412b38e None admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:39:57.975987 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application [None req-a46369d8-b548-446e-8dcb-4aa8f412b38e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 16:39:57.975987 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 16:39:57.975987 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 16:39:57.975987 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 16:39:57.975987 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 16:39:57.975987 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 16:39:57.975987 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 16:39:57.975987 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 16:39:57.975987 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 16:39:57.975987 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 16:39:57.975987 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 16:39:57.975987 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 16:39:57.975987 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 16:39:57.975987 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 16:39:57.975987 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 16:39:57.976889 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 16:39:57.976889 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 16:39:57.976889 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 16:39:57.976889 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 16:39:57.976889 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 16:39:57.976889 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 16:39:57.976889 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 16:39:57.976889 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 16:39:57.976889 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application with Lock( Aug 01 16:39:57.976889 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 16:39:57.976889 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self._enter() Aug 01 16:39:57.976889 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 16:39:57.976889 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 16:39:57.976889 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 16:39:57.976889 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self.creator() Aug 01 16:39:57.976889 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 16:39:57.976889 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application created_value = creator( Aug 01 16:39:57.977904 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 16:39:57.977904 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 16:39:57.977904 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 16:39:57.977904 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application Aug 01 16:39:57.977904 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 606/1228] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Aug 1 16:39:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 16:39:57.983049 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced3f456-61ab-46a0-93be-43d7b4eb8df1 None admin] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:39:58.082273 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced3f456-61ab-46a0-93be-43d7b4eb8df1 None admin] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:39:58.085097 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG dbcounter [-] [87058] Writing DB stats keystone:SELECT=100 {{(pid=87058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:39:58.117770 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5761536b-e17a-427e-be93-d8f246166317 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:39:58.118412 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5761536b-e17a-427e-be93-d8f246166317 None admin] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:39:58.118688 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5761536b-e17a-427e-be93-d8f246166317 None admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:39:58.118958 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5761536b-e17a-427e-be93-d8f246166317 None admin] PATH_INFO: `/v3/domains` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:39:58.119642 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5761536b-e17a-427e-be93-d8f246166317 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87058) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:39:58.119939 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5761536b-e17a-427e-be93-d8f246166317 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:39:58.121302 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5761536b-e17a-427e-be93-d8f246166317 None admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:39:58.132051 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 623/1229] 10.222.0.20 () {56 vars in 1109 bytes} [Mon Aug 1 16:39:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:39:58.137956 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a46369d8-b548-446e-8dcb-4aa8f412b38e None admin] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:39:58.148217 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG dbcounter [-] [87057] Writing DB stats keystone:SELECT=100 {{(pid=87057) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:39:58.237271 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a46369d8-b548-446e-8dcb-4aa8f412b38e None admin] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:39:58.269913 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b5ae70d-509a-4a2f-b2a5-3f62377ffe78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:39:58.270571 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5ae70d-509a-4a2f-b2a5-3f62377ffe78 None admin] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:39:58.270847 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5ae70d-509a-4a2f-b2a5-3f62377ffe78 None admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:39:58.271125 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5ae70d-509a-4a2f-b2a5-3f62377ffe78 None admin] PATH_INFO: `/v3/projects/service` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:39:58.278389 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b5ae70d-509a-4a2f-b2a5-3f62377ffe78 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:39:58.280690 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b5ae70d-509a-4a2f-b2a5-3f62377ffe78 None admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:39:58.291841 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application [None req-7b5ae70d-509a-4a2f-b2a5-3f62377ffe78 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 16:39:58.291841 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 16:39:58.291841 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 16:39:58.291841 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 16:39:58.291841 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 16:39:58.291841 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 16:39:58.291841 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 16:39:58.291841 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 16:39:58.291841 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 16:39:58.291841 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 16:39:58.291841 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 16:39:58.291841 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 16:39:58.291841 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 01 16:39:58.291841 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 01 16:39:58.291841 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 01 16:39:58.292893 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 01 16:39:58.292893 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 16:39:58.292893 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 16:39:58.292893 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 16:39:58.292893 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 16:39:58.292893 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 16:39:58.292893 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 16:39:58.292893 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 16:39:58.292893 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application with Lock( Aug 01 16:39:58.292893 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 16:39:58.292893 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self._enter() Aug 01 16:39:58.292893 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 16:39:58.292893 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 16:39:58.292893 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 16:39:58.292893 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self.creator() Aug 01 16:39:58.292893 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 16:39:58.292893 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application created_value = creator( Aug 01 16:39:58.294188 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 01 16:39:58.294188 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 01 16:39:58.294188 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 01 16:39:58.294188 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 01 16:39:58.294188 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 01 16:39:58.294188 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 01 16:39:58.294188 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 16:39:58.294188 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application Aug 01 16:39:58.294188 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 607/1230] 10.222.0.20 () {56 vars in 1111 bytes} [Mon Aug 1 16:39:58 2022] GET /identity/v3/projects/service => generated 88 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 16:39:58.299021 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5761536b-e17a-427e-be93-d8f246166317 None admin] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:39:58.400032 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5761536b-e17a-427e-be93-d8f246166317 None admin] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:39:58.432595 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c151b0-4aac-42d8-83db-5631514afeeb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:39:58.433208 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c151b0-4aac-42d8-83db-5631514afeeb None admin] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:39:58.433491 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c151b0-4aac-42d8-83db-5631514afeeb None admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:39:58.433763 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c151b0-4aac-42d8-83db-5631514afeeb None admin] PATH_INFO: `/v3/projects` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:39:58.434535 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3c151b0-4aac-42d8-83db-5631514afeeb None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87058) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:39:58.434842 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3c151b0-4aac-42d8-83db-5631514afeeb None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:39:58.436267 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3c151b0-4aac-42d8-83db-5631514afeeb None admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:39:58.448818 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 624/1231] 10.222.0.20 () {56 vars in 1148 bytes} [Mon Aug 1 16:39:58 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:39:58.454800 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b5ae70d-509a-4a2f-b2a5-3f62377ffe78 None admin] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:39:58.519980 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG dbcounter [-] [87057] Writing DB stats keystone:SELECT=100 {{(pid=87057) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:39:58.554840 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b5ae70d-509a-4a2f-b2a5-3f62377ffe78 None admin] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:39:58.588705 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e63f4d-a8b4-4cd6-81c3-0308dd9e39a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:39:58.589370 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e63f4d-a8b4-4cd6-81c3-0308dd9e39a5 None admin] REQUEST_METHOD: `PUT` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:39:58.589648 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e63f4d-a8b4-4cd6-81c3-0308dd9e39a5 None admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:39:58.589919 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e63f4d-a8b4-4cd6-81c3-0308dd9e39a5 None admin] PATH_INFO: `/v3/projects/7b992d6a65df4bf9b27c13192d57c5f0/users/8362293261614623bcdabc4120a97a9d/roles/51bb93e277c446b49bb80a32afcd3af7` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:39:58.604469 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6e63f4d-a8b4-4cd6-81c3-0308dd9e39a5 None admin] RBAC: Authorizing `identity:create_grant(project_id=7b992d6a65df4bf9b27c13192d57c5f0, user_id=8362293261614623bcdabc4120a97a9d, role_id=51bb93e277c446b49bb80a32afcd3af7)` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:39:58.610189 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6e63f4d-a8b4-4cd6-81c3-0308dd9e39a5 None admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:39:58.733010 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 608/1232] 10.222.0.20 () {58 vars in 1439 bytes} [Mon Aug 1 16:39:58 2022] PUT /identity/v3/projects/7b992d6a65df4bf9b27c13192d57c5f0/users/8362293261614623bcdabc4120a97a9d/roles/51bb93e277c446b49bb80a32afcd3af7 => generated 0 bytes in 279 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 16:40:00.733235 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e37182f-d32c-4bc6-84d0-201ec50645b5 None None] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:40:00.733832 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e37182f-d32c-4bc6-84d0-201ec50645b5 None None] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:40:00.733832 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e37182f-d32c-4bc6-84d0-201ec50645b5 None None] PATH_INFO: `/` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:40:00.735197 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 625/1233] 10.222.0.20 () {54 vars in 926 bytes} [Mon Aug 1 16:40:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 16:40:00.741035 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30a7a35-4185-41ab-9cb7-c9de275aa33a None None] REQUEST_METHOD: `POST` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:40:00.741469 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30a7a35-4185-41ab-9cb7-c9de275aa33a None None] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:40:00.741613 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30a7a35-4185-41ab-9cb7-c9de275aa33a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:40:00.759364 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.auth.core [None req-b30a7a35-4185-41ab-9cb7-c9de275aa33a None None] MFA Rules not processed for user `1007f8e379b74e169b98fb7c3c0ba5b4`. Rule list: `[]` (Enabled: `True`). {{(pid=87057) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:40:00.760550 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.fernet_utils [None req-b30a7a35-4185-41ab-9cb7-c9de275aa33a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87057) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:40:00.827018 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 609/1234] 10.222.0.20 () {58 vars in 1023 bytes} [Mon Aug 1 16:40:00 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 01 16:40:00.849759 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e37182f-d32c-4bc6-84d0-201ec50645b5 None None] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:40:00.875333 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG dbcounter [-] [87058] Writing DB stats keystone:SELECT=100 {{(pid=87058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:40:00.950689 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e37182f-d32c-4bc6-84d0-201ec50645b5 None None] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:40:00.984830 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04fbe8b7-7c70-49ce-9ea9-88fdb9f90ab6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:40:00.985408 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04fbe8b7-7c70-49ce-9ea9-88fdb9f90ab6 None admin] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:40:00.985686 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04fbe8b7-7c70-49ce-9ea9-88fdb9f90ab6 None admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:40:00.985959 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04fbe8b7-7c70-49ce-9ea9-88fdb9f90ab6 None admin] PATH_INFO: `/v3/roles/service` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:40:00.996357 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04fbe8b7-7c70-49ce-9ea9-88fdb9f90ab6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 16:40:01.001241 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application [None req-04fbe8b7-7c70-49ce-9ea9-88fdb9f90ab6 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 01 16:40:01.001241 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 16:40:01.001241 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 16:40:01.001241 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 16:40:01.001241 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 16:40:01.001241 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 16:40:01.001241 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 16:40:01.001241 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 16:40:01.001241 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 16:40:01.001241 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 16:40:01.001241 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 16:40:01.001241 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 16:40:01.001241 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 16:40:01.001241 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 16:40:01.001241 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 16:40:01.001241 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 16:40:01.002343 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 16:40:01.002343 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 16:40:01.002343 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 16:40:01.002343 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 16:40:01.002343 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 16:40:01.002343 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 16:40:01.002343 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 16:40:01.002343 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 16:40:01.002343 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 16:40:01.002343 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 16:40:01.002343 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 16:40:01.002343 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application with Lock( Aug 01 16:40:01.002343 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 16:40:01.002343 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self._enter() Aug 01 16:40:01.002343 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 16:40:01.002343 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 16:40:01.003311 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 16:40:01.003311 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self.creator() Aug 01 16:40:01.003311 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 16:40:01.003311 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application created_value = creator( Aug 01 16:40:01.003311 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 16:40:01.003311 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 16:40:01.003311 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 16:40:01.003311 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 16:40:01.003311 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 16:40:01.003311 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 16:40:01.003311 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 01 16:40:01.003311 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application Aug 01 16:40:01.003311 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 626/1235] 10.222.0.20 () {56 vars in 1102 bytes} [Mon Aug 1 16:40:00 2022] GET /identity/v3/roles/service => generated 85 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 16:40:01.008623 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b30a7a35-4185-41ab-9cb7-c9de275aa33a None None] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:40:01.054629 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG dbcounter [-] [87057] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87057) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:40:01.105876 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b30a7a35-4185-41ab-9cb7-c9de275aa33a None None] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:40:01.138263 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fdcc2c5-f54e-430a-b653-7509a439e824 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:40:01.139135 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fdcc2c5-f54e-430a-b653-7509a439e824 None admin] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:40:01.139515 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fdcc2c5-f54e-430a-b653-7509a439e824 None admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:40:01.140201 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fdcc2c5-f54e-430a-b653-7509a439e824 None admin] PATH_INFO: `/v3/roles` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:40:01.140927 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fdcc2c5-f54e-430a-b653-7509a439e824 None admin] RBAC: Adding query filter params (name=service) {{(pid=87057) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:40:01.141309 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fdcc2c5-f54e-430a-b653-7509a439e824 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:40:01.142712 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fdcc2c5-f54e-430a-b653-7509a439e824 None admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:40:01.151392 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 610/1236] 10.222.0.20 () {56 vars in 1103 bytes} [Mon Aug 1 16:40:01 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:40:01.157121 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04fbe8b7-7c70-49ce-9ea9-88fdb9f90ab6 None admin] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:40:01.241631 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG dbcounter [-] [87058] Writing DB stats keystone:SELECT=100 {{(pid=87058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:40:01.256733 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04fbe8b7-7c70-49ce-9ea9-88fdb9f90ab6 None admin] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:40:01.291389 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2527c6e3-1fed-4e88-806c-bbce64b56fe9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:40:01.292042 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2527c6e3-1fed-4e88-806c-bbce64b56fe9 None admin] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:40:01.292326 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2527c6e3-1fed-4e88-806c-bbce64b56fe9 None admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:40:01.292686 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2527c6e3-1fed-4e88-806c-bbce64b56fe9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:40:01.303312 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2527c6e3-1fed-4e88-806c-bbce64b56fe9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:40:01.304743 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2527c6e3-1fed-4e88-806c-bbce64b56fe9 None admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:40:01.319764 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application [None req-2527c6e3-1fed-4e88-806c-bbce64b56fe9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 16:40:01.319764 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 16:40:01.319764 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 16:40:01.319764 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 16:40:01.319764 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 16:40:01.319764 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 16:40:01.319764 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 16:40:01.319764 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 16:40:01.319764 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 16:40:01.319764 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 16:40:01.319764 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 16:40:01.319764 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 16:40:01.319764 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 16:40:01.319764 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 16:40:01.319764 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 16:40:01.320822 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 16:40:01.320822 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 16:40:01.320822 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 16:40:01.320822 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 16:40:01.320822 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 16:40:01.320822 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 16:40:01.320822 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 16:40:01.320822 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 16:40:01.320822 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application with Lock( Aug 01 16:40:01.320822 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 16:40:01.320822 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self._enter() Aug 01 16:40:01.320822 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 16:40:01.320822 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 16:40:01.320822 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 16:40:01.320822 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self.creator() Aug 01 16:40:01.320822 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 16:40:01.320822 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application created_value = creator( Aug 01 16:40:01.321805 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 16:40:01.321805 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 16:40:01.321805 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 16:40:01.321805 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application Aug 01 16:40:01.321805 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 627/1237] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Aug 1 16:40:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 16:40:01.326868 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fdcc2c5-f54e-430a-b653-7509a439e824 None admin] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:40:01.426211 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fdcc2c5-f54e-430a-b653-7509a439e824 None admin] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:40:01.435013 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG dbcounter [-] [87057] Writing DB stats keystone:SELECT=100 {{(pid=87057) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:40:01.460799 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5100b36-2439-4550-9bef-d64ebd1726bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:40:01.461411 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5100b36-2439-4550-9bef-d64ebd1726bc None admin] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:40:01.461698 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5100b36-2439-4550-9bef-d64ebd1726bc None admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:40:01.461969 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5100b36-2439-4550-9bef-d64ebd1726bc None admin] PATH_INFO: `/v3/domains` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:40:01.463116 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5100b36-2439-4550-9bef-d64ebd1726bc None admin] RBAC: Adding query filter params (name=Default) {{(pid=87057) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:40:01.463116 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5100b36-2439-4550-9bef-d64ebd1726bc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:40:01.464334 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5100b36-2439-4550-9bef-d64ebd1726bc None admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:40:01.476495 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 611/1238] 10.222.0.20 () {56 vars in 1109 bytes} [Mon Aug 1 16:40:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:40:01.482186 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2527c6e3-1fed-4e88-806c-bbce64b56fe9 None admin] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:40:01.580621 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2527c6e3-1fed-4e88-806c-bbce64b56fe9 None admin] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:40:01.614198 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6084dfd4-c3e1-4ce3-920d-ed7073a03260 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:40:01.614830 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6084dfd4-c3e1-4ce3-920d-ed7073a03260 None admin] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:40:01.615110 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6084dfd4-c3e1-4ce3-920d-ed7073a03260 None admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:40:01.615421 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6084dfd4-c3e1-4ce3-920d-ed7073a03260 None admin] PATH_INFO: `/v3/users/placement` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:40:01.618287 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG dbcounter [-] [87058] Writing DB stats keystone:SELECT=100 {{(pid=87058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:40:01.623964 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6084dfd4-c3e1-4ce3-920d-ed7073a03260 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:40:01.625393 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6084dfd4-c3e1-4ce3-920d-ed7073a03260 None admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:40:01.637337 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application [None req-6084dfd4-c3e1-4ce3-920d-ed7073a03260 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 01 16:40:01.637337 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 16:40:01.637337 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 16:40:01.637337 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 16:40:01.637337 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 16:40:01.637337 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 16:40:01.637337 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 16:40:01.637337 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 16:40:01.637337 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 16:40:01.637337 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 16:40:01.637337 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 16:40:01.637337 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 16:40:01.637337 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 01 16:40:01.637337 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 01 16:40:01.637337 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 01 16:40:01.637337 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 01 16:40:01.648277 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 16:40:01.648277 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 16:40:01.648277 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 01 16:40:01.648277 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 16:40:01.648277 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 01 16:40:01.648277 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 16:40:01.648277 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 16:40:01.648277 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 16:40:01.648277 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 16:40:01.648277 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 16:40:01.648277 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 16:40:01.648277 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application with Lock( Aug 01 16:40:01.648277 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 16:40:01.648277 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self._enter() Aug 01 16:40:01.648277 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 16:40:01.648277 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 16:40:01.648277 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 16:40:01.648277 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self.creator() Aug 01 16:40:01.649229 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 16:40:01.649229 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application created_value = creator( Aug 01 16:40:01.649229 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 01 16:40:01.649229 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 01 16:40:01.649229 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 01 16:40:01.649229 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 01 16:40:01.649229 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 01 16:40:01.649229 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 01 16:40:01.649229 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 01 16:40:01.649229 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application Aug 01 16:40:01.649229 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 628/1239] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Aug 1 16:40:01 2022] GET /identity/v3/users/placement => generated 87 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 16:40:01.649852 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5100b36-2439-4550-9bef-d64ebd1726bc None admin] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:40:01.742497 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5100b36-2439-4550-9bef-d64ebd1726bc None admin] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:40:01.776404 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f82ee60-06c3-4f48-9343-db6c5b523fbc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:40:01.777014 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f82ee60-06c3-4f48-9343-db6c5b523fbc None admin] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:40:01.777290 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f82ee60-06c3-4f48-9343-db6c5b523fbc None admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:40:01.777558 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f82ee60-06c3-4f48-9343-db6c5b523fbc None admin] PATH_INFO: `/v3/users` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:40:01.778373 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f82ee60-06c3-4f48-9343-db6c5b523fbc None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=87057) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:40:01.784565 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f82ee60-06c3-4f48-9343-db6c5b523fbc None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:40:01.784565 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f82ee60-06c3-4f48-9343-db6c5b523fbc None admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:40:01.796553 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 612/1240] 10.222.0.20 () {56 vars in 1143 bytes} [Mon Aug 1 16:40:01 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:40:01.802379 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6084dfd4-c3e1-4ce3-920d-ed7073a03260 None admin] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:40:01.903551 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6084dfd4-c3e1-4ce3-920d-ed7073a03260 None admin] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:40:01.937527 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6498f423-9087-410f-847f-80ad4a0160a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:40:01.938176 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6498f423-9087-410f-847f-80ad4a0160a8 None admin] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:40:01.938460 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6498f423-9087-410f-847f-80ad4a0160a8 None admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:40:01.938733 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6498f423-9087-410f-847f-80ad4a0160a8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:40:02.001288 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG dbcounter [-] [87058] Writing DB stats keystone:SELECT=100 {{(pid=87058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:40:02.007817 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6498f423-9087-410f-847f-80ad4a0160a8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:40:02.009265 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6498f423-9087-410f-847f-80ad4a0160a8 None admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:40:02.106371 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 629/1241] 10.222.0.20 () {58 vars in 1282 bytes} [Mon Aug 1 16:40:01 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 01 16:40:02.112128 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f82ee60-06c3-4f48-9343-db6c5b523fbc None admin] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:40:02.140296 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG dbcounter [-] [87057] Writing DB stats keystone:SELECT=100 {{(pid=87057) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:40:02.212332 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f82ee60-06c3-4f48-9343-db6c5b523fbc None admin] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:40:02.244645 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f64cfff-b575-4805-a2fe-7912f9954d95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:40:02.245273 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f64cfff-b575-4805-a2fe-7912f9954d95 None admin] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:40:02.245585 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f64cfff-b575-4805-a2fe-7912f9954d95 None admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:40:02.245860 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f64cfff-b575-4805-a2fe-7912f9954d95 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:40:02.255894 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f64cfff-b575-4805-a2fe-7912f9954d95 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:40:02.258004 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f64cfff-b575-4805-a2fe-7912f9954d95 None admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:40:02.273056 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application [None req-3f64cfff-b575-4805-a2fe-7912f9954d95 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 16:40:02.273056 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 16:40:02.273056 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 16:40:02.273056 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 16:40:02.273056 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 16:40:02.273056 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 16:40:02.273056 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 16:40:02.273056 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 16:40:02.273056 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 16:40:02.273056 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 16:40:02.273056 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 16:40:02.273056 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 16:40:02.273056 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 16:40:02.273056 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 16:40:02.273056 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 16:40:02.274156 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 16:40:02.274156 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 16:40:02.274156 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 16:40:02.274156 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 16:40:02.274156 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 16:40:02.274156 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 16:40:02.274156 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 16:40:02.274156 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 16:40:02.274156 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application with Lock( Aug 01 16:40:02.274156 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 16:40:02.274156 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self._enter() Aug 01 16:40:02.274156 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 16:40:02.274156 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 16:40:02.274156 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 16:40:02.274156 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self.creator() Aug 01 16:40:02.274156 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 16:40:02.274156 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application created_value = creator( Aug 01 16:40:02.275184 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 16:40:02.275184 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 16:40:02.275184 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 16:40:02.275184 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application Aug 01 16:40:02.275184 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 613/1242] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Aug 1 16:40:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 16:40:02.280176 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6498f423-9087-410f-847f-80ad4a0160a8 None admin] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:40:02.374588 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG dbcounter [-] [87058] Writing DB stats keystone:SELECT=100 {{(pid=87058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:40:02.381614 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6498f423-9087-410f-847f-80ad4a0160a8 None admin] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:40:02.416863 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a8cc49-4e6d-49ca-899b-af0ffe19f59b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:40:02.417489 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a8cc49-4e6d-49ca-899b-af0ffe19f59b None admin] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:40:02.417774 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a8cc49-4e6d-49ca-899b-af0ffe19f59b None admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:40:02.418070 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a8cc49-4e6d-49ca-899b-af0ffe19f59b None admin] PATH_INFO: `/v3/domains` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:40:02.418772 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19a8cc49-4e6d-49ca-899b-af0ffe19f59b None admin] RBAC: Adding query filter params (name=Default) {{(pid=87058) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:40:02.419075 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19a8cc49-4e6d-49ca-899b-af0ffe19f59b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:40:02.420443 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19a8cc49-4e6d-49ca-899b-af0ffe19f59b None admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:40:02.432675 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 630/1243] 10.222.0.20 () {56 vars in 1109 bytes} [Mon Aug 1 16:40:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:40:02.438463 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f64cfff-b575-4805-a2fe-7912f9954d95 None admin] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:40:02.515778 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG dbcounter [-] [87057] Writing DB stats keystone:SELECT=100 {{(pid=87057) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:40:02.538324 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f64cfff-b575-4805-a2fe-7912f9954d95 None admin] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:40:02.571430 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09bc514a-a543-4030-ab09-b4ef3185c232 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:40:02.572064 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09bc514a-a543-4030-ab09-b4ef3185c232 None admin] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:40:02.572344 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09bc514a-a543-4030-ab09-b4ef3185c232 None admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:40:02.572617 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09bc514a-a543-4030-ab09-b4ef3185c232 None admin] PATH_INFO: `/v3/projects/service` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:40:02.579071 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09bc514a-a543-4030-ab09-b4ef3185c232 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:40:02.580512 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09bc514a-a543-4030-ab09-b4ef3185c232 None admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:40:02.591826 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application [None req-09bc514a-a543-4030-ab09-b4ef3185c232 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 16:40:02.591826 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 16:40:02.591826 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 16:40:02.591826 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 16:40:02.591826 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 16:40:02.591826 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 16:40:02.591826 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 16:40:02.591826 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 16:40:02.591826 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 16:40:02.591826 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 16:40:02.591826 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 16:40:02.591826 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 16:40:02.591826 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 01 16:40:02.591826 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 01 16:40:02.591826 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 01 16:40:02.592860 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 01 16:40:02.592860 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 16:40:02.592860 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 16:40:02.592860 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 16:40:02.592860 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 16:40:02.592860 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 16:40:02.592860 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 16:40:02.592860 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 16:40:02.592860 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application with Lock( Aug 01 16:40:02.592860 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 16:40:02.592860 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self._enter() Aug 01 16:40:02.592860 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 16:40:02.592860 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 16:40:02.592860 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 16:40:02.592860 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self.creator() Aug 01 16:40:02.592860 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 16:40:02.592860 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application created_value = creator( Aug 01 16:40:02.593853 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 01 16:40:02.593853 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 01 16:40:02.593853 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 01 16:40:02.593853 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 01 16:40:02.593853 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 01 16:40:02.593853 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 01 16:40:02.593853 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 16:40:02.593853 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application Aug 01 16:40:02.593853 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 614/1244] 10.222.0.20 () {56 vars in 1111 bytes} [Mon Aug 1 16:40:02 2022] GET /identity/v3/projects/service => generated 88 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 16:40:02.598963 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a8cc49-4e6d-49ca-899b-af0ffe19f59b None admin] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:40:02.701450 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a8cc49-4e6d-49ca-899b-af0ffe19f59b None admin] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:40:02.734657 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e029c4ea-1963-45e9-979d-9223d1567b4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:40:02.735268 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e029c4ea-1963-45e9-979d-9223d1567b4b None admin] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:40:02.735541 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e029c4ea-1963-45e9-979d-9223d1567b4b None admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:40:02.735808 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e029c4ea-1963-45e9-979d-9223d1567b4b None admin] PATH_INFO: `/v3/projects` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:40:02.736542 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e029c4ea-1963-45e9-979d-9223d1567b4b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87058) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:40:02.736841 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e029c4ea-1963-45e9-979d-9223d1567b4b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:40:02.738303 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e029c4ea-1963-45e9-979d-9223d1567b4b None admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:40:02.750251 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 631/1245] 10.222.0.20 () {56 vars in 1148 bytes} [Mon Aug 1 16:40:02 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:40:02.757471 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09bc514a-a543-4030-ab09-b4ef3185c232 None admin] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:40:02.856995 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09bc514a-a543-4030-ab09-b4ef3185c232 None admin] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:40:02.890500 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c78745b1-4342-4396-bc47-da2b5c4323d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:40:02.891134 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78745b1-4342-4396-bc47-da2b5c4323d2 None admin] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:40:02.891410 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78745b1-4342-4396-bc47-da2b5c4323d2 None admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:40:02.891680 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78745b1-4342-4396-bc47-da2b5c4323d2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:40:02.892459 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c78745b1-4342-4396-bc47-da2b5c4323d2 None admin] RBAC: Adding query filter params (role.id=51bb93e277c446b49bb80a32afcd3af7, scope.project.id=7b992d6a65df4bf9b27c13192d57c5f0, user.id=8362293261614623bcdabc4120a97a9d) {{(pid=87057) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:40:02.892836 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c78745b1-4342-4396-bc47-da2b5c4323d2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:40:02.894300 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c78745b1-4342-4396-bc47-da2b5c4323d2 None admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:40:02.900448 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 615/1246] 10.222.0.20 () {56 vars in 1374 bytes} [Mon Aug 1 16:40:02 2022] GET /identity/v3/role_assignments?user.id=8362293261614623bcdabc4120a97a9d&scope.project.id=7b992d6a65df4bf9b27c13192d57c5f0&role.id=51bb93e277c446b49bb80a32afcd3af7 => generated 608 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:40:04.935281 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720a4090-a347-497a-9ce2-33bc41ee4000 None None] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:40:04.935759 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720a4090-a347-497a-9ce2-33bc41ee4000 None None] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:40:04.935880 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720a4090-a347-497a-9ce2-33bc41ee4000 None None] PATH_INFO: `/` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:40:04.937187 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 632/1247] 10.222.0.20 () {54 vars in 926 bytes} [Mon Aug 1 16:40:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 16:40:04.942320 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23df677-5fef-42f4-b3d9-b215a8ea5171 None None] REQUEST_METHOD: `POST` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:40:04.942759 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23df677-5fef-42f4-b3d9-b215a8ea5171 None None] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:40:04.942948 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23df677-5fef-42f4-b3d9-b215a8ea5171 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:40:04.949272 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG dbcounter [-] [87057] Writing DB stats keystone:SELECT=100 {{(pid=87057) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:40:04.962531 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.auth.core [None req-c23df677-5fef-42f4-b3d9-b215a8ea5171 None None] MFA Rules not processed for user `1007f8e379b74e169b98fb7c3c0ba5b4`. Rule list: `[]` (Enabled: `True`). {{(pid=87057) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:40:04.963732 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.fernet_utils [None req-c23df677-5fef-42f4-b3d9-b215a8ea5171 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87057) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:40:05.028221 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 616/1248] 10.222.0.20 () {58 vars in 1023 bytes} [Mon Aug 1 16:40:04 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 01 16:40:05.051700 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-720a4090-a347-497a-9ce2-33bc41ee4000 None None] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:40:05.070415 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG dbcounter [-] [87058] Writing DB stats keystone:SELECT=100 {{(pid=87058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:40:05.154250 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-720a4090-a347-497a-9ce2-33bc41ee4000 None None] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:40:05.188303 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d1f3723-3155-4b9f-b40d-284e75b26608 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:40:05.188860 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1f3723-3155-4b9f-b40d-284e75b26608 None admin] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:40:05.189139 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1f3723-3155-4b9f-b40d-284e75b26608 None admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:40:05.189410 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1f3723-3155-4b9f-b40d-284e75b26608 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:40:05.200119 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d1f3723-3155-4b9f-b40d-284e75b26608 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 16:40:05.204917 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application [None req-8d1f3723-3155-4b9f-b40d-284e75b26608 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 16:40:05.204917 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 16:40:05.204917 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 16:40:05.204917 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 16:40:05.204917 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 16:40:05.204917 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 16:40:05.204917 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 16:40:05.204917 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 16:40:05.204917 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 16:40:05.204917 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 16:40:05.204917 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 16:40:05.204917 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 16:40:05.204917 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 16:40:05.204917 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 16:40:05.204917 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 16:40:05.204917 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 16:40:05.206168 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 16:40:05.206168 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 16:40:05.206168 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 16:40:05.206168 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 16:40:05.206168 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 16:40:05.206168 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 16:40:05.206168 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 16:40:05.206168 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 16:40:05.206168 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 16:40:05.206168 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 16:40:05.206168 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 16:40:05.206168 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application with Lock( Aug 01 16:40:05.206168 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 16:40:05.206168 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self._enter() Aug 01 16:40:05.206168 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 16:40:05.206168 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 16:40:05.207579 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 16:40:05.207579 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self.creator() Aug 01 16:40:05.207579 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 16:40:05.207579 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application created_value = creator( Aug 01 16:40:05.207579 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 16:40:05.207579 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 16:40:05.207579 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 16:40:05.207579 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 16:40:05.207579 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 16:40:05.207579 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 16:40:05.207579 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 16:40:05.207579 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application Aug 01 16:40:05.207579 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 633/1249] 10.222.0.20 () {56 vars in 1096 bytes} [Mon Aug 1 16:40:05 2022] GET /identity/v3/roles/admin => generated 83 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 16:40:05.212782 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c23df677-5fef-42f4-b3d9-b215a8ea5171 None None] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:40:05.309519 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c23df677-5fef-42f4-b3d9-b215a8ea5171 None None] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:40:05.338583 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG dbcounter [-] [87057] Writing DB stats keystone:SELECT=100 {{(pid=87057) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:40:05.343373 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ccabe7-ce7f-4296-a9c7-a2b3d9c42a2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:40:05.343991 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ccabe7-ce7f-4296-a9c7-a2b3d9c42a2a None admin] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:40:05.344266 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ccabe7-ce7f-4296-a9c7-a2b3d9c42a2a None admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:40:05.344538 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ccabe7-ce7f-4296-a9c7-a2b3d9c42a2a None admin] PATH_INFO: `/v3/roles` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:40:05.345247 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12ccabe7-ce7f-4296-a9c7-a2b3d9c42a2a None admin] RBAC: Adding query filter params (name=admin) {{(pid=87057) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:40:05.345546 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12ccabe7-ce7f-4296-a9c7-a2b3d9c42a2a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:40:05.346929 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12ccabe7-ce7f-4296-a9c7-a2b3d9c42a2a None admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:40:05.354753 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 617/1250] 10.222.0.20 () {56 vars in 1099 bytes} [Mon Aug 1 16:40:05 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:40:05.360463 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d1f3723-3155-4b9f-b40d-284e75b26608 None admin] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:40:05.437783 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG dbcounter [-] [87058] Writing DB stats keystone:SELECT=100 {{(pid=87058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:40:05.460731 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d1f3723-3155-4b9f-b40d-284e75b26608 None admin] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:40:05.493326 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93dd13b6-c7e7-452a-996a-18bb186673a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:40:05.493950 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93dd13b6-c7e7-452a-996a-18bb186673a1 None admin] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:40:05.494253 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93dd13b6-c7e7-452a-996a-18bb186673a1 None admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:40:05.494522 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93dd13b6-c7e7-452a-996a-18bb186673a1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:40:05.504616 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93dd13b6-c7e7-452a-996a-18bb186673a1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:40:05.506074 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93dd13b6-c7e7-452a-996a-18bb186673a1 None admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:40:05.520663 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application [None req-93dd13b6-c7e7-452a-996a-18bb186673a1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 16:40:05.520663 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 16:40:05.520663 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 16:40:05.520663 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 16:40:05.520663 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 16:40:05.520663 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 16:40:05.520663 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 16:40:05.520663 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 16:40:05.520663 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 16:40:05.520663 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 16:40:05.520663 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 16:40:05.520663 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 16:40:05.520663 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 16:40:05.520663 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 16:40:05.520663 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 16:40:05.521744 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 16:40:05.521744 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 16:40:05.521744 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 16:40:05.521744 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 16:40:05.521744 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 16:40:05.521744 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 16:40:05.521744 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 16:40:05.521744 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 16:40:05.521744 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application with Lock( Aug 01 16:40:05.521744 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 16:40:05.521744 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self._enter() Aug 01 16:40:05.521744 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 16:40:05.521744 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 16:40:05.521744 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 16:40:05.521744 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self.creator() Aug 01 16:40:05.521744 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 16:40:05.521744 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application created_value = creator( Aug 01 16:40:05.522938 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 16:40:05.522938 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 16:40:05.522938 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 16:40:05.522938 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application Aug 01 16:40:05.522938 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 634/1251] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Aug 1 16:40:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 16:40:05.528039 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ccabe7-ce7f-4296-a9c7-a2b3d9c42a2a None admin] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:40:05.625919 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ccabe7-ce7f-4296-a9c7-a2b3d9c42a2a None admin] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:40:05.659421 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a322a11c-0275-4683-b390-1fe47f073841 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:40:05.660033 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a322a11c-0275-4683-b390-1fe47f073841 None admin] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:40:05.660315 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a322a11c-0275-4683-b390-1fe47f073841 None admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:40:05.660590 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a322a11c-0275-4683-b390-1fe47f073841 None admin] PATH_INFO: `/v3/domains` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:40:05.661280 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a322a11c-0275-4683-b390-1fe47f073841 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87057) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:40:05.661587 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a322a11c-0275-4683-b390-1fe47f073841 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:40:05.663032 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a322a11c-0275-4683-b390-1fe47f073841 None admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:40:05.674468 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 618/1252] 10.222.0.20 () {56 vars in 1109 bytes} [Mon Aug 1 16:40:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:40:05.680628 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93dd13b6-c7e7-452a-996a-18bb186673a1 None admin] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:40:05.779460 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93dd13b6-c7e7-452a-996a-18bb186673a1 None admin] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:40:05.805859 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG dbcounter [-] [87058] Writing DB stats keystone:SELECT=100 {{(pid=87058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:40:05.813476 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c81aec7-8f80-4948-9c46-d0caeeac5718 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:40:05.814147 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c81aec7-8f80-4948-9c46-d0caeeac5718 None admin] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:40:05.814431 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c81aec7-8f80-4948-9c46-d0caeeac5718 None admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:40:05.814706 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c81aec7-8f80-4948-9c46-d0caeeac5718 None admin] PATH_INFO: `/v3/users/placement` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:40:05.822232 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c81aec7-8f80-4948-9c46-d0caeeac5718 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:40:05.826580 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c81aec7-8f80-4948-9c46-d0caeeac5718 None admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:40:05.834720 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application [None req-7c81aec7-8f80-4948-9c46-d0caeeac5718 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 01 16:40:05.834720 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 16:40:05.834720 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 16:40:05.834720 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 16:40:05.834720 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 16:40:05.834720 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 16:40:05.834720 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 16:40:05.834720 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 16:40:05.834720 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 16:40:05.834720 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 16:40:05.834720 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 16:40:05.834720 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 16:40:05.834720 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 01 16:40:05.834720 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 01 16:40:05.834720 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 01 16:40:05.834720 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 01 16:40:05.835966 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 16:40:05.835966 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 16:40:05.835966 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 01 16:40:05.835966 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 16:40:05.835966 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 01 16:40:05.835966 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 16:40:05.835966 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 16:40:05.835966 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 16:40:05.835966 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 16:40:05.835966 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 16:40:05.835966 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 16:40:05.835966 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application with Lock( Aug 01 16:40:05.835966 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 16:40:05.835966 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self._enter() Aug 01 16:40:05.835966 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 16:40:05.835966 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 16:40:05.835966 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 16:40:05.835966 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self.creator() Aug 01 16:40:05.836987 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 16:40:05.836987 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application created_value = creator( Aug 01 16:40:05.836987 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 01 16:40:05.836987 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 01 16:40:05.836987 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 01 16:40:05.836987 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 01 16:40:05.836987 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 01 16:40:05.836987 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 01 16:40:05.836987 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 01 16:40:05.836987 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application Aug 01 16:40:05.836987 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 635/1253] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Aug 1 16:40:05 2022] GET /identity/v3/users/placement => generated 87 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 16:40:05.841859 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a322a11c-0275-4683-b390-1fe47f073841 None admin] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:40:05.897058 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG dbcounter [-] [87057] Writing DB stats keystone:SELECT=100 {{(pid=87057) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:40:05.941904 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a322a11c-0275-4683-b390-1fe47f073841 None admin] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:40:05.975498 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-412b6d99-bb27-464f-a545-56d667192b8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:40:05.976834 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412b6d99-bb27-464f-a545-56d667192b8d None admin] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:40:05.977118 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412b6d99-bb27-464f-a545-56d667192b8d None admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:40:05.977391 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412b6d99-bb27-464f-a545-56d667192b8d None admin] PATH_INFO: `/v3/users` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:40:05.978219 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-412b6d99-bb27-464f-a545-56d667192b8d None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=87057) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:40:05.978524 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-412b6d99-bb27-464f-a545-56d667192b8d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:40:05.979943 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-412b6d99-bb27-464f-a545-56d667192b8d None admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:40:05.997095 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 619/1254] 10.222.0.20 () {56 vars in 1143 bytes} [Mon Aug 1 16:40:05 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:40:06.003025 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c81aec7-8f80-4948-9c46-d0caeeac5718 None admin] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:40:06.105857 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c81aec7-8f80-4948-9c46-d0caeeac5718 None admin] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:40:06.139317 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb61ecce-574a-4c4d-8574-3c068bc899a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:40:06.139933 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb61ecce-574a-4c4d-8574-3c068bc899a9 None admin] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:40:06.140213 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb61ecce-574a-4c4d-8574-3c068bc899a9 None admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:40:06.140484 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb61ecce-574a-4c4d-8574-3c068bc899a9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:40:06.196588 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG dbcounter [-] [87058] Writing DB stats keystone:SELECT=100 {{(pid=87058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:40:06.208503 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb61ecce-574a-4c4d-8574-3c068bc899a9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:40:06.210483 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb61ecce-574a-4c4d-8574-3c068bc899a9 None admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:40:06.307181 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 636/1255] 10.222.0.20 () {58 vars in 1282 bytes} [Mon Aug 1 16:40:06 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 01 16:40:06.313333 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-412b6d99-bb27-464f-a545-56d667192b8d None admin] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:40:06.412326 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-412b6d99-bb27-464f-a545-56d667192b8d None admin] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:40:06.425871 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG dbcounter [-] [87057] Writing DB stats keystone:SELECT=100 {{(pid=87057) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:40:06.447078 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4619f003-b12d-4999-886c-caa83ba908b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:40:06.447706 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4619f003-b12d-4999-886c-caa83ba908b1 None admin] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:40:06.447982 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4619f003-b12d-4999-886c-caa83ba908b1 None admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:40:06.448248 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4619f003-b12d-4999-886c-caa83ba908b1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:40:06.458287 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4619f003-b12d-4999-886c-caa83ba908b1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:40:06.460412 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4619f003-b12d-4999-886c-caa83ba908b1 None admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:40:06.475191 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application [None req-4619f003-b12d-4999-886c-caa83ba908b1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 16:40:06.475191 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 16:40:06.475191 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 16:40:06.475191 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 16:40:06.475191 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 16:40:06.475191 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 16:40:06.475191 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 16:40:06.475191 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 16:40:06.475191 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 16:40:06.475191 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 16:40:06.475191 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 16:40:06.475191 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 16:40:06.475191 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 16:40:06.475191 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 16:40:06.475191 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 16:40:06.476918 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 16:40:06.476918 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 16:40:06.476918 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 16:40:06.476918 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 16:40:06.476918 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 16:40:06.476918 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 16:40:06.476918 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 16:40:06.476918 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 16:40:06.476918 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application with Lock( Aug 01 16:40:06.476918 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 16:40:06.476918 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self._enter() Aug 01 16:40:06.476918 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 16:40:06.476918 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 16:40:06.476918 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 16:40:06.476918 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self.creator() Aug 01 16:40:06.476918 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 16:40:06.476918 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application created_value = creator( Aug 01 16:40:06.478126 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 16:40:06.478126 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 16:40:06.478126 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 16:40:06.478126 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application Aug 01 16:40:06.478126 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 620/1256] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Aug 1 16:40:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 16:40:06.482814 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb61ecce-574a-4c4d-8574-3c068bc899a9 None admin] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:40:06.570319 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG dbcounter [-] [87058] Writing DB stats keystone:SELECT=100 {{(pid=87058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:40:06.581615 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb61ecce-574a-4c4d-8574-3c068bc899a9 None admin] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:40:06.614582 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed7e45bf-8aa8-4ce3-b2e0-5c919d549a03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:40:06.615190 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7e45bf-8aa8-4ce3-b2e0-5c919d549a03 None admin] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:40:06.615472 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7e45bf-8aa8-4ce3-b2e0-5c919d549a03 None admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:40:06.615747 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7e45bf-8aa8-4ce3-b2e0-5c919d549a03 None admin] PATH_INFO: `/v3/domains` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:40:06.616561 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed7e45bf-8aa8-4ce3-b2e0-5c919d549a03 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87058) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:40:06.616867 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed7e45bf-8aa8-4ce3-b2e0-5c919d549a03 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:40:06.618263 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed7e45bf-8aa8-4ce3-b2e0-5c919d549a03 None admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:40:06.629058 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 637/1257] 10.222.0.20 () {56 vars in 1109 bytes} [Mon Aug 1 16:40:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:40:06.635258 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4619f003-b12d-4999-886c-caa83ba908b1 None admin] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:40:06.732981 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4619f003-b12d-4999-886c-caa83ba908b1 None admin] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:40:06.765841 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffa898bd-51fe-43ab-ba95-31379909b5cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:40:06.766506 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa898bd-51fe-43ab-ba95-31379909b5cc None admin] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:40:06.766788 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa898bd-51fe-43ab-ba95-31379909b5cc None admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:40:06.767063 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa898bd-51fe-43ab-ba95-31379909b5cc None admin] PATH_INFO: `/v3/projects/service` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:40:06.774264 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffa898bd-51fe-43ab-ba95-31379909b5cc None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:40:06.775674 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffa898bd-51fe-43ab-ba95-31379909b5cc None admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:40:06.786574 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application [None req-ffa898bd-51fe-43ab-ba95-31379909b5cc None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 16:40:06.786574 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 16:40:06.786574 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 16:40:06.786574 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 16:40:06.786574 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 16:40:06.786574 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 16:40:06.786574 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 16:40:06.786574 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 16:40:06.786574 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 16:40:06.786574 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 16:40:06.786574 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 16:40:06.786574 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 16:40:06.786574 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 01 16:40:06.786574 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 01 16:40:06.786574 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 01 16:40:06.798894 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 01 16:40:06.798894 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 16:40:06.798894 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 16:40:06.798894 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 16:40:06.798894 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 16:40:06.798894 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 16:40:06.798894 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 16:40:06.798894 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 16:40:06.798894 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application with Lock( Aug 01 16:40:06.798894 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 16:40:06.798894 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self._enter() Aug 01 16:40:06.798894 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 16:40:06.798894 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 16:40:06.798894 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 16:40:06.798894 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self.creator() Aug 01 16:40:06.798894 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 16:40:06.798894 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application created_value = creator( Aug 01 16:40:06.799943 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 01 16:40:06.799943 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 01 16:40:06.799943 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 01 16:40:06.799943 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 01 16:40:06.799943 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 01 16:40:06.799943 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 01 16:40:06.799943 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 16:40:06.799943 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application Aug 01 16:40:06.799943 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 621/1258] 10.222.0.20 () {56 vars in 1111 bytes} [Mon Aug 1 16:40:06 2022] GET /identity/v3/projects/service => generated 88 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 16:40:06.800504 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed7e45bf-8aa8-4ce3-b2e0-5c919d549a03 None admin] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:40:06.892518 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed7e45bf-8aa8-4ce3-b2e0-5c919d549a03 None admin] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:40:06.926026 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-820a9ee1-2ce4-400c-9e7d-3bd97364176e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:40:06.926665 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-820a9ee1-2ce4-400c-9e7d-3bd97364176e None admin] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:40:06.926941 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-820a9ee1-2ce4-400c-9e7d-3bd97364176e None admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:40:06.928253 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-820a9ee1-2ce4-400c-9e7d-3bd97364176e None admin] PATH_INFO: `/v3/projects` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:40:06.928253 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-820a9ee1-2ce4-400c-9e7d-3bd97364176e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87058) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:40:06.928253 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-820a9ee1-2ce4-400c-9e7d-3bd97364176e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:40:06.930324 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-820a9ee1-2ce4-400c-9e7d-3bd97364176e None admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:40:06.942508 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 638/1259] 10.222.0.20 () {56 vars in 1148 bytes} [Mon Aug 1 16:40:06 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:40:06.949297 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffa898bd-51fe-43ab-ba95-31379909b5cc None admin] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:40:06.975365 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG dbcounter [-] [87057] Writing DB stats keystone:SELECT=100 {{(pid=87057) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:40:07.047635 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffa898bd-51fe-43ab-ba95-31379909b5cc None admin] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:40:07.080874 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-994cd8fe-ac76-41cc-a05f-2885fbc955cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:40:07.081481 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994cd8fe-ac76-41cc-a05f-2885fbc955cd None admin] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:40:07.081764 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994cd8fe-ac76-41cc-a05f-2885fbc955cd None admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:40:07.082066 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994cd8fe-ac76-41cc-a05f-2885fbc955cd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:40:07.082849 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-994cd8fe-ac76-41cc-a05f-2885fbc955cd None admin] RBAC: Adding query filter params (role.id=440bb943dd5b46fea038b903b6eaaf81, scope.project.id=7b992d6a65df4bf9b27c13192d57c5f0, user.id=8362293261614623bcdabc4120a97a9d) {{(pid=87057) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:40:07.083158 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-994cd8fe-ac76-41cc-a05f-2885fbc955cd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:40:07.084580 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-994cd8fe-ac76-41cc-a05f-2885fbc955cd None admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:40:07.090602 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 622/1260] 10.222.0.20 () {56 vars in 1374 bytes} [Mon Aug 1 16:40:06 2022] GET /identity/v3/role_assignments?user.id=8362293261614623bcdabc4120a97a9d&scope.project.id=7b992d6a65df4bf9b27c13192d57c5f0&role.id=440bb943dd5b46fea038b903b6eaaf81 => generated 259 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:40:09.073428 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65be47ef-a43f-449b-ad66-6aaede8ffcfd None None] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:40:09.073957 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65be47ef-a43f-449b-ad66-6aaede8ffcfd None None] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:40:09.073957 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65be47ef-a43f-449b-ad66-6aaede8ffcfd None None] PATH_INFO: `/` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:40:09.075426 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 639/1261] 10.222.0.20 () {54 vars in 926 bytes} [Mon Aug 1 16:40:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 16:40:09.080592 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b398c5f5-68aa-43f6-b9ba-dc00922e3a2c None None] REQUEST_METHOD: `POST` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:40:09.081023 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b398c5f5-68aa-43f6-b9ba-dc00922e3a2c None None] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:40:09.081177 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b398c5f5-68aa-43f6-b9ba-dc00922e3a2c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:40:09.099032 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.auth.core [None req-b398c5f5-68aa-43f6-b9ba-dc00922e3a2c None None] MFA Rules not processed for user `1007f8e379b74e169b98fb7c3c0ba5b4`. Rule list: `[]` (Enabled: `True`). {{(pid=87057) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:40:09.100236 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.fernet_utils [None req-b398c5f5-68aa-43f6-b9ba-dc00922e3a2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87057) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:40:09.166548 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 623/1262] 10.222.0.20 () {58 vars in 1023 bytes} [Mon Aug 1 16:40:09 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 01 16:40:09.189174 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65be47ef-a43f-449b-ad66-6aaede8ffcfd None None] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:40:09.200127 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG dbcounter [-] [87058] Writing DB stats keystone:SELECT=100 {{(pid=87058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:40:09.290819 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65be47ef-a43f-449b-ad66-6aaede8ffcfd None None] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:40:09.324738 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b333a0-1102-48db-9732-cef7978572ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:40:09.325380 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b333a0-1102-48db-9732-cef7978572ba None admin] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:40:09.325660 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b333a0-1102-48db-9732-cef7978572ba None admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:40:09.325931 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b333a0-1102-48db-9732-cef7978572ba None admin] PATH_INFO: `/v3/roles/admin` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:40:09.336588 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1b333a0-1102-48db-9732-cef7978572ba None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 16:40:09.341470 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application [None req-e1b333a0-1102-48db-9732-cef7978572ba None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 16:40:09.341470 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 16:40:09.341470 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 16:40:09.341470 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 16:40:09.341470 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 16:40:09.341470 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 16:40:09.341470 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 16:40:09.341470 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 16:40:09.341470 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 16:40:09.341470 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 16:40:09.341470 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 16:40:09.341470 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 16:40:09.341470 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 16:40:09.341470 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 16:40:09.341470 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 16:40:09.341470 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 16:40:09.342624 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 16:40:09.342624 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 16:40:09.342624 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 16:40:09.342624 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 16:40:09.342624 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 16:40:09.342624 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 16:40:09.342624 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 16:40:09.342624 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 16:40:09.342624 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 16:40:09.342624 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 16:40:09.342624 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 16:40:09.342624 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application with Lock( Aug 01 16:40:09.342624 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 16:40:09.342624 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self._enter() Aug 01 16:40:09.342624 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 16:40:09.342624 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 16:40:09.343518 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 16:40:09.343518 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self.creator() Aug 01 16:40:09.343518 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 16:40:09.343518 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application created_value = creator( Aug 01 16:40:09.343518 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 16:40:09.343518 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 16:40:09.343518 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 16:40:09.343518 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 16:40:09.343518 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 16:40:09.343518 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 16:40:09.343518 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 16:40:09.343518 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application Aug 01 16:40:09.343518 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 640/1263] 10.222.0.20 () {56 vars in 1096 bytes} [Mon Aug 1 16:40:09 2022] GET /identity/v3/roles/admin => generated 83 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 16:40:09.348838 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b398c5f5-68aa-43f6-b9ba-dc00922e3a2c None None] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:40:09.367601 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG dbcounter [-] [87057] Writing DB stats keystone:SELECT=100 {{(pid=87057) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:40:09.446823 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b398c5f5-68aa-43f6-b9ba-dc00922e3a2c None None] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:40:09.481088 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-050b8973-a082-46f3-bcb6-a854da2314b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:40:09.481744 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050b8973-a082-46f3-bcb6-a854da2314b6 None admin] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:40:09.482025 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050b8973-a082-46f3-bcb6-a854da2314b6 None admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:40:09.488553 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050b8973-a082-46f3-bcb6-a854da2314b6 None admin] PATH_INFO: `/v3/roles` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:40:09.488553 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-050b8973-a082-46f3-bcb6-a854da2314b6 None admin] RBAC: Adding query filter params (name=admin) {{(pid=87057) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:40:09.488553 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-050b8973-a082-46f3-bcb6-a854da2314b6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:40:09.488553 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-050b8973-a082-46f3-bcb6-a854da2314b6 None admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:40:09.492609 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 624/1264] 10.222.0.20 () {56 vars in 1099 bytes} [Mon Aug 1 16:40:09 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 16:40:09.498392 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b333a0-1102-48db-9732-cef7978572ba None admin] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:40:09.565895 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG dbcounter [-] [87058] Writing DB stats keystone:SELECT=100 {{(pid=87058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:40:09.599263 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b333a0-1102-48db-9732-cef7978572ba None admin] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:40:09.634236 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cba08132-e2c0-40ba-8ec7-a0f28af9b6df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:40:09.634876 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba08132-e2c0-40ba-8ec7-a0f28af9b6df None admin] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:40:09.635160 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba08132-e2c0-40ba-8ec7-a0f28af9b6df None admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:40:09.635437 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba08132-e2c0-40ba-8ec7-a0f28af9b6df None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:40:09.645563 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cba08132-e2c0-40ba-8ec7-a0f28af9b6df None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:40:09.647053 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cba08132-e2c0-40ba-8ec7-a0f28af9b6df None admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:40:09.662237 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application [None req-cba08132-e2c0-40ba-8ec7-a0f28af9b6df None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 16:40:09.662237 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 16:40:09.662237 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 16:40:09.662237 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 16:40:09.662237 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 16:40:09.662237 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 16:40:09.662237 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 16:40:09.662237 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 16:40:09.662237 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 16:40:09.662237 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 16:40:09.662237 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 16:40:09.662237 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 16:40:09.662237 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 16:40:09.662237 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 16:40:09.662237 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 16:40:09.663511 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 16:40:09.663511 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 16:40:09.663511 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 16:40:09.663511 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 16:40:09.663511 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 16:40:09.663511 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 16:40:09.663511 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 16:40:09.663511 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 16:40:09.663511 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application with Lock( Aug 01 16:40:09.663511 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 16:40:09.663511 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self._enter() Aug 01 16:40:09.663511 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 16:40:09.663511 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 16:40:09.663511 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 16:40:09.663511 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self.creator() Aug 01 16:40:09.663511 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 16:40:09.663511 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application created_value = creator( Aug 01 16:40:09.664511 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 16:40:09.664511 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 16:40:09.664511 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 16:40:09.664511 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application Aug 01 16:40:09.664511 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 641/1265] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Aug 1 16:40:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 16:40:09.669653 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-050b8973-a082-46f3-bcb6-a854da2314b6 None admin] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:40:09.750724 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG dbcounter [-] [87057] Writing DB stats keystone:SELECT=100 {{(pid=87057) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:40:09.769780 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-050b8973-a082-46f3-bcb6-a854da2314b6 None admin] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:40:09.802839 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a25dd87-5e09-4632-ab50-7c1bbfc67b34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:40:09.803454 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a25dd87-5e09-4632-ab50-7c1bbfc67b34 None admin] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:40:09.803740 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a25dd87-5e09-4632-ab50-7c1bbfc67b34 None admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:40:09.804016 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a25dd87-5e09-4632-ab50-7c1bbfc67b34 None admin] PATH_INFO: `/v3/domains` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:40:09.804709 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a25dd87-5e09-4632-ab50-7c1bbfc67b34 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87057) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:40:09.805015 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a25dd87-5e09-4632-ab50-7c1bbfc67b34 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:40:09.806400 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a25dd87-5e09-4632-ab50-7c1bbfc67b34 None admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:40:09.817241 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 625/1266] 10.222.0.20 () {56 vars in 1109 bytes} [Mon Aug 1 16:40:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:40:09.823632 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cba08132-e2c0-40ba-8ec7-a0f28af9b6df None admin] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:40:09.925662 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cba08132-e2c0-40ba-8ec7-a0f28af9b6df None admin] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:40:09.946208 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG dbcounter [-] [87058] Writing DB stats keystone:SELECT=100 {{(pid=87058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:40:09.959707 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d05db3-92da-4b38-849e-9ed513cdc716 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:40:09.960403 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d05db3-92da-4b38-849e-9ed513cdc716 None admin] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:40:09.960680 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d05db3-92da-4b38-849e-9ed513cdc716 None admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:40:09.960924 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d05db3-92da-4b38-849e-9ed513cdc716 None admin] PATH_INFO: `/v3/users/placement` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:40:09.968323 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19d05db3-92da-4b38-849e-9ed513cdc716 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:40:09.969776 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19d05db3-92da-4b38-849e-9ed513cdc716 None admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:40:09.980972 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application [None req-19d05db3-92da-4b38-849e-9ed513cdc716 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 01 16:40:09.980972 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 16:40:09.980972 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 16:40:09.980972 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 16:40:09.980972 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 16:40:09.980972 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 16:40:09.980972 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 16:40:09.980972 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 16:40:09.980972 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 16:40:09.980972 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 16:40:09.980972 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 16:40:09.980972 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 16:40:09.980972 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 01 16:40:09.980972 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 01 16:40:09.980972 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 01 16:40:09.980972 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 01 16:40:09.982083 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 16:40:09.982083 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 16:40:09.982083 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 01 16:40:09.982083 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 16:40:09.982083 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 01 16:40:09.982083 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 16:40:09.982083 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 16:40:09.982083 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 16:40:09.982083 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 16:40:09.982083 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 16:40:09.982083 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 16:40:09.982083 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application with Lock( Aug 01 16:40:09.982083 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 16:40:09.982083 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self._enter() Aug 01 16:40:09.982083 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 16:40:09.982083 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 16:40:09.982083 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 16:40:09.982083 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self.creator() Aug 01 16:40:09.983533 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 16:40:09.983533 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application created_value = creator( Aug 01 16:40:09.983533 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 01 16:40:09.983533 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 01 16:40:09.983533 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 01 16:40:09.983533 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 01 16:40:09.983533 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 01 16:40:09.983533 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 01 16:40:09.983533 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 01 16:40:09.983533 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application Aug 01 16:40:09.983533 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 642/1267] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Aug 1 16:40:09 2022] GET /identity/v3/users/placement => generated 87 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 16:40:09.988070 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a25dd87-5e09-4632-ab50-7c1bbfc67b34 None admin] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:40:10.086977 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a25dd87-5e09-4632-ab50-7c1bbfc67b34 None admin] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:40:10.119875 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed66e89-fa1a-43e8-883a-3f795fdb431d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:40:10.120486 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed66e89-fa1a-43e8-883a-3f795fdb431d None admin] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:40:10.120764 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed66e89-fa1a-43e8-883a-3f795fdb431d None admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:40:10.121039 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed66e89-fa1a-43e8-883a-3f795fdb431d None admin] PATH_INFO: `/v3/users` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:40:10.121877 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ed66e89-fa1a-43e8-883a-3f795fdb431d None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=87057) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:40:10.122227 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ed66e89-fa1a-43e8-883a-3f795fdb431d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:40:10.123689 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ed66e89-fa1a-43e8-883a-3f795fdb431d None admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:40:10.133444 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG dbcounter [-] [87057] Writing DB stats keystone:SELECT=100 {{(pid=87057) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:40:10.141635 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 626/1268] 10.222.0.20 () {56 vars in 1143 bytes} [Mon Aug 1 16:40:09 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:40:10.148072 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d05db3-92da-4b38-849e-9ed513cdc716 None admin] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:40:10.248837 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d05db3-92da-4b38-849e-9ed513cdc716 None admin] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:40:10.281780 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ed22f5-62c2-4bb0-b7dd-92dc519a69a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:40:10.282432 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ed22f5-62c2-4bb0-b7dd-92dc519a69a9 None admin] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:40:10.282710 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ed22f5-62c2-4bb0-b7dd-92dc519a69a9 None admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:40:10.282978 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ed22f5-62c2-4bb0-b7dd-92dc519a69a9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:40:10.293604 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3ed22f5-62c2-4bb0-b7dd-92dc519a69a9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:40:10.295058 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3ed22f5-62c2-4bb0-b7dd-92dc519a69a9 None admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:40:10.309310 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application [None req-f3ed22f5-62c2-4bb0-b7dd-92dc519a69a9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 16:40:10.309310 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 16:40:10.309310 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 16:40:10.309310 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 16:40:10.309310 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 16:40:10.309310 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 16:40:10.309310 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 16:40:10.309310 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 16:40:10.309310 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 16:40:10.309310 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 16:40:10.309310 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 16:40:10.309310 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 16:40:10.309310 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 16:40:10.309310 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 16:40:10.309310 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 16:40:10.310459 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 16:40:10.310459 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 16:40:10.310459 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 16:40:10.310459 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 16:40:10.310459 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 16:40:10.310459 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 16:40:10.310459 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 16:40:10.310459 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 16:40:10.310459 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application with Lock( Aug 01 16:40:10.310459 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 16:40:10.310459 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self._enter() Aug 01 16:40:10.310459 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 16:40:10.310459 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 16:40:10.310459 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 16:40:10.310459 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self.creator() Aug 01 16:40:10.310459 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 16:40:10.310459 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application created_value = creator( Aug 01 16:40:10.317128 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 16:40:10.317128 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 16:40:10.317128 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 16:40:10.317128 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application Aug 01 16:40:10.317128 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 643/1269] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Aug 1 16:40:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 16:40:10.317494 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed66e89-fa1a-43e8-883a-3f795fdb431d None admin] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:40:10.416427 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed66e89-fa1a-43e8-883a-3f795fdb431d None admin] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:40:10.451408 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd0c5d95-d1c2-48b6-92d9-8136c3631f06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:40:10.452018 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0c5d95-d1c2-48b6-92d9-8136c3631f06 None admin] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:40:10.452295 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0c5d95-d1c2-48b6-92d9-8136c3631f06 None admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:40:10.452614 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0c5d95-d1c2-48b6-92d9-8136c3631f06 None admin] PATH_INFO: `/v3/domains` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:40:10.453301 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd0c5d95-d1c2-48b6-92d9-8136c3631f06 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87057) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:40:10.453604 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd0c5d95-d1c2-48b6-92d9-8136c3631f06 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:40:10.454996 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd0c5d95-d1c2-48b6-92d9-8136c3631f06 None admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:40:10.466186 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 627/1270] 10.222.0.20 () {56 vars in 1109 bytes} [Mon Aug 1 16:40:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:40:10.471922 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ed22f5-62c2-4bb0-b7dd-92dc519a69a9 None admin] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:40:10.504216 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG dbcounter [-] [87058] Writing DB stats keystone:SELECT=100 {{(pid=87058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:40:10.571343 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ed22f5-62c2-4bb0-b7dd-92dc519a69a9 None admin] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:40:10.604875 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6d24b6d-486e-4ad6-9931-7990906e33fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:40:10.605512 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d24b6d-486e-4ad6-9931-7990906e33fc None admin] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:40:10.605792 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d24b6d-486e-4ad6-9931-7990906e33fc None admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:40:10.606111 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d24b6d-486e-4ad6-9931-7990906e33fc None admin] PATH_INFO: `/v3/projects/service` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:40:10.613052 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6d24b6d-486e-4ad6-9931-7990906e33fc None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:40:10.614472 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6d24b6d-486e-4ad6-9931-7990906e33fc None admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:40:10.626328 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application [None req-a6d24b6d-486e-4ad6-9931-7990906e33fc None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 16:40:10.626328 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 16:40:10.626328 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 16:40:10.626328 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 16:40:10.626328 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 16:40:10.626328 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 16:40:10.626328 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 16:40:10.626328 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 16:40:10.626328 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 16:40:10.626328 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 16:40:10.626328 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 16:40:10.626328 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 16:40:10.626328 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 01 16:40:10.626328 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 01 16:40:10.626328 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 01 16:40:10.627442 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 01 16:40:10.627442 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 16:40:10.627442 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 16:40:10.627442 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 16:40:10.627442 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 16:40:10.627442 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 16:40:10.627442 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 16:40:10.627442 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 16:40:10.627442 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application with Lock( Aug 01 16:40:10.627442 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 16:40:10.627442 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self._enter() Aug 01 16:40:10.627442 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 16:40:10.627442 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 16:40:10.627442 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 16:40:10.627442 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self.creator() Aug 01 16:40:10.627442 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 16:40:10.627442 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application created_value = creator( Aug 01 16:40:10.628583 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 01 16:40:10.628583 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 01 16:40:10.628583 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 01 16:40:10.628583 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 01 16:40:10.628583 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 01 16:40:10.628583 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 01 16:40:10.628583 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 16:40:10.628583 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application Aug 01 16:40:10.628583 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 644/1271] 10.222.0.20 () {56 vars in 1111 bytes} [Mon Aug 1 16:40:10 2022] GET /identity/v3/projects/service => generated 88 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 16:40:10.633509 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd0c5d95-d1c2-48b6-92d9-8136c3631f06 None admin] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:40:10.691488 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG dbcounter [-] [87057] Writing DB stats keystone:SELECT=100 {{(pid=87057) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:40:10.733113 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd0c5d95-d1c2-48b6-92d9-8136c3631f06 None admin] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:40:10.766269 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26c2ec3f-7c50-4ceb-b829-856886cfe117 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:40:10.766884 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c2ec3f-7c50-4ceb-b829-856886cfe117 None admin] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:40:10.767165 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c2ec3f-7c50-4ceb-b829-856886cfe117 None admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:40:10.767438 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c2ec3f-7c50-4ceb-b829-856886cfe117 None admin] PATH_INFO: `/v3/projects` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:40:10.768183 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26c2ec3f-7c50-4ceb-b829-856886cfe117 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87057) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:40:10.768488 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26c2ec3f-7c50-4ceb-b829-856886cfe117 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:40:10.769904 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26c2ec3f-7c50-4ceb-b829-856886cfe117 None admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:40:10.782519 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 628/1272] 10.222.0.20 () {56 vars in 1148 bytes} [Mon Aug 1 16:40:10 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:40:10.789276 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6d24b6d-486e-4ad6-9931-7990906e33fc None admin] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:40:10.879892 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG dbcounter [-] [87058] Writing DB stats keystone:SELECT=100 {{(pid=87058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:40:10.889341 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6d24b6d-486e-4ad6-9931-7990906e33fc None admin] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:40:10.925054 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d7ec5bb-b82b-445f-a810-4d0d12d5a3d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:40:10.925918 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7ec5bb-b82b-445f-a810-4d0d12d5a3d7 None admin] REQUEST_METHOD: `PUT` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:40:10.926422 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7ec5bb-b82b-445f-a810-4d0d12d5a3d7 None admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:40:10.926706 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7ec5bb-b82b-445f-a810-4d0d12d5a3d7 None admin] PATH_INFO: `/v3/projects/7b992d6a65df4bf9b27c13192d57c5f0/users/8362293261614623bcdabc4120a97a9d/roles/440bb943dd5b46fea038b903b6eaaf81` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:40:10.929198 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d7ec5bb-b82b-445f-a810-4d0d12d5a3d7 None admin] RBAC: Authorizing `identity:create_grant(project_id=7b992d6a65df4bf9b27c13192d57c5f0, user_id=8362293261614623bcdabc4120a97a9d, role_id=440bb943dd5b46fea038b903b6eaaf81)` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:40:10.930680 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d7ec5bb-b82b-445f-a810-4d0d12d5a3d7 None admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:40:11.018270 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 645/1273] 10.222.0.20 () {58 vars in 1439 bytes} [Mon Aug 1 16:40:10 2022] PUT /identity/v3/projects/7b992d6a65df4bf9b27c13192d57c5f0/users/8362293261614623bcdabc4120a97a9d/roles/440bb943dd5b46fea038b903b6eaaf81 => generated 0 bytes in 230 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 16:40:13.034968 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75e4ba1-3d50-4f8d-a345-8d6c651e3053 None None] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:40:13.035543 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75e4ba1-3d50-4f8d-a345-8d6c651e3053 None None] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:40:13.035543 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75e4ba1-3d50-4f8d-a345-8d6c651e3053 None None] PATH_INFO: `/` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:40:13.036883 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 629/1274] 10.222.0.20 () {54 vars in 926 bytes} [Mon Aug 1 16:40:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 16:40:13.043020 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267a0db3-2853-493f-831e-6556c62e02a1 None None] REQUEST_METHOD: `POST` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:40:13.043452 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267a0db3-2853-493f-831e-6556c62e02a1 None None] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:40:13.043561 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267a0db3-2853-493f-831e-6556c62e02a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:40:13.062640 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.auth.core [None req-267a0db3-2853-493f-831e-6556c62e02a1 None None] MFA Rules not processed for user `1007f8e379b74e169b98fb7c3c0ba5b4`. Rule list: `[]` (Enabled: `True`). {{(pid=87058) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:40:13.063828 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.fernet_utils [None req-267a0db3-2853-493f-831e-6556c62e02a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:40:13.130603 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 646/1275] 10.222.0.20 () {58 vars in 1023 bytes} [Mon Aug 1 16:40:13 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 01 16:40:13.154883 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e75e4ba1-3d50-4f8d-a345-8d6c651e3053 None None] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:40:13.256642 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e75e4ba1-3d50-4f8d-a345-8d6c651e3053 None None] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:40:13.274327 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG dbcounter [-] [87057] Writing DB stats keystone:SELECT=100 {{(pid=87057) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:40:13.291141 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc18077-2434-44dd-ae7c-81a3a9ba9892 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:40:13.291776 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc18077-2434-44dd-ae7c-81a3a9ba9892 None admin] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:40:13.292059 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc18077-2434-44dd-ae7c-81a3a9ba9892 None admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:40:13.292334 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc18077-2434-44dd-ae7c-81a3a9ba9892 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:40:13.302391 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fc18077-2434-44dd-ae7c-81a3a9ba9892 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 16:40:13.307433 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application [None req-9fc18077-2434-44dd-ae7c-81a3a9ba9892 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 16:40:13.307433 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 16:40:13.307433 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 16:40:13.307433 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 16:40:13.307433 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 16:40:13.307433 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 16:40:13.307433 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 16:40:13.307433 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 16:40:13.307433 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 16:40:13.307433 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 16:40:13.307433 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 16:40:13.307433 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 16:40:13.307433 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 16:40:13.307433 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 16:40:13.307433 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 16:40:13.307433 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 16:40:13.308672 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 16:40:13.308672 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 16:40:13.308672 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 16:40:13.308672 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 16:40:13.308672 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 16:40:13.308672 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 16:40:13.308672 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 16:40:13.308672 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 16:40:13.308672 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 16:40:13.308672 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 16:40:13.308672 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 16:40:13.308672 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application with Lock( Aug 01 16:40:13.308672 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 16:40:13.308672 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self._enter() Aug 01 16:40:13.308672 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 16:40:13.308672 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 16:40:13.309946 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 16:40:13.309946 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self.creator() Aug 01 16:40:13.309946 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 16:40:13.309946 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application created_value = creator( Aug 01 16:40:13.309946 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 16:40:13.309946 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 16:40:13.309946 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 16:40:13.309946 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 16:40:13.309946 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 16:40:13.309946 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 16:40:13.309946 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 16:40:13.309946 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application Aug 01 16:40:13.309946 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 630/1276] 10.222.0.20 () {56 vars in 1096 bytes} [Mon Aug 1 16:40:13 2022] GET /identity/v3/roles/admin => generated 83 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 16:40:13.314769 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-267a0db3-2853-493f-831e-6556c62e02a1 None None] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:40:13.394821 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG dbcounter [-] [87058] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:40:13.412560 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-267a0db3-2853-493f-831e-6556c62e02a1 None None] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:40:13.446992 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdbf8fbf-25f0-406c-b719-0cead3418160 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:40:13.447603 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdbf8fbf-25f0-406c-b719-0cead3418160 None admin] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:40:13.447882 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdbf8fbf-25f0-406c-b719-0cead3418160 None admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:40:13.448154 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdbf8fbf-25f0-406c-b719-0cead3418160 None admin] PATH_INFO: `/v3/roles` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:40:13.448867 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdbf8fbf-25f0-406c-b719-0cead3418160 None admin] RBAC: Adding query filter params (name=admin) {{(pid=87058) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:40:13.449168 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdbf8fbf-25f0-406c-b719-0cead3418160 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:40:13.450642 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdbf8fbf-25f0-406c-b719-0cead3418160 None admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:40:13.458497 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 647/1277] 10.222.0.20 () {56 vars in 1099 bytes} [Mon Aug 1 16:40:13 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:40:13.464923 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc18077-2434-44dd-ae7c-81a3a9ba9892 None admin] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:40:13.563454 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc18077-2434-44dd-ae7c-81a3a9ba9892 None admin] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:40:13.597058 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-913b8640-4d70-455a-881a-4245dd8a8f5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:40:13.597680 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913b8640-4d70-455a-881a-4245dd8a8f5e None admin] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:40:13.597957 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913b8640-4d70-455a-881a-4245dd8a8f5e None admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:40:13.598254 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913b8640-4d70-455a-881a-4245dd8a8f5e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:40:13.608536 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-913b8640-4d70-455a-881a-4245dd8a8f5e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:40:13.609967 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-913b8640-4d70-455a-881a-4245dd8a8f5e None admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:40:13.624634 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application [None req-913b8640-4d70-455a-881a-4245dd8a8f5e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 16:40:13.624634 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 16:40:13.624634 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 16:40:13.624634 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 16:40:13.624634 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 16:40:13.624634 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 16:40:13.624634 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 16:40:13.624634 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 16:40:13.624634 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 16:40:13.624634 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 16:40:13.624634 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 16:40:13.624634 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 16:40:13.624634 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 16:40:13.624634 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 16:40:13.624634 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 16:40:13.625696 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 16:40:13.625696 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 16:40:13.625696 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 16:40:13.625696 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 16:40:13.625696 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 16:40:13.625696 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 16:40:13.625696 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 16:40:13.625696 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 16:40:13.625696 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application with Lock( Aug 01 16:40:13.625696 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 16:40:13.625696 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self._enter() Aug 01 16:40:13.625696 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 16:40:13.625696 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 16:40:13.625696 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 16:40:13.625696 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self.creator() Aug 01 16:40:13.625696 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 16:40:13.625696 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application created_value = creator( Aug 01 16:40:13.626855 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 16:40:13.626855 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 16:40:13.626855 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 16:40:13.626855 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application Aug 01 16:40:13.626855 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 631/1278] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Aug 1 16:40:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 16:40:13.632306 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdbf8fbf-25f0-406c-b719-0cead3418160 None admin] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:40:13.732062 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdbf8fbf-25f0-406c-b719-0cead3418160 None admin] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:40:13.765899 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4effa4cb-b38d-40f7-b39f-a801e8247456 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:40:13.766556 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4effa4cb-b38d-40f7-b39f-a801e8247456 None admin] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:40:13.766834 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4effa4cb-b38d-40f7-b39f-a801e8247456 None admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:40:13.767104 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4effa4cb-b38d-40f7-b39f-a801e8247456 None admin] PATH_INFO: `/v3/domains` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:40:13.767782 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4effa4cb-b38d-40f7-b39f-a801e8247456 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87058) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:40:13.768079 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4effa4cb-b38d-40f7-b39f-a801e8247456 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:40:13.769420 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4effa4cb-b38d-40f7-b39f-a801e8247456 None admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:40:13.780962 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 648/1279] 10.222.0.20 () {56 vars in 1109 bytes} [Mon Aug 1 16:40:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:40:13.786835 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-913b8640-4d70-455a-881a-4245dd8a8f5e None admin] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:40:13.818377 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG dbcounter [-] [87057] Writing DB stats keystone:SELECT=100 {{(pid=87057) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:40:13.886876 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-913b8640-4d70-455a-881a-4245dd8a8f5e None admin] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:40:13.924348 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fabecb3-26b2-4c72-9ce8-312cac1803a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:40:13.924985 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fabecb3-26b2-4c72-9ce8-312cac1803a9 None admin] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:40:13.925259 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fabecb3-26b2-4c72-9ce8-312cac1803a9 None admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:40:13.925527 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fabecb3-26b2-4c72-9ce8-312cac1803a9 None admin] PATH_INFO: `/v3/users/placement` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:40:13.933246 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fabecb3-26b2-4c72-9ce8-312cac1803a9 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:40:13.934713 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fabecb3-26b2-4c72-9ce8-312cac1803a9 None admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:40:13.946735 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application [None req-2fabecb3-26b2-4c72-9ce8-312cac1803a9 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 01 16:40:13.946735 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 16:40:13.946735 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 16:40:13.946735 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 16:40:13.946735 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 16:40:13.946735 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 16:40:13.946735 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 16:40:13.946735 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 16:40:13.946735 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 16:40:13.946735 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 16:40:13.946735 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 16:40:13.946735 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 16:40:13.946735 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 01 16:40:13.946735 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 01 16:40:13.946735 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 01 16:40:13.946735 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 01 16:40:13.947921 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 16:40:13.947921 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 16:40:13.947921 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 01 16:40:13.947921 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 16:40:13.947921 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 01 16:40:13.947921 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 16:40:13.947921 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 16:40:13.947921 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 16:40:13.947921 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 16:40:13.947921 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 16:40:13.947921 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 16:40:13.947921 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application with Lock( Aug 01 16:40:13.947921 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 16:40:13.947921 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self._enter() Aug 01 16:40:13.947921 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 16:40:13.947921 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 16:40:13.947921 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 16:40:13.947921 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self.creator() Aug 01 16:40:13.949102 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 16:40:13.949102 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application created_value = creator( Aug 01 16:40:13.949102 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 01 16:40:13.949102 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 01 16:40:13.949102 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 01 16:40:13.949102 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 01 16:40:13.949102 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 01 16:40:13.949102 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 01 16:40:13.949102 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 01 16:40:13.949102 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application Aug 01 16:40:13.949102 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 632/1280] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Aug 1 16:40:13 2022] GET /identity/v3/users/placement => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 16:40:13.953629 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4effa4cb-b38d-40f7-b39f-a801e8247456 None admin] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:40:13.957374 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG dbcounter [-] [87058] Writing DB stats keystone:SELECT=100 {{(pid=87058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:40:14.052151 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4effa4cb-b38d-40f7-b39f-a801e8247456 None admin] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:40:14.085109 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dee98a5-412f-49d1-a444-2df735aa3a18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:40:14.085711 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dee98a5-412f-49d1-a444-2df735aa3a18 None admin] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:40:14.085986 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dee98a5-412f-49d1-a444-2df735aa3a18 None admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:40:14.086284 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dee98a5-412f-49d1-a444-2df735aa3a18 None admin] PATH_INFO: `/v3/users` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:40:14.087115 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dee98a5-412f-49d1-a444-2df735aa3a18 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=87058) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:40:14.087416 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dee98a5-412f-49d1-a444-2df735aa3a18 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:40:14.088817 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dee98a5-412f-49d1-a444-2df735aa3a18 None admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:40:14.105916 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 649/1281] 10.222.0.20 () {56 vars in 1143 bytes} [Mon Aug 1 16:40:13 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:40:14.117123 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fabecb3-26b2-4c72-9ce8-312cac1803a9 None admin] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:40:14.207108 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG dbcounter [-] [87057] Writing DB stats keystone:SELECT=100 {{(pid=87057) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:40:14.212076 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fabecb3-26b2-4c72-9ce8-312cac1803a9 None admin] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:40:14.245796 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89404d86-4d50-4607-82e0-d2a11145fe3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:40:14.246442 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89404d86-4d50-4607-82e0-d2a11145fe3d None admin] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:40:14.246721 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89404d86-4d50-4607-82e0-d2a11145fe3d None admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:40:14.246994 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89404d86-4d50-4607-82e0-d2a11145fe3d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:40:14.312430 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89404d86-4d50-4607-82e0-d2a11145fe3d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:40:14.313869 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89404d86-4d50-4607-82e0-d2a11145fe3d None admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:40:14.403213 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG dbcounter [-] [87057] Writing DB stats keystone:SELECT=100 {{(pid=87057) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:40:14.413422 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 633/1282] 10.222.0.20 () {58 vars in 1282 bytes} [Mon Aug 1 16:40:14 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 302 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 01 16:40:14.419217 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dee98a5-412f-49d1-a444-2df735aa3a18 None admin] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:40:14.480159 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG dbcounter [-] [87058] Writing DB stats keystone:SELECT=100 {{(pid=87058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:40:14.518888 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dee98a5-412f-49d1-a444-2df735aa3a18 None admin] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:40:14.551768 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ed0d285-811e-408d-a21d-ee4621be3469 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:40:14.552397 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed0d285-811e-408d-a21d-ee4621be3469 None admin] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:40:14.552675 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed0d285-811e-408d-a21d-ee4621be3469 None admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:40:14.552949 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed0d285-811e-408d-a21d-ee4621be3469 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:40:14.564919 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ed0d285-811e-408d-a21d-ee4621be3469 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:40:14.566326 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ed0d285-811e-408d-a21d-ee4621be3469 None admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:40:14.580700 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application [None req-2ed0d285-811e-408d-a21d-ee4621be3469 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 16:40:14.580700 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 16:40:14.580700 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 16:40:14.580700 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 16:40:14.580700 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 16:40:14.580700 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 16:40:14.580700 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 16:40:14.580700 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 16:40:14.580700 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 16:40:14.580700 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 16:40:14.580700 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 16:40:14.580700 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 16:40:14.580700 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 16:40:14.580700 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 16:40:14.580700 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 16:40:14.581837 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 16:40:14.581837 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 16:40:14.581837 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 16:40:14.581837 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 16:40:14.581837 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 16:40:14.581837 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 16:40:14.581837 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 16:40:14.581837 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 16:40:14.581837 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application with Lock( Aug 01 16:40:14.581837 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 16:40:14.581837 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self._enter() Aug 01 16:40:14.581837 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 16:40:14.581837 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 16:40:14.581837 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 16:40:14.581837 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self.creator() Aug 01 16:40:14.581837 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 16:40:14.581837 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application created_value = creator( Aug 01 16:40:14.589015 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 16:40:14.589015 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 16:40:14.589015 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 16:40:14.589015 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application Aug 01 16:40:14.589015 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 650/1283] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Aug 1 16:40:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 16:40:14.589378 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89404d86-4d50-4607-82e0-d2a11145fe3d None admin] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:40:14.685283 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89404d86-4d50-4607-82e0-d2a11145fe3d None admin] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:40:14.718582 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a4a469b-5639-436a-86b9-71df44ac3adc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:40:14.719218 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4a469b-5639-436a-86b9-71df44ac3adc None admin] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:40:14.719493 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4a469b-5639-436a-86b9-71df44ac3adc None admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:40:14.719761 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4a469b-5639-436a-86b9-71df44ac3adc None admin] PATH_INFO: `/v3/domains` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:40:14.720441 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a4a469b-5639-436a-86b9-71df44ac3adc None admin] RBAC: Adding query filter params (name=Default) {{(pid=87057) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:40:14.720736 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a4a469b-5639-436a-86b9-71df44ac3adc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:40:14.722271 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a4a469b-5639-436a-86b9-71df44ac3adc None admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:40:14.733254 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 634/1284] 10.222.0.20 () {56 vars in 1109 bytes} [Mon Aug 1 16:40:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:40:14.739000 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ed0d285-811e-408d-a21d-ee4621be3469 None admin] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:40:14.838921 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ed0d285-811e-408d-a21d-ee4621be3469 None admin] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:40:14.852662 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG dbcounter [-] [87058] Writing DB stats keystone:SELECT=100 {{(pid=87058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:40:14.871678 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-359a27f3-9593-43dd-a30b-1d6a33ddbfcd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:40:14.872330 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359a27f3-9593-43dd-a30b-1d6a33ddbfcd None admin] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:40:14.872608 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359a27f3-9593-43dd-a30b-1d6a33ddbfcd None admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:40:14.872877 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359a27f3-9593-43dd-a30b-1d6a33ddbfcd None admin] PATH_INFO: `/v3/projects/service` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:40:14.879292 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-359a27f3-9593-43dd-a30b-1d6a33ddbfcd None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:40:14.880718 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-359a27f3-9593-43dd-a30b-1d6a33ddbfcd None admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:40:14.892434 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application [None req-359a27f3-9593-43dd-a30b-1d6a33ddbfcd None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 16:40:14.892434 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 16:40:14.892434 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 16:40:14.892434 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 16:40:14.892434 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 16:40:14.892434 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 16:40:14.892434 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 16:40:14.892434 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 16:40:14.892434 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 16:40:14.892434 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 16:40:14.892434 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 16:40:14.892434 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 16:40:14.892434 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 01 16:40:14.892434 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 01 16:40:14.892434 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 01 16:40:14.893515 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 01 16:40:14.893515 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 16:40:14.893515 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 16:40:14.893515 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 16:40:14.893515 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 16:40:14.893515 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 16:40:14.893515 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 16:40:14.893515 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 16:40:14.893515 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application with Lock( Aug 01 16:40:14.893515 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 16:40:14.893515 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self._enter() Aug 01 16:40:14.893515 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 16:40:14.893515 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 16:40:14.893515 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 16:40:14.893515 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self.creator() Aug 01 16:40:14.893515 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 16:40:14.893515 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application created_value = creator( Aug 01 16:40:14.894847 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 01 16:40:14.894847 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 01 16:40:14.894847 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 01 16:40:14.894847 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 01 16:40:14.894847 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 01 16:40:14.894847 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 01 16:40:14.894847 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 16:40:14.894847 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: ERROR keystone.server.flask.application Aug 01 16:40:14.894847 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 651/1285] 10.222.0.20 () {56 vars in 1111 bytes} [Mon Aug 1 16:40:14 2022] GET /identity/v3/projects/service => generated 88 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 16:40:14.899460 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a4a469b-5639-436a-86b9-71df44ac3adc None admin] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:40:14.951299 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG dbcounter [-] [87057] Writing DB stats keystone:SELECT=100 {{(pid=87057) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:40:15.007052 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a4a469b-5639-436a-86b9-71df44ac3adc None admin] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:40:15.040633 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9970b355-b7f1-4db1-b6dd-e0db24f1caf0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:40:15.041243 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9970b355-b7f1-4db1-b6dd-e0db24f1caf0 None admin] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:40:15.041523 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9970b355-b7f1-4db1-b6dd-e0db24f1caf0 None admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:40:15.041792 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9970b355-b7f1-4db1-b6dd-e0db24f1caf0 None admin] PATH_INFO: `/v3/projects` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:40:15.042571 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9970b355-b7f1-4db1-b6dd-e0db24f1caf0 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87057) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:40:15.042877 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9970b355-b7f1-4db1-b6dd-e0db24f1caf0 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:40:15.044294 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9970b355-b7f1-4db1-b6dd-e0db24f1caf0 None admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:40:15.059523 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 635/1286] 10.222.0.20 () {56 vars in 1148 bytes} [Mon Aug 1 16:40:14 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:40:15.065795 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-359a27f3-9593-43dd-a30b-1d6a33ddbfcd None admin] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:40:15.183307 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-359a27f3-9593-43dd-a30b-1d6a33ddbfcd None admin] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:40:15.229753 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9977c7e1-4cde-4154-88dc-3bf439cefab1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:40:15.233374 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9977c7e1-4cde-4154-88dc-3bf439cefab1 None admin] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:40:15.233374 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9977c7e1-4cde-4154-88dc-3bf439cefab1 None admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:40:15.233374 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9977c7e1-4cde-4154-88dc-3bf439cefab1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:40:15.233374 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9977c7e1-4cde-4154-88dc-3bf439cefab1 None admin] RBAC: Adding query filter params (role.id=440bb943dd5b46fea038b903b6eaaf81, scope.project.id=7b992d6a65df4bf9b27c13192d57c5f0, user.id=8362293261614623bcdabc4120a97a9d) {{(pid=87058) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:40:15.233374 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9977c7e1-4cde-4154-88dc-3bf439cefab1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:40:15.235081 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9977c7e1-4cde-4154-88dc-3bf439cefab1 None admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:40:15.244800 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 652/1287] 10.222.0.20 () {56 vars in 1374 bytes} [Mon Aug 1 16:40:15 2022] GET /identity/v3/role_assignments?user.id=8362293261614623bcdabc4120a97a9d&scope.project.id=7b992d6a65df4bf9b27c13192d57c5f0&role.id=440bb943dd5b46fea038b903b6eaaf81 => generated 608 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 16:40:17.300142 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157b5a8d-f7d7-4c3d-8c26-fa1162921608 None None] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:40:17.300692 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157b5a8d-f7d7-4c3d-8c26-fa1162921608 None None] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:40:17.300692 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157b5a8d-f7d7-4c3d-8c26-fa1162921608 None None] PATH_INFO: `/` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:40:17.302009 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 636/1288] 10.222.0.20 () {54 vars in 926 bytes} [Mon Aug 1 16:40:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 16:40:17.307327 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d4bc3e-cb4c-49bd-ae99-fcfbe7ac9390 None None] REQUEST_METHOD: `POST` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:40:17.307750 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d4bc3e-cb4c-49bd-ae99-fcfbe7ac9390 None None] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:40:17.308064 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d4bc3e-cb4c-49bd-ae99-fcfbe7ac9390 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:40:17.326303 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.auth.core [None req-f9d4bc3e-cb4c-49bd-ae99-fcfbe7ac9390 None None] MFA Rules not processed for user `1007f8e379b74e169b98fb7c3c0ba5b4`. Rule list: `[]` (Enabled: `True`). {{(pid=87058) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:40:17.327517 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.fernet_utils [None req-f9d4bc3e-cb4c-49bd-ae99-fcfbe7ac9390 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:40:17.358116 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG dbcounter [-] [87058] Writing DB stats keystone:SELECT=100 {{(pid=87058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:40:17.393822 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 653/1289] 10.222.0.20 () {58 vars in 1023 bytes} [Mon Aug 1 16:40:17 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 01 16:40:17.416395 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-157b5a8d-f7d7-4c3d-8c26-fa1162921608 None None] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:40:17.521378 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-157b5a8d-f7d7-4c3d-8c26-fa1162921608 None None] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:40:17.532738 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG dbcounter [-] [87057] Writing DB stats keystone:SELECT=100 {{(pid=87057) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:40:17.555551 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5760ceea-6b14-4320-8a36-23aa9290584a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:40:17.556123 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5760ceea-6b14-4320-8a36-23aa9290584a None admin] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:40:17.556403 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5760ceea-6b14-4320-8a36-23aa9290584a None admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:40:17.556676 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5760ceea-6b14-4320-8a36-23aa9290584a None admin] PATH_INFO: `/v3/services/placement` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:40:17.557299 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5760ceea-6b14-4320-8a36-23aa9290584a None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:40:17.558727 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5760ceea-6b14-4320-8a36-23aa9290584a None admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:40:17.567463 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application [None req-5760ceea-6b14-4320-8a36-23aa9290584a None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Aug 01 16:40:17.567463 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 16:40:17.567463 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 01 16:40:17.567463 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 01 16:40:17.567463 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 01 16:40:17.567463 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 01 16:40:17.567463 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 01 16:40:17.567463 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 01 16:40:17.567463 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 01 16:40:17.567463 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application Aug 01 16:40:17.567463 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 01 16:40:17.567463 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application Aug 01 16:40:17.567463 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 16:40:17.567463 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 16:40:17.567463 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 16:40:17.567463 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 16:40:17.567463 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 16:40:17.568739 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 16:40:17.568739 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 16:40:17.568739 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 16:40:17.568739 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 16:40:17.568739 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 16:40:17.568739 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 16:40:17.568739 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 01 16:40:17.568739 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 01 16:40:17.568739 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 01 16:40:17.568739 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 01 16:40:17.568739 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 16:40:17.568739 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 16:40:17.568739 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 16:40:17.568739 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 16:40:17.568739 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 16:40:17.568739 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 16:40:17.570368 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 16:40:17.570368 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application with Lock( Aug 01 16:40:17.570368 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 16:40:17.570368 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self._enter() Aug 01 16:40:17.570368 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 16:40:17.570368 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 16:40:17.570368 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 16:40:17.570368 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self.creator() Aug 01 16:40:17.570368 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 16:40:17.570368 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application created_value = creator( Aug 01 16:40:17.570368 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 01 16:40:17.570368 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 01 16:40:17.570368 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 01 16:40:17.570368 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application Aug 01 16:40:17.573548 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 637/1290] 10.222.0.20 () {56 vars in 1117 bytes} [Mon Aug 1 16:40:17 2022] GET /identity/v3/services/placement => generated 90 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 01 16:40:17.575932 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9d4bc3e-cb4c-49bd-ae99-fcfbe7ac9390 None None] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:40:17.685512 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9d4bc3e-cb4c-49bd-ae99-fcfbe7ac9390 None None] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:40:17.722280 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99f61886-eea8-4082-965a-d8ae5fded9fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:40:17.722912 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f61886-eea8-4082-965a-d8ae5fded9fb None admin] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:40:17.723189 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f61886-eea8-4082-965a-d8ae5fded9fb None admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:40:17.723461 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f61886-eea8-4082-965a-d8ae5fded9fb None admin] PATH_INFO: `/v3/services` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:40:17.724212 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99f61886-eea8-4082-965a-d8ae5fded9fb None admin] RBAC: Adding query filter params (name=placement) {{(pid=87058) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:40:17.724537 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99f61886-eea8-4082-965a-d8ae5fded9fb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:40:17.726541 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99f61886-eea8-4082-965a-d8ae5fded9fb None admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:40:17.732133 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 654/1291] 10.222.0.20 () {56 vars in 1116 bytes} [Mon Aug 1 16:40:17 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:40:17.737849 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5760ceea-6b14-4320-8a36-23aa9290584a None admin] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:40:17.844239 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5760ceea-6b14-4320-8a36-23aa9290584a None admin] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:40:17.878943 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c3121cd-d99d-4cff-b5b0-247da9dd1d61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:40:17.879564 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3121cd-d99d-4cff-b5b0-247da9dd1d61 None admin] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:40:17.879841 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3121cd-d99d-4cff-b5b0-247da9dd1d61 None admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:40:17.880114 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3121cd-d99d-4cff-b5b0-247da9dd1d61 None admin] PATH_INFO: `/v3/services` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:40:17.880794 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c3121cd-d99d-4cff-b5b0-247da9dd1d61 None admin] RBAC: Adding query filter params (type=placement) {{(pid=87057) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:40:17.881092 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c3121cd-d99d-4cff-b5b0-247da9dd1d61 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:40:17.882811 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c3121cd-d99d-4cff-b5b0-247da9dd1d61 None admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:40:17.888170 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 638/1292] 10.222.0.20 () {56 vars in 1116 bytes} [Mon Aug 1 16:40:17 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:40:19.897492 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a589b35a-8d9c-4f32-941a-c38000ba17c9 None None] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:40:19.898060 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a589b35a-8d9c-4f32-941a-c38000ba17c9 None None] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:40:19.898060 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a589b35a-8d9c-4f32-941a-c38000ba17c9 None None] PATH_INFO: `/` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:40:19.899410 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 655/1293] 10.222.0.20 () {54 vars in 926 bytes} [Mon Aug 1 16:40:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 16:40:19.905079 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc08f51f-58ae-4965-aee4-9b83796a2ea3 None None] REQUEST_METHOD: `POST` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:40:19.905482 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc08f51f-58ae-4965-aee4-9b83796a2ea3 None None] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:40:19.905482 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc08f51f-58ae-4965-aee4-9b83796a2ea3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:40:19.924030 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.auth.core [None req-bc08f51f-58ae-4965-aee4-9b83796a2ea3 None None] MFA Rules not processed for user `1007f8e379b74e169b98fb7c3c0ba5b4`. Rule list: `[]` (Enabled: `True`). {{(pid=87057) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:40:19.925222 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.fernet_utils [None req-bc08f51f-58ae-4965-aee4-9b83796a2ea3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87057) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:40:19.954612 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG dbcounter [-] [87057] Writing DB stats keystone:SELECT=100 {{(pid=87057) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:40:19.993259 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 639/1294] 10.222.0.20 () {58 vars in 1023 bytes} [Mon Aug 1 16:40:19 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 01 16:40:20.016079 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a589b35a-8d9c-4f32-941a-c38000ba17c9 None None] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:40:20.047193 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG dbcounter [-] [87058] Writing DB stats keystone:SELECT=100 {{(pid=87058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:40:20.116918 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a589b35a-8d9c-4f32-941a-c38000ba17c9 None None] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:40:20.150452 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cefee24-9a83-4067-b483-47073f0793d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:40:20.151003 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cefee24-9a83-4067-b483-47073f0793d1 None admin] REQUEST_METHOD: `POST` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:40:20.151275 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cefee24-9a83-4067-b483-47073f0793d1 None admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:40:20.151543 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cefee24-9a83-4067-b483-47073f0793d1 None admin] PATH_INFO: `/v3/services` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:40:20.152229 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cefee24-9a83-4067-b483-47073f0793d1 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:40:20.153670 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cefee24-9a83-4067-b483-47073f0793d1 None admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:40:20.236909 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 656/1295] 10.222.0.20 () {60 vars in 1141 bytes} [Mon Aug 1 16:40:20 2022] POST /identity/v3/services => generated 246 bytes in 221 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 16:40:22.273305 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0bbc70e-018f-4c39-ac96-9a1e8119cefb None None] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:40:22.273877 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0bbc70e-018f-4c39-ac96-9a1e8119cefb None None] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:40:22.273877 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0bbc70e-018f-4c39-ac96-9a1e8119cefb None None] PATH_INFO: `/` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:40:22.275215 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 640/1296] 10.222.0.20 () {54 vars in 926 bytes} [Mon Aug 1 16:40:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 16:40:22.280401 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae7da7c7-87ef-4c54-ab63-516b3fb316cc None None] REQUEST_METHOD: `POST` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:40:22.280805 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae7da7c7-87ef-4c54-ab63-516b3fb316cc None None] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:40:22.280964 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae7da7c7-87ef-4c54-ab63-516b3fb316cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:40:22.299998 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.auth.core [None req-ae7da7c7-87ef-4c54-ab63-516b3fb316cc None None] MFA Rules not processed for user `1007f8e379b74e169b98fb7c3c0ba5b4`. Rule list: `[]` (Enabled: `True`). {{(pid=87058) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:40:22.301196 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.fernet_utils [None req-ae7da7c7-87ef-4c54-ab63-516b3fb316cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:40:22.373853 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 657/1297] 10.222.0.20 () {58 vars in 1023 bytes} [Mon Aug 1 16:40:22 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 01 16:40:22.396044 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0bbc70e-018f-4c39-ac96-9a1e8119cefb None None] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:40:22.495614 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0bbc70e-018f-4c39-ac96-9a1e8119cefb None None] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:40:22.528621 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def124d4-de04-48ad-baad-bec6cbeff3bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:40:22.529183 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def124d4-de04-48ad-baad-bec6cbeff3bc None admin] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:40:22.529457 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def124d4-de04-48ad-baad-bec6cbeff3bc None admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:40:22.529723 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def124d4-de04-48ad-baad-bec6cbeff3bc None admin] PATH_INFO: `/v3/services/placement` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:40:22.530369 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-def124d4-de04-48ad-baad-bec6cbeff3bc None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:40:22.531793 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-def124d4-de04-48ad-baad-bec6cbeff3bc None admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:40:22.541148 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application [None req-def124d4-de04-48ad-baad-bec6cbeff3bc None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Aug 01 16:40:22.541148 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 16:40:22.541148 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 01 16:40:22.541148 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 01 16:40:22.541148 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 01 16:40:22.541148 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 01 16:40:22.541148 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 01 16:40:22.541148 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 01 16:40:22.541148 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 01 16:40:22.541148 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application Aug 01 16:40:22.541148 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 01 16:40:22.541148 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application Aug 01 16:40:22.541148 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 16:40:22.541148 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 16:40:22.541148 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 16:40:22.541148 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 16:40:22.541148 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 16:40:22.542395 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 16:40:22.542395 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 16:40:22.542395 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 16:40:22.542395 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 16:40:22.542395 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 16:40:22.542395 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 16:40:22.542395 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 01 16:40:22.542395 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 01 16:40:22.542395 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 01 16:40:22.542395 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 01 16:40:22.542395 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 16:40:22.542395 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 16:40:22.542395 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 16:40:22.542395 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 16:40:22.542395 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 16:40:22.542395 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 16:40:22.543427 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 16:40:22.543427 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application with Lock( Aug 01 16:40:22.543427 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 16:40:22.543427 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self._enter() Aug 01 16:40:22.543427 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 16:40:22.543427 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 16:40:22.543427 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 16:40:22.543427 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self.creator() Aug 01 16:40:22.543427 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 16:40:22.543427 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application created_value = creator( Aug 01 16:40:22.543427 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 01 16:40:22.543427 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 01 16:40:22.543427 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 01 16:40:22.543427 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application Aug 01 16:40:22.543427 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 641/1298] 10.222.0.20 () {56 vars in 1117 bytes} [Mon Aug 1 16:40:22 2022] GET /identity/v3/services/placement => generated 90 bytes in 147 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 16:40:22.548354 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae7da7c7-87ef-4c54-ab63-516b3fb316cc None None] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:40:22.567486 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG dbcounter [-] [87058] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:40:22.646811 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae7da7c7-87ef-4c54-ab63-516b3fb316cc None None] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:40:22.681344 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5242422-a5e0-41ee-8d65-62ded7b40874 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:40:22.681962 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5242422-a5e0-41ee-8d65-62ded7b40874 None admin] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:40:22.682270 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5242422-a5e0-41ee-8d65-62ded7b40874 None admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:40:22.682545 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5242422-a5e0-41ee-8d65-62ded7b40874 None admin] PATH_INFO: `/v3/services` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:40:22.683224 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5242422-a5e0-41ee-8d65-62ded7b40874 None admin] RBAC: Adding query filter params (name=placement) {{(pid=87058) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:40:22.683529 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5242422-a5e0-41ee-8d65-62ded7b40874 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:40:22.684893 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5242422-a5e0-41ee-8d65-62ded7b40874 None admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:40:22.689952 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 658/1299] 10.222.0.20 () {56 vars in 1116 bytes} [Mon Aug 1 16:40:22 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:40:22.696526 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def124d4-de04-48ad-baad-bec6cbeff3bc None admin] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:40:22.728275 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG dbcounter [-] [87057] Writing DB stats keystone:SELECT=100 {{(pid=87057) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:40:22.797741 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def124d4-de04-48ad-baad-bec6cbeff3bc None admin] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:40:22.831790 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46634d4a-988b-491a-9a18-06873a1bc205 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:40:22.832402 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46634d4a-988b-491a-9a18-06873a1bc205 None admin] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:40:22.832683 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46634d4a-988b-491a-9a18-06873a1bc205 None admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:40:22.832955 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46634d4a-988b-491a-9a18-06873a1bc205 None admin] PATH_INFO: `/v3/endpoints` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:40:22.833709 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46634d4a-988b-491a-9a18-06873a1bc205 None admin] RBAC: Adding query filter params (interface=public, service_id=b9d94585ffa04f55bdb3a06f3308eff2, region_id=RegionOne) {{(pid=87057) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:40:22.834018 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46634d4a-988b-491a-9a18-06873a1bc205 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:40:22.835490 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46634d4a-988b-491a-9a18-06873a1bc205 None admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:40:22.840410 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 642/1300] 10.222.0.20 () {56 vars in 1251 bytes} [Mon Aug 1 16:40:22 2022] GET /identity/v3/endpoints?service_id=b9d94585ffa04f55bdb3a06f3308eff2&interface=public®ion_id=RegionOne => generated 194 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:40:22.846182 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5242422-a5e0-41ee-8d65-62ded7b40874 None admin] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:40:22.929279 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG dbcounter [-] [87058] Writing DB stats keystone:SELECT=100 {{(pid=87058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:40:22.946986 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5242422-a5e0-41ee-8d65-62ded7b40874 None admin] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:40:22.979957 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9246384d-3db9-4ec8-ac14-580359d2f6dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:40:22.980562 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9246384d-3db9-4ec8-ac14-580359d2f6dd None admin] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:40:22.980841 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9246384d-3db9-4ec8-ac14-580359d2f6dd None admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:40:22.981107 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9246384d-3db9-4ec8-ac14-580359d2f6dd None admin] PATH_INFO: `/v3/services` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:40:22.981743 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9246384d-3db9-4ec8-ac14-580359d2f6dd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:40:22.983134 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9246384d-3db9-4ec8-ac14-580359d2f6dd None admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:40:22.991734 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 659/1301] 10.222.0.20 () {56 vars in 1087 bytes} [Mon Aug 1 16:40:22 2022] GET /identity/v3/services => generated 2191 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 16:40:25.014163 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d333a1-01ea-4a67-91f7-226ce1430e2b None None] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:40:25.014680 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d333a1-01ea-4a67-91f7-226ce1430e2b None None] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:40:25.014795 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d333a1-01ea-4a67-91f7-226ce1430e2b None None] PATH_INFO: `/` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:40:25.016079 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 643/1302] 10.222.0.20 () {54 vars in 926 bytes} [Mon Aug 1 16:40:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 16:40:25.021223 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65825583-48d2-4a97-bb53-5850e3986af2 None None] REQUEST_METHOD: `POST` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:40:25.021744 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65825583-48d2-4a97-bb53-5850e3986af2 None None] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:40:25.022007 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65825583-48d2-4a97-bb53-5850e3986af2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:40:25.040443 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.auth.core [None req-65825583-48d2-4a97-bb53-5850e3986af2 None None] MFA Rules not processed for user `1007f8e379b74e169b98fb7c3c0ba5b4`. Rule list: `[]` (Enabled: `True`). {{(pid=87058) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:40:25.041739 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.fernet_utils [None req-65825583-48d2-4a97-bb53-5850e3986af2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:40:25.108473 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 660/1303] 10.222.0.20 () {58 vars in 1023 bytes} [Mon Aug 1 16:40:25 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 01 16:40:25.131803 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0d333a1-01ea-4a67-91f7-226ce1430e2b None None] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:40:25.224182 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG dbcounter [-] [87057] Writing DB stats keystone:SELECT=100 {{(pid=87057) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:40:25.231242 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0d333a1-01ea-4a67-91f7-226ce1430e2b None None] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:40:25.264216 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b03542d-8779-4b40-aa37-8cd38bb9fdbc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:40:25.264849 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b03542d-8779-4b40-aa37-8cd38bb9fdbc None admin] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:40:25.265125 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b03542d-8779-4b40-aa37-8cd38bb9fdbc None admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:40:25.265396 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b03542d-8779-4b40-aa37-8cd38bb9fdbc None admin] PATH_INFO: `/v3/services/placement` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:40:25.266017 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b03542d-8779-4b40-aa37-8cd38bb9fdbc None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:40:25.267399 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b03542d-8779-4b40-aa37-8cd38bb9fdbc None admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:40:25.275923 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application [None req-9b03542d-8779-4b40-aa37-8cd38bb9fdbc None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Aug 01 16:40:25.275923 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 16:40:25.275923 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 01 16:40:25.275923 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 01 16:40:25.275923 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 01 16:40:25.275923 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 01 16:40:25.275923 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 01 16:40:25.275923 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 01 16:40:25.275923 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 01 16:40:25.275923 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application Aug 01 16:40:25.275923 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 01 16:40:25.275923 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application Aug 01 16:40:25.275923 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 16:40:25.275923 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 16:40:25.275923 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 16:40:25.275923 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 16:40:25.275923 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 16:40:25.277071 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 16:40:25.277071 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 16:40:25.277071 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 16:40:25.277071 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 16:40:25.277071 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 16:40:25.277071 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 16:40:25.277071 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 01 16:40:25.277071 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 01 16:40:25.277071 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 01 16:40:25.277071 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 01 16:40:25.277071 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 16:40:25.277071 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 16:40:25.277071 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 16:40:25.277071 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 16:40:25.277071 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 16:40:25.277071 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 16:40:25.277973 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 16:40:25.277973 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application with Lock( Aug 01 16:40:25.277973 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 16:40:25.277973 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self._enter() Aug 01 16:40:25.277973 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 16:40:25.277973 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 16:40:25.277973 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 16:40:25.277973 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application return self.creator() Aug 01 16:40:25.277973 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 16:40:25.277973 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application created_value = creator( Aug 01 16:40:25.277973 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 01 16:40:25.277973 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 01 16:40:25.277973 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 01 16:40:25.277973 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: ERROR keystone.server.flask.application Aug 01 16:40:25.277973 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 644/1304] 10.222.0.20 () {56 vars in 1117 bytes} [Mon Aug 1 16:40:25 2022] GET /identity/v3/services/placement => generated 90 bytes in 146 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 16:40:25.283125 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65825583-48d2-4a97-bb53-5850e3986af2 None None] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:40:25.357515 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG dbcounter [-] [87058] Writing DB stats keystone:SELECT=100 {{(pid=87058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:40:25.379006 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65825583-48d2-4a97-bb53-5850e3986af2 None None] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:40:25.411917 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b614cd-c370-40c3-878b-37d3e6dc01da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:40:25.413188 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b614cd-c370-40c3-878b-37d3e6dc01da None admin] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:40:25.413470 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b614cd-c370-40c3-878b-37d3e6dc01da None admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:40:25.413741 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b614cd-c370-40c3-878b-37d3e6dc01da None admin] PATH_INFO: `/v3/services` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:40:25.414443 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94b614cd-c370-40c3-878b-37d3e6dc01da None admin] RBAC: Adding query filter params (name=placement) {{(pid=87058) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:40:25.414744 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94b614cd-c370-40c3-878b-37d3e6dc01da None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:40:25.416098 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94b614cd-c370-40c3-878b-37d3e6dc01da None admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:40:25.421047 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 661/1305] 10.222.0.20 () {56 vars in 1116 bytes} [Mon Aug 1 16:40:25 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 16:40:25.427042 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b03542d-8779-4b40-aa37-8cd38bb9fdbc None admin] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:40:25.524251 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b03542d-8779-4b40-aa37-8cd38bb9fdbc None admin] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:40:25.556787 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5add011-03a5-4ef8-9089-5d4308d5a9c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:40:25.557385 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5add011-03a5-4ef8-9089-5d4308d5a9c6 None admin] REQUEST_METHOD: `POST` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:40:25.557659 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5add011-03a5-4ef8-9089-5d4308d5a9c6 None admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:40:25.557925 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5add011-03a5-4ef8-9089-5d4308d5a9c6 None admin] PATH_INFO: `/v3/endpoints` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:40:25.558652 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5add011-03a5-4ef8-9089-5d4308d5a9c6 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:40:25.559998 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5add011-03a5-4ef8-9089-5d4308d5a9c6 None admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:40:25.636854 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 645/1306] 10.222.0.20 () {60 vars in 1144 bytes} [Mon Aug 1 16:40:25 2022] POST /identity/v3/endpoints => generated 331 bytes in 210 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 16:40:35.430766 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG dbcounter [-] [87058] Writing DB stats keystone:SELECT=30 {{(pid=87058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:40:35.631906 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG dbcounter [-] [87057] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=87057) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:46:27.878235 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2317510-7e36-40b9-8a80-be0e38cf39c6 None None] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:46:27.879107 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2317510-7e36-40b9-8a80-be0e38cf39c6 None None] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:46:27.879544 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2317510-7e36-40b9-8a80-be0e38cf39c6 None None] PATH_INFO: `/` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:46:27.886486 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4292b7f-b0bc-41f4-90a4-51f41e643ceb None None] REQUEST_METHOD: `POST` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:46:27.887294 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4292b7f-b0bc-41f4-90a4-51f41e643ceb None None] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:46:27.887718 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4292b7f-b0bc-41f4-90a4-51f41e643ceb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:46:27.890317 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 662/1307] 10.222.0.20 () {54 vars in 926 bytes} [Mon Aug 1 16:46:27 2022] GET /identity => generated 270 bytes in 13 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 16:46:28.063053 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.auth.core [None req-c4292b7f-b0bc-41f4-90a4-51f41e643ceb None None] MFA Rules not processed for user `1007f8e379b74e169b98fb7c3c0ba5b4`. Rule list: `[]` (Enabled: `True`). {{(pid=87057) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:46:28.066648 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.fernet_utils [None req-c4292b7f-b0bc-41f4-90a4-51f41e643ceb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87057) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:46:28.261714 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 646/1308] 10.222.0.20 () {58 vars in 1023 bytes} [Mon Aug 1 16:46:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 376 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 16:46:28.320591 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ce4425-5987-48ee-b4bb-1ba8e0c47ba8 None None] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:46:28.321115 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ce4425-5987-48ee-b4bb-1ba8e0c47ba8 None None] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:46:28.321555 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ce4425-5987-48ee-b4bb-1ba8e0c47ba8 None None] PATH_INFO: `/` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:46:28.323353 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 663/1309] 10.222.0.20 () {54 vars in 965 bytes} [Mon Aug 1 16:46:28 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 16:46:28.329122 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21241a64-bcff-4e8a-b42c-ba00efce1448 None None] REQUEST_METHOD: `POST` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:46:28.329507 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21241a64-bcff-4e8a-b42c-ba00efce1448 None None] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:46:28.329856 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21241a64-bcff-4e8a-b42c-ba00efce1448 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:46:28.428901 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.auth.core [None req-21241a64-bcff-4e8a-b42c-ba00efce1448 None None] MFA Rules not processed for user `3d9bd8d649154c979be3744bfd016f95`. Rule list: `[]` (Enabled: `True`). {{(pid=87057) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:46:28.430251 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.fernet_utils [None req-21241a64-bcff-4e8a-b42c-ba00efce1448 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87057) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:46:28.454869 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG dbcounter [-] [87057] Writing DB stats keystone:SELECT=100 {{(pid=87057) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:46:28.536943 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ce4425-5987-48ee-b4bb-1ba8e0c47ba8 None None] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:46:28.538847 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.fernet_utils [None req-a2ce4425-5987-48ee-b4bb-1ba8e0c47ba8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:46:28.548400 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 647/1310] 10.222.0.20 () {58 vars in 1062 bytes} [Mon Aug 1 16:46:28 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 220 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:46:28.692247 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ce4425-5987-48ee-b4bb-1ba8e0c47ba8 None None] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:46:28.782534 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be08e05b-efc5-4840-a569-a46d5df3e672 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d9bd8d649154c979be3744bfd016f95', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b992d6a65df4bf9b27c13192d57c5f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:46:28.783194 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be08e05b-efc5-4840-a569-a46d5df3e672 service neutron] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:46:28.783471 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be08e05b-efc5-4840-a569-a46d5df3e672 service neutron] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:46:28.783740 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be08e05b-efc5-4840-a569-a46d5df3e672 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:46:28.785439 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.fernet_utils [None req-be08e05b-efc5-4840-a569-a46d5df3e672 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:46:28.850715 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG dbcounter [-] [87058] Writing DB stats keystone:SELECT=100 {{(pid=87058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:46:28.922059 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be08e05b-efc5-4840-a569-a46d5df3e672 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:46:28.923523 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be08e05b-efc5-4840-a569-a46d5df3e672 service neutron] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:46:28.956610 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 664/1311] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 1 16:46:28 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 420 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 01 16:46:31.280439 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4a384c-0d60-47f9-8af6-b524a93d362a None None] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:46:31.280998 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4a384c-0d60-47f9-8af6-b524a93d362a None None] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:46:31.280998 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4a384c-0d60-47f9-8af6-b524a93d362a None None] PATH_INFO: `/` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:46:31.282519 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 648/1312] 10.222.0.20 () {54 vars in 926 bytes} [Mon Aug 1 16:46:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 16:46:31.287459 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83785bac-421b-4649-8742-e2afd1702a46 None None] REQUEST_METHOD: `POST` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:46:31.287886 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83785bac-421b-4649-8742-e2afd1702a46 None None] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:46:31.288017 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83785bac-421b-4649-8742-e2afd1702a46 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:46:31.361938 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.auth.core [None req-83785bac-421b-4649-8742-e2afd1702a46 None None] MFA Rules not processed for user `1007f8e379b74e169b98fb7c3c0ba5b4`. Rule list: `[]` (Enabled: `True`). {{(pid=87058) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:46:31.363224 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.fernet_utils [None req-83785bac-421b-4649-8742-e2afd1702a46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:46:31.469719 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 665/1313] 10.222.0.20 () {58 vars in 1023 bytes} [Mon Aug 1 16:46:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:46:31.509635 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c4a384c-0d60-47f9-8af6-b524a93d362a None None] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:46:31.511238 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.fernet_utils [None req-1c4a384c-0d60-47f9-8af6-b524a93d362a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87057) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:46:31.620292 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c4a384c-0d60-47f9-8af6-b524a93d362a None None] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:46:31.692787 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG dbcounter [-] [87057] Writing DB stats keystone:SELECT=100 {{(pid=87057) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:46:31.695317 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb878af8-e8a0-4014-bba1-561646a5dcb7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d9bd8d649154c979be3744bfd016f95', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b992d6a65df4bf9b27c13192d57c5f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:46:31.695936 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb878af8-e8a0-4014-bba1-561646a5dcb7 service neutron] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:46:31.696214 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb878af8-e8a0-4014-bba1-561646a5dcb7 service neutron] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:46:31.696483 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb878af8-e8a0-4014-bba1-561646a5dcb7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:46:31.698140 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.fernet_utils [None req-bb878af8-e8a0-4014-bba1-561646a5dcb7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87057) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:46:31.806740 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb878af8-e8a0-4014-bba1-561646a5dcb7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:46:31.808189 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb878af8-e8a0-4014-bba1-561646a5dcb7 service neutron] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:46:31.829108 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 649/1314] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 1 16:46:31 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 16:46:34.308677 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d720c8d-32ac-434d-a392-402950e71743 None None] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:46:34.309213 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d720c8d-32ac-434d-a392-402950e71743 None None] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:46:34.309213 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d720c8d-32ac-434d-a392-402950e71743 None None] PATH_INFO: `/` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:46:34.311172 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 666/1315] 10.222.0.20 () {54 vars in 926 bytes} [Mon Aug 1 16:46:34 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 16:46:34.316438 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d6d5a9-74b6-4ffa-abd2-57da0c542c8a None None] REQUEST_METHOD: `POST` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:46:34.316881 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d6d5a9-74b6-4ffa-abd2-57da0c542c8a None None] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:46:34.316980 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d6d5a9-74b6-4ffa-abd2-57da0c542c8a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:46:34.381564 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.auth.core [None req-21d6d5a9-74b6-4ffa-abd2-57da0c542c8a None None] MFA Rules not processed for user `d96b664ac2df42d2aa31b69bc44ebf19`. Rule list: `[]` (Enabled: `True`). {{(pid=87057) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:46:34.383060 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.fernet_utils [None req-21d6d5a9-74b6-4ffa-abd2-57da0c542c8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87057) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:46:34.433852 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG dbcounter [-] [87057] Writing DB stats keystone:SELECT=100 {{(pid=87057) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:46:34.477990 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 650/1316] 10.222.0.20 () {58 vars in 1023 bytes} [Mon Aug 1 16:46:34 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:46:34.518523 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d720c8d-32ac-434d-a392-402950e71743 None None] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:46:34.520087 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.fernet_utils [None req-4d720c8d-32ac-434d-a392-402950e71743 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:46:34.529832 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG dbcounter [-] [87058] Writing DB stats keystone:SELECT=100 {{(pid=87058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:46:34.622929 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d720c8d-32ac-434d-a392-402950e71743 None None] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:46:34.694943 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86478cf1-aa01-4605-8240-1a74b7a48f7e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d9bd8d649154c979be3744bfd016f95', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b992d6a65df4bf9b27c13192d57c5f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:46:34.695553 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86478cf1-aa01-4605-8240-1a74b7a48f7e service neutron] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:46:34.695826 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86478cf1-aa01-4605-8240-1a74b7a48f7e service neutron] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:46:34.696091 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86478cf1-aa01-4605-8240-1a74b7a48f7e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:46:34.697686 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.fernet_utils [None req-86478cf1-aa01-4605-8240-1a74b7a48f7e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:46:34.759046 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG dbcounter [-] [87058] Writing DB stats keystone:SELECT=100 {{(pid=87058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:46:34.806151 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86478cf1-aa01-4605-8240-1a74b7a48f7e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:46:34.807612 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86478cf1-aa01-4605-8240-1a74b7a48f7e service neutron] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:46:34.825131 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 667/1317] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 1 16:46:34 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 16:46:37.577039 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8bf23b-dedf-40bf-9dc6-9e708fc80ab4 None None] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:46:37.577605 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8bf23b-dedf-40bf-9dc6-9e708fc80ab4 None None] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:46:37.577605 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8bf23b-dedf-40bf-9dc6-9e708fc80ab4 None None] PATH_INFO: `/` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:46:37.578982 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 651/1318] 10.222.0.20 () {54 vars in 926 bytes} [Mon Aug 1 16:46:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 16:46:37.584103 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bbd1e07-1072-47b3-9034-7f27fd403a4f None None] REQUEST_METHOD: `POST` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:46:37.584535 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bbd1e07-1072-47b3-9034-7f27fd403a4f None None] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:46:37.584630 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bbd1e07-1072-47b3-9034-7f27fd403a4f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:46:37.649133 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.auth.core [None req-0bbd1e07-1072-47b3-9034-7f27fd403a4f None None] MFA Rules not processed for user `1007f8e379b74e169b98fb7c3c0ba5b4`. Rule list: `[]` (Enabled: `True`). {{(pid=87058) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:46:37.650346 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.fernet_utils [None req-0bbd1e07-1072-47b3-9034-7f27fd403a4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:46:37.747517 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 668/1319] 10.222.0.20 () {58 vars in 1023 bytes} [Mon Aug 1 16:46:37 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:46:37.787286 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc8bf23b-dedf-40bf-9dc6-9e708fc80ab4 None None] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:46:37.788906 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.fernet_utils [None req-dc8bf23b-dedf-40bf-9dc6-9e708fc80ab4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87057) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:46:37.892899 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc8bf23b-dedf-40bf-9dc6-9e708fc80ab4 None None] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:46:37.968475 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55a5cd43-9625-45a1-8fff-1fd01c6a07e0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d9bd8d649154c979be3744bfd016f95', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b992d6a65df4bf9b27c13192d57c5f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:46:37.969085 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a5cd43-9625-45a1-8fff-1fd01c6a07e0 service neutron] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:46:37.969360 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a5cd43-9625-45a1-8fff-1fd01c6a07e0 service neutron] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:46:37.969630 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a5cd43-9625-45a1-8fff-1fd01c6a07e0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:46:37.971278 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.fernet_utils [None req-55a5cd43-9625-45a1-8fff-1fd01c6a07e0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87057) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:46:37.977924 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG dbcounter [-] [87057] Writing DB stats keystone:SELECT=100 {{(pid=87057) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:46:38.195589 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55a5cd43-9625-45a1-8fff-1fd01c6a07e0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:46:38.196924 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55a5cd43-9625-45a1-8fff-1fd01c6a07e0 service neutron] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:46:38.214308 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 652/1320] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 1 16:46:37 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 428 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 16:46:41.213835 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f522ef62-d071-4e19-9789-628e747a489d None None] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:46:41.214372 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f522ef62-d071-4e19-9789-628e747a489d None None] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:46:41.214372 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f522ef62-d071-4e19-9789-628e747a489d None None] PATH_INFO: `/` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:46:41.215723 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 669/1321] 10.222.0.20 () {54 vars in 926 bytes} [Mon Aug 1 16:46:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 16:46:41.221125 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91d2639-da9f-4360-8d82-f5ca102cc707 None None] REQUEST_METHOD: `POST` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:46:41.221556 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91d2639-da9f-4360-8d82-f5ca102cc707 None None] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:46:41.221701 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91d2639-da9f-4360-8d82-f5ca102cc707 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:46:41.287267 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.auth.core [None req-a91d2639-da9f-4360-8d82-f5ca102cc707 None None] MFA Rules not processed for user `1007f8e379b74e169b98fb7c3c0ba5b4`. Rule list: `[]` (Enabled: `True`). {{(pid=87057) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:46:41.288481 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.fernet_utils [None req-a91d2639-da9f-4360-8d82-f5ca102cc707 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87057) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:46:41.347325 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG dbcounter [-] [87057] Writing DB stats keystone:SELECT=100 {{(pid=87057) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:46:41.388369 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 653/1322] 10.222.0.20 () {58 vars in 1023 bytes} [Mon Aug 1 16:46:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:46:41.424956 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f522ef62-d071-4e19-9789-628e747a489d None None] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:46:41.426513 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.fernet_utils [None req-f522ef62-d071-4e19-9789-628e747a489d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:46:41.448108 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG dbcounter [-] [87058] Writing DB stats keystone:SELECT=100 {{(pid=87058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:46:41.543093 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f522ef62-d071-4e19-9789-628e747a489d None None] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:46:41.621439 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c912ed69-9094-4256-98cd-97572d2490d9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d9bd8d649154c979be3744bfd016f95', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b992d6a65df4bf9b27c13192d57c5f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:46:41.622080 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c912ed69-9094-4256-98cd-97572d2490d9 service neutron] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:46:41.622359 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c912ed69-9094-4256-98cd-97572d2490d9 service neutron] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:46:41.622626 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c912ed69-9094-4256-98cd-97572d2490d9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:46:41.624882 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.fernet_utils [None req-c912ed69-9094-4256-98cd-97572d2490d9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:46:41.702799 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG dbcounter [-] [87058] Writing DB stats keystone:SELECT=100 {{(pid=87058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:46:41.741560 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c912ed69-9094-4256-98cd-97572d2490d9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:46:41.743039 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c912ed69-9094-4256-98cd-97572d2490d9 service neutron] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:46:41.761842 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 670/1323] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 1 16:46:41 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 337 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 16:46:43.833775 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62d4989-1a89-40c5-9c85-c986fe806eff None None] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:46:43.834435 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62d4989-1a89-40c5-9c85-c986fe806eff None None] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:46:43.834435 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62d4989-1a89-40c5-9c85-c986fe806eff None None] PATH_INFO: `/` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:46:43.835710 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 654/1324] 10.222.0.20 () {54 vars in 926 bytes} [Mon Aug 1 16:46:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 16:46:43.840940 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b2c57b-0290-4a36-9bbf-63cb97891d1e None None] REQUEST_METHOD: `POST` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:46:43.841394 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b2c57b-0290-4a36-9bbf-63cb97891d1e None None] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:46:43.841615 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b2c57b-0290-4a36-9bbf-63cb97891d1e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:46:43.911026 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.auth.core [None req-99b2c57b-0290-4a36-9bbf-63cb97891d1e None None] MFA Rules not processed for user `1007f8e379b74e169b98fb7c3c0ba5b4`. Rule list: `[]` (Enabled: `True`). {{(pid=87058) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:46:43.912274 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.fernet_utils [None req-99b2c57b-0290-4a36-9bbf-63cb97891d1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:46:44.009739 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 671/1325] 10.222.0.20 () {58 vars in 1023 bytes} [Mon Aug 1 16:46:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:46:44.045574 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f62d4989-1a89-40c5-9c85-c986fe806eff None None] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:46:44.048015 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.fernet_utils [None req-f62d4989-1a89-40c5-9c85-c986fe806eff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87057) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:46:44.151317 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f62d4989-1a89-40c5-9c85-c986fe806eff None None] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:46:44.224978 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cbae296-0e14-4586-9de8-210bb4a8f34b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d9bd8d649154c979be3744bfd016f95', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b992d6a65df4bf9b27c13192d57c5f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:46:44.225596 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cbae296-0e14-4586-9de8-210bb4a8f34b service neutron] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:46:44.225876 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cbae296-0e14-4586-9de8-210bb4a8f34b service neutron] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:46:44.226178 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cbae296-0e14-4586-9de8-210bb4a8f34b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:46:44.227795 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.fernet_utils [None req-3cbae296-0e14-4586-9de8-210bb4a8f34b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87057) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:46:44.244579 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG dbcounter [-] [87057] Writing DB stats keystone:SELECT=100 {{(pid=87057) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:46:44.339443 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cbae296-0e14-4586-9de8-210bb4a8f34b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:46:44.341235 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cbae296-0e14-4586-9de8-210bb4a8f34b service neutron] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:46:44.362460 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 655/1326] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 1 16:46:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 16:46:46.441115 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e62840-271b-469e-a380-53a32f665c4d None None] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:46:46.441657 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e62840-271b-469e-a380-53a32f665c4d None None] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:46:46.441754 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e62840-271b-469e-a380-53a32f665c4d None None] PATH_INFO: `/` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:46:46.443170 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 672/1327] 10.222.0.20 () {54 vars in 926 bytes} [Mon Aug 1 16:46:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 16:46:46.448510 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3f89d4-ebf6-4a27-9a8c-1e270d23fd78 None None] REQUEST_METHOD: `POST` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:46:46.448919 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3f89d4-ebf6-4a27-9a8c-1e270d23fd78 None None] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:46:46.449063 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3f89d4-ebf6-4a27-9a8c-1e270d23fd78 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:46:46.521190 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.auth.core [None req-0a3f89d4-ebf6-4a27-9a8c-1e270d23fd78 None None] MFA Rules not processed for user `d96b664ac2df42d2aa31b69bc44ebf19`. Rule list: `[]` (Enabled: `True`). {{(pid=87057) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:46:46.522434 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.fernet_utils [None req-0a3f89d4-ebf6-4a27-9a8c-1e270d23fd78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87057) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:46:46.586481 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG dbcounter [-] [87057] Writing DB stats keystone:SELECT=100 {{(pid=87057) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:46:46.618080 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 656/1328] 10.222.0.20 () {58 vars in 1023 bytes} [Mon Aug 1 16:46:46 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:46:46.671983 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40a1e68-ced1-4561-bdaf-b482d7ae2bd4 None None] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:46:46.672355 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40a1e68-ced1-4561-bdaf-b482d7ae2bd4 None None] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:46:46.672630 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40a1e68-ced1-4561-bdaf-b482d7ae2bd4 None None] PATH_INFO: `/` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:46:46.674013 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 673/1329] 10.222.0.20 () {54 vars in 965 bytes} [Mon Aug 1 16:46:46 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 16:46:46.680368 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5659bfe-beca-495a-bcdb-3062d7674a0d None None] REQUEST_METHOD: `POST` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:46:46.680649 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5659bfe-beca-495a-bcdb-3062d7674a0d None None] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:46:46.680921 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5659bfe-beca-495a-bcdb-3062d7674a0d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:46:46.759391 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.auth.core [None req-b5659bfe-beca-495a-bcdb-3062d7674a0d None None] MFA Rules not processed for user `3d9bd8d649154c979be3744bfd016f95`. Rule list: `[]` (Enabled: `True`). {{(pid=87057) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:46:46.760627 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.fernet_utils [None req-b5659bfe-beca-495a-bcdb-3062d7674a0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87057) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:46:46.842629 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 657/1330] 10.222.0.20 () {58 vars in 1062 bytes} [Mon Aug 1 16:46:46 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:46:46.851998 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b40a1e68-ced1-4561-bdaf-b482d7ae2bd4 None None] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:46:46.853523 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.fernet_utils [None req-b40a1e68-ced1-4561-bdaf-b482d7ae2bd4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:46:46.884875 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG dbcounter [-] [87058] Writing DB stats keystone:SELECT=100 {{(pid=87058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:46:46.960029 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b40a1e68-ced1-4561-bdaf-b482d7ae2bd4 None None] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:46:47.037368 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c31b0ec3-a539-4a7c-b8c8-d81aa8a72dc6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d9bd8d649154c979be3744bfd016f95', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b992d6a65df4bf9b27c13192d57c5f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:46:47.037993 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31b0ec3-a539-4a7c-b8c8-d81aa8a72dc6 service neutron] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:46:47.038302 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31b0ec3-a539-4a7c-b8c8-d81aa8a72dc6 service neutron] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:46:47.038573 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31b0ec3-a539-4a7c-b8c8-d81aa8a72dc6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:46:47.040198 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.fernet_utils [None req-c31b0ec3-a539-4a7c-b8c8-d81aa8a72dc6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:46:47.116765 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG dbcounter [-] [87058] Writing DB stats keystone:SELECT=100 {{(pid=87058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:46:47.149318 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c31b0ec3-a539-4a7c-b8c8-d81aa8a72dc6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:46:47.150794 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c31b0ec3-a539-4a7c-b8c8-d81aa8a72dc6 service neutron] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:46:47.186121 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 674/1331] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 1 16:46:46 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 335 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 16:46:50.021618 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473344a8-0e14-4c3b-8846-cc0a9c26bf43 None None] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:46:50.022281 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473344a8-0e14-4c3b-8846-cc0a9c26bf43 None None] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:46:50.022281 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473344a8-0e14-4c3b-8846-cc0a9c26bf43 None None] PATH_INFO: `/` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:46:50.024035 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 658/1332] 10.222.0.20 () {54 vars in 926 bytes} [Mon Aug 1 16:46:50 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 16:46:50.029444 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253545c9-adf3-47e3-9406-ef82b7a84d3c None None] REQUEST_METHOD: `POST` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:46:50.029925 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253545c9-adf3-47e3-9406-ef82b7a84d3c None None] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:46:50.030059 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253545c9-adf3-47e3-9406-ef82b7a84d3c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:46:50.097037 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.auth.core [None req-253545c9-adf3-47e3-9406-ef82b7a84d3c None None] MFA Rules not processed for user `1007f8e379b74e169b98fb7c3c0ba5b4`. Rule list: `[]` (Enabled: `True`). {{(pid=87058) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:46:50.098222 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.fernet_utils [None req-253545c9-adf3-47e3-9406-ef82b7a84d3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:46:50.192845 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 675/1333] 10.222.0.20 () {58 vars in 1023 bytes} [Mon Aug 1 16:46:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:46:50.228972 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-473344a8-0e14-4c3b-8846-cc0a9c26bf43 None None] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:46:50.230530 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.fernet_utils [None req-473344a8-0e14-4c3b-8846-cc0a9c26bf43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87057) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:46:50.299097 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG dbcounter [-] [87057] Writing DB stats keystone:SELECT=100 {{(pid=87057) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:46:50.332136 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-473344a8-0e14-4c3b-8846-cc0a9c26bf43 None None] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:46:50.404086 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8401e735-2585-4480-8f7b-4590677c5285 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d9bd8d649154c979be3744bfd016f95', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b992d6a65df4bf9b27c13192d57c5f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:46:50.404620 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8401e735-2585-4480-8f7b-4590677c5285 service neutron] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:46:50.404896 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8401e735-2585-4480-8f7b-4590677c5285 service neutron] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:46:50.405167 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8401e735-2585-4480-8f7b-4590677c5285 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:46:50.407467 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.fernet_utils [None req-8401e735-2585-4480-8f7b-4590677c5285 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87057) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:46:50.513265 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8401e735-2585-4480-8f7b-4590677c5285 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:46:50.515356 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8401e735-2585-4480-8f7b-4590677c5285 service neutron] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:46:50.524973 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG dbcounter [-] [87057] Writing DB stats keystone:SELECT=100 {{(pid=87057) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:46:50.533288 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 659/1334] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 1 16:46:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 16:46:52.572617 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49161051-3339-457a-8608-5107a829fb42 None None] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:46:52.573182 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49161051-3339-457a-8608-5107a829fb42 None None] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:46:52.573182 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49161051-3339-457a-8608-5107a829fb42 None None] PATH_INFO: `/` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:46:52.574578 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 676/1335] 10.222.0.20 () {54 vars in 926 bytes} [Mon Aug 1 16:46:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 16:46:52.579856 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ce84e1-6b6b-4050-82ab-5e701621d1ff None None] REQUEST_METHOD: `POST` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:46:52.580241 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ce84e1-6b6b-4050-82ab-5e701621d1ff None None] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:46:52.580489 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ce84e1-6b6b-4050-82ab-5e701621d1ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:46:52.644987 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.auth.core [None req-a7ce84e1-6b6b-4050-82ab-5e701621d1ff None None] MFA Rules not processed for user `1007f8e379b74e169b98fb7c3c0ba5b4`. Rule list: `[]` (Enabled: `True`). {{(pid=87057) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:46:52.646209 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.fernet_utils [None req-a7ce84e1-6b6b-4050-82ab-5e701621d1ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87057) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:46:52.740195 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 660/1336] 10.222.0.20 () {58 vars in 1023 bytes} [Mon Aug 1 16:46:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:46:52.781240 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49161051-3339-457a-8608-5107a829fb42 None None] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:46:52.782824 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.fernet_utils [None req-49161051-3339-457a-8608-5107a829fb42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:46:52.822720 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG dbcounter [-] [87058] Writing DB stats keystone:SELECT=100 {{(pid=87058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:46:52.886260 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49161051-3339-457a-8608-5107a829fb42 None None] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:46:52.960934 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0fd4bfc-5dff-4ada-ad2d-11afefac2219 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d9bd8d649154c979be3744bfd016f95', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b992d6a65df4bf9b27c13192d57c5f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:46:52.961562 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0fd4bfc-5dff-4ada-ad2d-11afefac2219 service neutron] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:46:52.961846 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0fd4bfc-5dff-4ada-ad2d-11afefac2219 service neutron] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:46:52.962161 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0fd4bfc-5dff-4ada-ad2d-11afefac2219 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:46:52.963866 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.fernet_utils [None req-b0fd4bfc-5dff-4ada-ad2d-11afefac2219 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:46:53.051911 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG dbcounter [-] [87058] Writing DB stats keystone:SELECT=100 {{(pid=87058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:46:53.078303 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0fd4bfc-5dff-4ada-ad2d-11afefac2219 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:46:53.079680 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0fd4bfc-5dff-4ada-ad2d-11afefac2219 service neutron] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:46:53.098044 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 677/1337] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 1 16:46:52 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 01 16:46:55.832826 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d81188a-9ba8-4320-93dc-9fc91d70f680 None None] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:46:55.833390 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d81188a-9ba8-4320-93dc-9fc91d70f680 None None] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:46:55.833390 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d81188a-9ba8-4320-93dc-9fc91d70f680 None None] PATH_INFO: `/` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:46:55.834778 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 661/1338] 10.222.0.20 () {54 vars in 926 bytes} [Mon Aug 1 16:46:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 16:46:55.839861 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f8ec3d-f365-4dc2-85f7-abf54422f59a None None] REQUEST_METHOD: `POST` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:46:55.840325 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f8ec3d-f365-4dc2-85f7-abf54422f59a None None] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:46:55.840425 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f8ec3d-f365-4dc2-85f7-abf54422f59a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:46:55.907372 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.auth.core [None req-65f8ec3d-f365-4dc2-85f7-abf54422f59a None None] MFA Rules not processed for user `1007f8e379b74e169b98fb7c3c0ba5b4`. Rule list: `[]` (Enabled: `True`). {{(pid=87058) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:46:55.908583 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.fernet_utils [None req-65f8ec3d-f365-4dc2-85f7-abf54422f59a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:46:56.002437 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 678/1339] 10.222.0.20 () {58 vars in 1023 bytes} [Mon Aug 1 16:46:55 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:46:56.038898 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d81188a-9ba8-4320-93dc-9fc91d70f680 None None] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:46:56.040429 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.fernet_utils [None req-1d81188a-9ba8-4320-93dc-9fc91d70f680 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87057) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:46:56.116958 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG dbcounter [-] [87057] Writing DB stats keystone:SELECT=100 {{(pid=87057) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:46:56.145557 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d81188a-9ba8-4320-93dc-9fc91d70f680 None None] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:46:56.217158 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6997225-1228-4870-b90a-0ca8134a722e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d9bd8d649154c979be3744bfd016f95', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b992d6a65df4bf9b27c13192d57c5f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:46:56.217991 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6997225-1228-4870-b90a-0ca8134a722e service neutron] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:46:56.218264 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6997225-1228-4870-b90a-0ca8134a722e service neutron] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:46:56.218536 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6997225-1228-4870-b90a-0ca8134a722e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:46:56.220144 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.fernet_utils [None req-b6997225-1228-4870-b90a-0ca8134a722e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87057) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:46:56.326055 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6997225-1228-4870-b90a-0ca8134a722e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:46:56.327514 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6997225-1228-4870-b90a-0ca8134a722e service neutron] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:46:56.345340 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 662/1340] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 1 16:46:56 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 16:47:00.571355 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08852a4-27df-43ec-a9d6-93a12345d935 None None] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:47:00.571904 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08852a4-27df-43ec-a9d6-93a12345d935 None None] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:47:00.571904 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08852a4-27df-43ec-a9d6-93a12345d935 None None] PATH_INFO: `/` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:47:00.573256 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 679/1341] 10.222.0.20 () {54 vars in 926 bytes} [Mon Aug 1 16:47:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 16:47:00.578951 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d71b67-d388-47f6-8fad-57f734394598 None None] REQUEST_METHOD: `POST` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:47:00.579398 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d71b67-d388-47f6-8fad-57f734394598 None None] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:47:00.579490 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d71b67-d388-47f6-8fad-57f734394598 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:47:00.585133 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG dbcounter [-] [87057] Writing DB stats keystone:SELECT=100 {{(pid=87057) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:47:00.647564 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.auth.core [None req-51d71b67-d388-47f6-8fad-57f734394598 None None] MFA Rules not processed for user `1007f8e379b74e169b98fb7c3c0ba5b4`. Rule list: `[]` (Enabled: `True`). {{(pid=87057) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:47:00.648796 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.fernet_utils [None req-51d71b67-d388-47f6-8fad-57f734394598 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87057) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:47:00.747562 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 663/1342] 10.222.0.20 () {58 vars in 1023 bytes} [Mon Aug 1 16:47:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:47:00.786917 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f08852a4-27df-43ec-a9d6-93a12345d935 None None] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:47:00.788612 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.fernet_utils [None req-f08852a4-27df-43ec-a9d6-93a12345d935 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:47:00.838412 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG dbcounter [-] [87058] Writing DB stats keystone:SELECT=100 {{(pid=87058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:47:00.893608 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f08852a4-27df-43ec-a9d6-93a12345d935 None None] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:47:00.971075 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1e163bf-6a2d-47fe-814c-4be7fa7f78df service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d9bd8d649154c979be3744bfd016f95', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b992d6a65df4bf9b27c13192d57c5f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:47:00.971788 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e163bf-6a2d-47fe-814c-4be7fa7f78df service neutron] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:47:00.972136 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e163bf-6a2d-47fe-814c-4be7fa7f78df service neutron] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:47:00.972494 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e163bf-6a2d-47fe-814c-4be7fa7f78df service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:47:00.974281 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.fernet_utils [None req-b1e163bf-6a2d-47fe-814c-4be7fa7f78df service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:47:01.067867 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG dbcounter [-] [87058] Writing DB stats keystone:SELECT=100 {{(pid=87058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:47:01.083932 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1e163bf-6a2d-47fe-814c-4be7fa7f78df service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:47:01.085383 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1e163bf-6a2d-47fe-814c-4be7fa7f78df service neutron] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:47:01.104595 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 680/1343] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 1 16:47:00 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 16:47:04.188025 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9936479-b605-405f-ba10-281d149ec49b None None] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:47:04.188538 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9936479-b605-405f-ba10-281d149ec49b None None] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:47:04.188538 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9936479-b605-405f-ba10-281d149ec49b None None] PATH_INFO: `/` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:47:04.189983 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 664/1344] 10.222.0.20 () {54 vars in 926 bytes} [Mon Aug 1 16:47:04 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 16:47:04.194977 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be5a9a2-5c34-4764-91bc-2c919cc27f01 None None] REQUEST_METHOD: `POST` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:47:04.195372 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be5a9a2-5c34-4764-91bc-2c919cc27f01 None None] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:47:04.195536 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be5a9a2-5c34-4764-91bc-2c919cc27f01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:47:04.261826 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.auth.core [None req-3be5a9a2-5c34-4764-91bc-2c919cc27f01 None None] MFA Rules not processed for user `1007f8e379b74e169b98fb7c3c0ba5b4`. Rule list: `[]` (Enabled: `True`). {{(pid=87058) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:47:04.263061 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.fernet_utils [None req-3be5a9a2-5c34-4764-91bc-2c919cc27f01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:47:04.361983 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 681/1345] 10.222.0.20 () {58 vars in 1023 bytes} [Mon Aug 1 16:47:04 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:47:04.398926 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9936479-b605-405f-ba10-281d149ec49b None None] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:47:04.400461 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.fernet_utils [None req-e9936479-b605-405f-ba10-281d149ec49b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87057) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:47:04.488965 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG dbcounter [-] [87057] Writing DB stats keystone:SELECT=100 {{(pid=87057) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:47:04.505815 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9936479-b605-405f-ba10-281d149ec49b None None] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:47:04.585322 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e927e808-400f-40f1-9d0a-6e8c242392de service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d9bd8d649154c979be3744bfd016f95', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b992d6a65df4bf9b27c13192d57c5f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:47:04.585945 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e927e808-400f-40f1-9d0a-6e8c242392de service neutron] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:47:04.586261 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e927e808-400f-40f1-9d0a-6e8c242392de service neutron] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:47:04.586534 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e927e808-400f-40f1-9d0a-6e8c242392de service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:47:04.588157 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.fernet_utils [None req-e927e808-400f-40f1-9d0a-6e8c242392de service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87057) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:47:04.700619 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e927e808-400f-40f1-9d0a-6e8c242392de service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:47:04.702728 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e927e808-400f-40f1-9d0a-6e8c242392de service neutron] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:47:04.722699 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 665/1346] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 1 16:47:04 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 16:47:09.108505 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e4fe5a-36b8-4f52-8a7f-01c6369e94d1 None None] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:47:09.109047 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e4fe5a-36b8-4f52-8a7f-01c6369e94d1 None None] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:47:09.109047 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e4fe5a-36b8-4f52-8a7f-01c6369e94d1 None None] PATH_INFO: `/` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:47:09.110433 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 682/1347] 10.222.0.20 () {54 vars in 926 bytes} [Mon Aug 1 16:47:09 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 16:47:09.115637 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53b2994-d642-4a7d-a349-e9d9d9053bc6 None None] REQUEST_METHOD: `POST` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:47:09.116084 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53b2994-d642-4a7d-a349-e9d9d9053bc6 None None] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:47:09.116182 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53b2994-d642-4a7d-a349-e9d9d9053bc6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:47:09.133313 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG dbcounter [-] [87057] Writing DB stats keystone:SELECT=100 {{(pid=87057) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:47:09.188719 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.auth.core [None req-f53b2994-d642-4a7d-a349-e9d9d9053bc6 None None] MFA Rules not processed for user `1007f8e379b74e169b98fb7c3c0ba5b4`. Rule list: `[]` (Enabled: `True`). {{(pid=87057) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:47:09.189951 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.fernet_utils [None req-f53b2994-d642-4a7d-a349-e9d9d9053bc6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87057) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:47:09.293893 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 666/1348] 10.222.0.20 () {58 vars in 1023 bytes} [Mon Aug 1 16:47:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:47:09.333080 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e4fe5a-36b8-4f52-8a7f-01c6369e94d1 None None] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:47:09.334657 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.fernet_utils [None req-00e4fe5a-36b8-4f52-8a7f-01c6369e94d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:47:09.394834 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG dbcounter [-] [87058] Writing DB stats keystone:SELECT=100 {{(pid=87058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:47:09.444182 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e4fe5a-36b8-4f52-8a7f-01c6369e94d1 None None] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:47:09.521577 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a70036eb-fdd9-46fc-b3df-2e3f5f03d247 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d9bd8d649154c979be3744bfd016f95', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b992d6a65df4bf9b27c13192d57c5f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:47:09.522258 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70036eb-fdd9-46fc-b3df-2e3f5f03d247 service neutron] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:47:09.522540 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70036eb-fdd9-46fc-b3df-2e3f5f03d247 service neutron] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:47:09.522811 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70036eb-fdd9-46fc-b3df-2e3f5f03d247 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:47:09.524431 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.fernet_utils [None req-a70036eb-fdd9-46fc-b3df-2e3f5f03d247 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:47:09.630915 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG dbcounter [-] [87058] Writing DB stats keystone:SELECT=100 {{(pid=87058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:47:09.634439 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a70036eb-fdd9-46fc-b3df-2e3f5f03d247 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:47:09.635946 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a70036eb-fdd9-46fc-b3df-2e3f5f03d247 service neutron] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:47:09.654279 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 683/1349] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 1 16:47:09 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 16:47:14.590782 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307ccc8b-c554-4e17-9106-d82c19522f39 None None] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:47:14.591623 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307ccc8b-c554-4e17-9106-d82c19522f39 None None] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:47:14.593983 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307ccc8b-c554-4e17-9106-d82c19522f39 None None] PATH_INFO: `/` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:47:14.595818 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 667/1350] 10.222.0.20 () {54 vars in 921 bytes} [Mon Aug 1 16:47:14 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 16:47:14.601654 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f25a030-e350-4d5e-b122-16bc91a0ee2d None None] REQUEST_METHOD: `POST` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:47:14.602468 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f25a030-e350-4d5e-b122-16bc91a0ee2d None None] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:47:14.602908 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f25a030-e350-4d5e-b122-16bc91a0ee2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:47:14.768257 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.auth.core [None req-4f25a030-e350-4d5e-b122-16bc91a0ee2d None None] MFA Rules not processed for user `8362293261614623bcdabc4120a97a9d`. Rule list: `[]` (Enabled: `True`). {{(pid=87058) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:47:14.769829 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.fernet_utils [None req-4f25a030-e350-4d5e-b122-16bc91a0ee2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:47:14.889692 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 684/1351] 10.222.0.20 () {58 vars in 1018 bytes} [Mon Aug 1 16:47:14 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 288 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:47:18.999384 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8caefa-82b3-4c30-ad4f-0706785bb1e2 None None] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:47:19.000295 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8caefa-82b3-4c30-ad4f-0706785bb1e2 None None] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:47:19.000575 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8caefa-82b3-4c30-ad4f-0706785bb1e2 None None] PATH_INFO: `/` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:47:19.001964 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 668/1352] 10.222.0.20 () {54 vars in 921 bytes} [Mon Aug 1 16:47:18 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 16:47:19.007612 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212a115b-e751-4ef2-b86e-3f00499a2f1f None None] REQUEST_METHOD: `POST` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:47:19.008406 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212a115b-e751-4ef2-b86e-3f00499a2f1f None None] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:47:19.008750 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212a115b-e751-4ef2-b86e-3f00499a2f1f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:47:19.066797 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG dbcounter [-] [87058] Writing DB stats keystone:SELECT=100 {{(pid=87058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:47:19.090394 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.auth.core [None req-212a115b-e751-4ef2-b86e-3f00499a2f1f None None] MFA Rules not processed for user `8362293261614623bcdabc4120a97a9d`. Rule list: `[]` (Enabled: `True`). {{(pid=87058) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:47:19.091623 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.fernet_utils [None req-212a115b-e751-4ef2-b86e-3f00499a2f1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:47:19.218299 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 685/1353] 10.222.0.20 () {58 vars in 1018 bytes} [Mon Aug 1 16:47:19 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 212 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:47:19.293702 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG dbcounter [-] [87057] Writing DB stats keystone:SELECT=59 {{(pid=87057) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:47:20.701454 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb958c7-e834-40a2-8640-519683051c68 None None] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:47:20.702149 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb958c7-e834-40a2-8640-519683051c68 None None] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:47:20.702149 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb958c7-e834-40a2-8640-519683051c68 None None] PATH_INFO: `/` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:47:20.703565 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 669/1354] 10.222.0.20 () {54 vars in 921 bytes} [Mon Aug 1 16:47:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 16:47:20.708369 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0758f5d2-2e57-4fed-bf62-d77d06aa402d None None] REQUEST_METHOD: `POST` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:47:20.708799 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0758f5d2-2e57-4fed-bf62-d77d06aa402d None None] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:47:20.712341 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0758f5d2-2e57-4fed-bf62-d77d06aa402d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:47:20.787276 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.auth.core [None req-0758f5d2-2e57-4fed-bf62-d77d06aa402d None None] MFA Rules not processed for user `8362293261614623bcdabc4120a97a9d`. Rule list: `[]` (Enabled: `True`). {{(pid=87058) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:47:20.788488 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.fernet_utils [None req-0758f5d2-2e57-4fed-bf62-d77d06aa402d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:47:20.829989 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG dbcounter [-] [87058] Writing DB stats keystone:SELECT=100 {{(pid=87058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:47:20.896664 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 686/1355] 10.222.0.20 () {58 vars in 1018 bytes} [Mon Aug 1 16:47:20 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:47:24.226562 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817363f2-3eab-4809-bb41-feb59a6d15a5 None None] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:47:24.227114 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817363f2-3eab-4809-bb41-feb59a6d15a5 None None] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:47:24.227114 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817363f2-3eab-4809-bb41-feb59a6d15a5 None None] PATH_INFO: `/` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:47:24.229452 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 670/1356] 10.222.0.20 () {54 vars in 926 bytes} [Mon Aug 1 16:47:24 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 16:47:24.235153 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5846f85e-d42b-489f-9cdb-15a5a325a698 None None] REQUEST_METHOD: `POST` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:47:24.235613 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5846f85e-d42b-489f-9cdb-15a5a325a698 None None] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:47:24.235710 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5846f85e-d42b-489f-9cdb-15a5a325a698 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:47:24.322331 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.auth.core [None req-5846f85e-d42b-489f-9cdb-15a5a325a698 None None] MFA Rules not processed for user `1007f8e379b74e169b98fb7c3c0ba5b4`. Rule list: `[]` (Enabled: `True`). {{(pid=87058) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:47:24.323772 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.fernet_utils [None req-5846f85e-d42b-489f-9cdb-15a5a325a698 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:47:24.432558 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 687/1357] 10.222.0.20 () {58 vars in 1023 bytes} [Mon Aug 1 16:47:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:47:24.486927 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c5df43-c8f9-413f-a1b1-5b91d2923ab2 None None] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:47:24.487473 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c5df43-c8f9-413f-a1b1-5b91d2923ab2 None None] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:47:24.487957 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c5df43-c8f9-413f-a1b1-5b91d2923ab2 None None] PATH_INFO: `/` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:47:24.489552 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 671/1358] 10.222.0.20 () {54 vars in 962 bytes} [Mon Aug 1 16:47:24 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 16:47:24.497667 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35212ab9-97f1-4f51-960f-5b24f92921ff None None] REQUEST_METHOD: `POST` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:47:24.498511 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35212ab9-97f1-4f51-960f-5b24f92921ff None None] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:47:24.505078 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35212ab9-97f1-4f51-960f-5b24f92921ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:47:24.518876 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG dbcounter [-] [87058] Writing DB stats keystone:SELECT=100 {{(pid=87058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:47:24.586556 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.auth.core [None req-35212ab9-97f1-4f51-960f-5b24f92921ff None None] MFA Rules not processed for user `6533106d03eb4ca185a8839336d2aff9`. Rule list: `[]` (Enabled: `True`). {{(pid=87058) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:47:24.588040 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.fernet_utils [None req-35212ab9-97f1-4f51-960f-5b24f92921ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:47:24.713886 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 688/1359] 10.222.0.20 () {58 vars in 1059 bytes} [Mon Aug 1 16:47:24 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 218 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:47:24.721652 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c5df43-c8f9-413f-a1b1-5b91d2923ab2 None None] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:47:24.723199 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.fernet_utils [None req-17c5df43-c8f9-413f-a1b1-5b91d2923ab2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87057) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:47:24.882745 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c5df43-c8f9-413f-a1b1-5b91d2923ab2 None None] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:47:24.974754 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG dbcounter [-] [87057] Writing DB stats keystone:SELECT=100 {{(pid=87057) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:47:25.018177 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72902097-bc01-470b-9fce-c1b70a48e010 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6533106d03eb4ca185a8839336d2aff9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b992d6a65df4bf9b27c13192d57c5f0', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:47:25.018796 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72902097-bc01-470b-9fce-c1b70a48e010 service nova] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:47:25.019097 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72902097-bc01-470b-9fce-c1b70a48e010 service nova] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:47:25.019345 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72902097-bc01-470b-9fce-c1b70a48e010 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:47:25.020962 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.fernet_utils [None req-72902097-bc01-470b-9fce-c1b70a48e010 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87057) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:47:25.117259 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72902097-bc01-470b-9fce-c1b70a48e010 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:47:25.118714 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72902097-bc01-470b-9fce-c1b70a48e010 service nova] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:47:25.145924 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 672/1360] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 1 16:47:24 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 425 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 16:47:27.363334 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dbee9dd-dcda-45ea-82e9-69e9ae2a0fdc None None] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:47:27.363841 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dbee9dd-dcda-45ea-82e9-69e9ae2a0fdc None None] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:47:27.363841 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dbee9dd-dcda-45ea-82e9-69e9ae2a0fdc None None] PATH_INFO: `/` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:47:27.365329 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 689/1361] 10.222.0.20 () {54 vars in 926 bytes} [Mon Aug 1 16:47:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 16:47:27.370930 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e597c47-cafd-4014-a5d3-c9fb47aed818 None None] REQUEST_METHOD: `POST` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:47:27.370930 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e597c47-cafd-4014-a5d3-c9fb47aed818 None None] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:47:27.370930 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e597c47-cafd-4014-a5d3-c9fb47aed818 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:47:27.444108 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.auth.core [None req-0e597c47-cafd-4014-a5d3-c9fb47aed818 None None] MFA Rules not processed for user `1007f8e379b74e169b98fb7c3c0ba5b4`. Rule list: `[]` (Enabled: `True`). {{(pid=87057) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:47:27.445320 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.fernet_utils [None req-0e597c47-cafd-4014-a5d3-c9fb47aed818 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87057) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:47:27.486202 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG dbcounter [-] [87057] Writing DB stats keystone:SELECT=100 {{(pid=87057) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:47:27.549812 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 673/1362] 10.222.0.20 () {58 vars in 1023 bytes} [Mon Aug 1 16:47:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 16:47:27.595195 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dbee9dd-dcda-45ea-82e9-69e9ae2a0fdc None None] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:47:27.596712 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.fernet_utils [None req-7dbee9dd-dcda-45ea-82e9-69e9ae2a0fdc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:47:27.672476 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG dbcounter [-] [87058] Writing DB stats keystone:SELECT=100 {{(pid=87058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:47:27.769617 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dbee9dd-dcda-45ea-82e9-69e9ae2a0fdc None None] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:47:27.892698 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbde603b-314c-4aa5-bbb6-08b2a8f1f70b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6533106d03eb4ca185a8839336d2aff9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b992d6a65df4bf9b27c13192d57c5f0', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:47:27.893330 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbde603b-314c-4aa5-bbb6-08b2a8f1f70b service nova] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:47:27.893612 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbde603b-314c-4aa5-bbb6-08b2a8f1f70b service nova] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:47:27.893886 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbde603b-314c-4aa5-bbb6-08b2a8f1f70b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:47:27.895546 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.fernet_utils [None req-dbde603b-314c-4aa5-bbb6-08b2a8f1f70b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:47:27.926767 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG dbcounter [-] [87058] Writing DB stats keystone:SELECT=100 {{(pid=87058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:47:28.005949 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbde603b-314c-4aa5-bbb6-08b2a8f1f70b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:47:28.007695 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbde603b-314c-4aa5-bbb6-08b2a8f1f70b service nova] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:47:28.027282 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 690/1363] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 1 16:47:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 433 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 01 16:47:30.509451 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7013260c-6ee0-4506-911c-e301899695ff None None] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:47:30.509979 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7013260c-6ee0-4506-911c-e301899695ff None None] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:47:30.510170 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7013260c-6ee0-4506-911c-e301899695ff None None] PATH_INFO: `/` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:47:30.511630 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 674/1364] 10.222.0.20 () {54 vars in 926 bytes} [Mon Aug 1 16:47:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 16:47:30.517551 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b284a61a-d4cb-403a-ab9c-48a65f650aee None None] REQUEST_METHOD: `POST` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:47:30.518076 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b284a61a-d4cb-403a-ab9c-48a65f650aee None None] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:47:30.518272 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b284a61a-d4cb-403a-ab9c-48a65f650aee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:47:30.605894 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.auth.core [None req-b284a61a-d4cb-403a-ab9c-48a65f650aee None None] MFA Rules not processed for user `1007f8e379b74e169b98fb7c3c0ba5b4`. Rule list: `[]` (Enabled: `True`). {{(pid=87058) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:47:30.607725 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.fernet_utils [None req-b284a61a-d4cb-403a-ab9c-48a65f650aee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:47:30.711405 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG dbcounter [-] [87058] Writing DB stats keystone:SELECT=100 {{(pid=87058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:47:30.722120 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 691/1365] 10.222.0.20 () {58 vars in 1023 bytes} [Mon Aug 1 16:47:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:47:30.765339 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c5185b-f4b3-462c-b861-8f7e31443ffa None None] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:47:30.765339 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c5185b-f4b3-462c-b861-8f7e31443ffa None None] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:47:30.765339 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c5185b-f4b3-462c-b861-8f7e31443ffa None None] PATH_INFO: `/` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:47:30.765339 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 675/1366] 10.222.0.20 () {54 vars in 962 bytes} [Mon Aug 1 16:47:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 16:47:30.770551 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d427447-37e0-4366-a54f-6fede26357ad None None] REQUEST_METHOD: `POST` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:47:30.771062 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d427447-37e0-4366-a54f-6fede26357ad None None] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:47:30.771568 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d427447-37e0-4366-a54f-6fede26357ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:47:30.853976 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.auth.core [None req-7d427447-37e0-4366-a54f-6fede26357ad None None] MFA Rules not processed for user `6533106d03eb4ca185a8839336d2aff9`. Rule list: `[]` (Enabled: `True`). {{(pid=87058) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:47:30.855383 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.fernet_utils [None req-7d427447-37e0-4366-a54f-6fede26357ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:47:30.989039 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 692/1367] 10.222.0.20 () {58 vars in 1059 bytes} [Mon Aug 1 16:47:30 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 220 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:47:30.997666 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c5185b-f4b3-462c-b861-8f7e31443ffa None None] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:47:30.999997 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.fernet_utils [None req-02c5185b-f4b3-462c-b861-8f7e31443ffa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87057) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:47:31.158974 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c5185b-f4b3-462c-b861-8f7e31443ffa None None] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:47:31.182749 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG dbcounter [-] [87057] Writing DB stats keystone:SELECT=100 {{(pid=87057) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:47:31.283777 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd55097-5161-46ab-b8e2-dd4bc4e3b924 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6533106d03eb4ca185a8839336d2aff9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b992d6a65df4bf9b27c13192d57c5f0', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:47:31.284394 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd55097-5161-46ab-b8e2-dd4bc4e3b924 service nova] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:47:31.284667 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd55097-5161-46ab-b8e2-dd4bc4e3b924 service nova] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:47:31.284933 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd55097-5161-46ab-b8e2-dd4bc4e3b924 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:47:31.286571 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.fernet_utils [None req-0dd55097-5161-46ab-b8e2-dd4bc4e3b924 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87057) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:47:31.380982 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dd55097-5161-46ab-b8e2-dd4bc4e3b924 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:47:31.382657 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dd55097-5161-46ab-b8e2-dd4bc4e3b924 service nova] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:47:31.401278 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 676/1368] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 1 16:47:30 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 404 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 01 16:47:32.425915 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1410a4c6-52dd-410c-a79d-046a5618ccf3 None None] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:47:32.426495 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1410a4c6-52dd-410c-a79d-046a5618ccf3 None None] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:47:32.426495 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1410a4c6-52dd-410c-a79d-046a5618ccf3 None None] PATH_INFO: `/` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:47:32.428058 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 693/1369] 10.222.0.20 () {54 vars in 926 bytes} [Mon Aug 1 16:47:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 16:47:32.432958 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbea787c-410c-4824-9d70-a2796839491c None None] REQUEST_METHOD: `POST` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:47:32.433365 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbea787c-410c-4824-9d70-a2796839491c None None] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:47:32.433530 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbea787c-410c-4824-9d70-a2796839491c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:47:32.465257 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG dbcounter [-] [87057] Writing DB stats keystone:SELECT=100 {{(pid=87057) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:47:32.500076 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.auth.core [None req-fbea787c-410c-4824-9d70-a2796839491c None None] MFA Rules not processed for user `1007f8e379b74e169b98fb7c3c0ba5b4`. Rule list: `[]` (Enabled: `True`). {{(pid=87057) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:47:32.501377 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.fernet_utils [None req-fbea787c-410c-4824-9d70-a2796839491c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87057) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:47:32.603364 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 677/1370] 10.222.0.20 () {58 vars in 1023 bytes} [Mon Aug 1 16:47:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:47:32.949816 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60cbb50-0f54-42c3-a65b-0d894065abaf None None] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:47:32.949816 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60cbb50-0f54-42c3-a65b-0d894065abaf None None] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:47:32.949816 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60cbb50-0f54-42c3-a65b-0d894065abaf None None] PATH_INFO: `/` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:47:32.952105 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 694/1371] 10.222.0.20 () {54 vars in 964 bytes} [Mon Aug 1 16:47:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 16:47:32.955979 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c23fed0-ea3d-48e7-9f64-f1e00f2da701 None None] REQUEST_METHOD: `POST` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:47:32.956269 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c23fed0-ea3d-48e7-9f64-f1e00f2da701 None None] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:47:32.956544 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c23fed0-ea3d-48e7-9f64-f1e00f2da701 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:47:33.036836 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.auth.core [None req-6c23fed0-ea3d-48e7-9f64-f1e00f2da701 None None] MFA Rules not processed for user `25970b52ca15448fa1abb1f974d64038`. Rule list: `[]` (Enabled: `True`). {{(pid=87057) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:47:33.038089 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.fernet_utils [None req-6c23fed0-ea3d-48e7-9f64-f1e00f2da701 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87057) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:47:33.087614 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG dbcounter [-] [87057] Writing DB stats keystone:SELECT=100 {{(pid=87057) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:47:33.120336 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 678/1372] 10.222.0.20 () {58 vars in 1061 bytes} [Mon Aug 1 16:47:32 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:47:33.127954 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d60cbb50-0f54-42c3-a65b-0d894065abaf None None] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:47:33.129761 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.fernet_utils [None req-d60cbb50-0f54-42c3-a65b-0d894065abaf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:47:33.190347 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG dbcounter [-] [87058] Writing DB stats keystone:SELECT=100 {{(pid=87058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:47:33.251782 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d60cbb50-0f54-42c3-a65b-0d894065abaf None None] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:47:33.333942 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9a6995f-aec8-4aac-ae9d-10a82a80f31f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25970b52ca15448fa1abb1f974d64038', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b992d6a65df4bf9b27c13192d57c5f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:47:33.335001 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a6995f-aec8-4aac-ae9d-10a82a80f31f service cinder] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:47:33.335682 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a6995f-aec8-4aac-ae9d-10a82a80f31f service cinder] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:47:33.336116 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a6995f-aec8-4aac-ae9d-10a82a80f31f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:47:33.337913 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.fernet_utils [None req-d9a6995f-aec8-4aac-ae9d-10a82a80f31f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:47:33.447292 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG dbcounter [-] [87058] Writing DB stats keystone:SELECT=100 {{(pid=87058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:47:33.462572 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9a6995f-aec8-4aac-ae9d-10a82a80f31f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:47:33.464379 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9a6995f-aec8-4aac-ae9d-10a82a80f31f service cinder] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:47:33.485446 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 695/1373] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 1 16:47:33 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 358 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 16:47:33.649854 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b2226f-fb66-4cef-a907-cde0f6b43a29 None None] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:47:33.650498 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b2226f-fb66-4cef-a907-cde0f6b43a29 None None] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:47:33.650498 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b2226f-fb66-4cef-a907-cde0f6b43a29 None None] PATH_INFO: `/` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:47:33.652203 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 679/1374] 10.222.0.20 () {54 vars in 926 bytes} [Mon Aug 1 16:47:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 16:47:33.657343 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c24b046-69ba-4654-96ff-c38aafab06f1 None None] REQUEST_METHOD: `POST` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:47:33.657516 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c24b046-69ba-4654-96ff-c38aafab06f1 None None] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:47:33.657830 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c24b046-69ba-4654-96ff-c38aafab06f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:47:33.733684 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.auth.core [None req-0c24b046-69ba-4654-96ff-c38aafab06f1 None None] MFA Rules not processed for user `1007f8e379b74e169b98fb7c3c0ba5b4`. Rule list: `[]` (Enabled: `True`). {{(pid=87058) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:47:33.734948 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.fernet_utils [None req-0c24b046-69ba-4654-96ff-c38aafab06f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:47:33.862443 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 696/1375] 10.222.0.20 () {58 vars in 1023 bytes} [Mon Aug 1 16:47:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 206 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:47:33.896276 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b2226f-fb66-4cef-a907-cde0f6b43a29 None None] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:47:33.900722 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.fernet_utils [None req-e5b2226f-fb66-4cef-a907-cde0f6b43a29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87057) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:47:34.051489 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b2226f-fb66-4cef-a907-cde0f6b43a29 None None] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:47:34.101715 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG dbcounter [-] [87057] Writing DB stats keystone:SELECT=100 {{(pid=87057) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:47:34.164446 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee0c0dd5-7d54-409e-8e43-693cade2f87e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6533106d03eb4ca185a8839336d2aff9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b992d6a65df4bf9b27c13192d57c5f0', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:47:34.165058 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0c0dd5-7d54-409e-8e43-693cade2f87e service nova] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:47:34.165333 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0c0dd5-7d54-409e-8e43-693cade2f87e service nova] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:47:34.165598 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0c0dd5-7d54-409e-8e43-693cade2f87e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:47:34.167296 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.fernet_utils [None req-ee0c0dd5-7d54-409e-8e43-693cade2f87e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87057) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:47:34.258003 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee0c0dd5-7d54-409e-8e43-693cade2f87e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:47:34.259655 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee0c0dd5-7d54-409e-8e43-693cade2f87e service nova] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:47:34.277585 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 680/1376] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 1 16:47:33 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 382 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 16:47:36.387925 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27bd1144-f923-4587-ac1c-32c270890c80 None None] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:47:36.389271 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27bd1144-f923-4587-ac1c-32c270890c80 None None] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:47:36.389271 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27bd1144-f923-4587-ac1c-32c270890c80 None None] PATH_INFO: `/` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:47:36.390963 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 697/1377] 10.222.0.20 () {54 vars in 926 bytes} [Mon Aug 1 16:47:36 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 16:47:36.395998 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806834f0-7b11-4d17-bf05-123de72da6cc None None] REQUEST_METHOD: `POST` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:47:36.396417 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806834f0-7b11-4d17-bf05-123de72da6cc None None] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:47:36.396550 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806834f0-7b11-4d17-bf05-123de72da6cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:47:36.463842 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.auth.core [None req-806834f0-7b11-4d17-bf05-123de72da6cc None None] MFA Rules not processed for user `1007f8e379b74e169b98fb7c3c0ba5b4`. Rule list: `[]` (Enabled: `True`). {{(pid=87057) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:47:36.467342 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.fernet_utils [None req-806834f0-7b11-4d17-bf05-123de72da6cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87057) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:47:36.477645 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG dbcounter [-] [87057] Writing DB stats keystone:SELECT=100 {{(pid=87057) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:47:36.587290 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 681/1378] 10.222.0.20 () {58 vars in 1023 bytes} [Mon Aug 1 16:47:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:47:36.632354 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27bd1144-f923-4587-ac1c-32c270890c80 None None] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:47:36.633855 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.fernet_utils [None req-27bd1144-f923-4587-ac1c-32c270890c80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:47:36.694467 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG dbcounter [-] [87058] Writing DB stats keystone:SELECT=100 {{(pid=87058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:47:36.784195 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27bd1144-f923-4587-ac1c-32c270890c80 None None] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:47:36.906330 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da62572-982d-485c-9413-ab22a2619182 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6533106d03eb4ca185a8839336d2aff9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b992d6a65df4bf9b27c13192d57c5f0', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:47:36.908706 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da62572-982d-485c-9413-ab22a2619182 service nova] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:47:36.908706 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da62572-982d-485c-9413-ab22a2619182 service nova] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:47:36.908706 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da62572-982d-485c-9413-ab22a2619182 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:47:36.909200 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.fernet_utils [None req-6da62572-982d-485c-9413-ab22a2619182 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:47:36.940086 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG dbcounter [-] [87058] Writing DB stats keystone:SELECT=100 {{(pid=87058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:47:37.031013 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6da62572-982d-485c-9413-ab22a2619182 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:47:37.032643 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6da62572-982d-485c-9413-ab22a2619182 service nova] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:47:37.053855 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 698/1379] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 1 16:47:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 422 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 16:47:39.240827 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83daa8f-7fb0-4b10-8d58-3c8173abf2b1 None None] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:47:39.241398 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83daa8f-7fb0-4b10-8d58-3c8173abf2b1 None None] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:47:39.241398 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83daa8f-7fb0-4b10-8d58-3c8173abf2b1 None None] PATH_INFO: `/` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:47:39.242911 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 682/1380] 10.222.0.20 () {54 vars in 926 bytes} [Mon Aug 1 16:47:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 01 16:47:39.248771 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4615c90-7afa-4367-8614-3a07090c852d None None] REQUEST_METHOD: `POST` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:47:39.249538 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4615c90-7afa-4367-8614-3a07090c852d None None] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:47:39.249869 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4615c90-7afa-4367-8614-3a07090c852d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:47:39.321575 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.auth.core [None req-c4615c90-7afa-4367-8614-3a07090c852d None None] MFA Rules not processed for user `1007f8e379b74e169b98fb7c3c0ba5b4`. Rule list: `[]` (Enabled: `True`). {{(pid=87058) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:47:39.322821 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.fernet_utils [None req-c4615c90-7afa-4367-8614-3a07090c852d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:47:39.426897 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG dbcounter [-] [87058] Writing DB stats keystone:SELECT=100 {{(pid=87058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:47:39.433222 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 699/1381] 10.222.0.20 () {58 vars in 1023 bytes} [Mon Aug 1 16:47:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:47:39.477539 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f83daa8f-7fb0-4b10-8d58-3c8173abf2b1 None None] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:47:39.479325 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.fernet_utils [None req-f83daa8f-7fb0-4b10-8d58-3c8173abf2b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87057) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:47:39.624480 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG dbcounter [-] [87057] Writing DB stats keystone:SELECT=100 {{(pid=87057) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:47:39.635791 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f83daa8f-7fb0-4b10-8d58-3c8173abf2b1 None None] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:47:39.757483 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4fdffa3-e874-4819-90aa-0978fa4e4cb4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6533106d03eb4ca185a8839336d2aff9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b992d6a65df4bf9b27c13192d57c5f0', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:47:39.758129 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4fdffa3-e874-4819-90aa-0978fa4e4cb4 service nova] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:47:39.758410 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4fdffa3-e874-4819-90aa-0978fa4e4cb4 service nova] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:47:39.758682 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4fdffa3-e874-4819-90aa-0978fa4e4cb4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:47:39.760304 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.fernet_utils [None req-d4fdffa3-e874-4819-90aa-0978fa4e4cb4 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87057) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:47:39.858533 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4fdffa3-e874-4819-90aa-0978fa4e4cb4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:47:39.860271 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4fdffa3-e874-4819-90aa-0978fa4e4cb4 service nova] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:47:39.895146 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG dbcounter [-] [87057] Writing DB stats keystone:SELECT=100 {{(pid=87057) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:47:39.895402 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 683/1382] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 1 16:47:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 418 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 16:47:42.020637 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541d2651-e747-497e-8cb7-40b4ffa35bb6 None None] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:47:42.021172 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541d2651-e747-497e-8cb7-40b4ffa35bb6 None None] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:47:42.021172 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541d2651-e747-497e-8cb7-40b4ffa35bb6 None None] PATH_INFO: `/` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:47:42.022601 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 700/1383] 10.222.0.20 () {54 vars in 926 bytes} [Mon Aug 1 16:47:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 16:47:42.028079 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4a048b-fa40-4b4b-8be5-69a9ecfe259b None None] REQUEST_METHOD: `POST` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:47:42.028491 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4a048b-fa40-4b4b-8be5-69a9ecfe259b None None] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:47:42.028720 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4a048b-fa40-4b4b-8be5-69a9ecfe259b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:47:42.103100 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.auth.core [None req-fd4a048b-fa40-4b4b-8be5-69a9ecfe259b None None] MFA Rules not processed for user `1007f8e379b74e169b98fb7c3c0ba5b4`. Rule list: `[]` (Enabled: `True`). {{(pid=87057) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:47:42.104976 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.fernet_utils [None req-fd4a048b-fa40-4b4b-8be5-69a9ecfe259b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87057) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:47:42.231829 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 684/1384] 10.222.0.20 () {58 vars in 1023 bytes} [Mon Aug 1 16:47:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:47:42.273978 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-541d2651-e747-497e-8cb7-40b4ffa35bb6 None None] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:47:42.275526 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.fernet_utils [None req-541d2651-e747-497e-8cb7-40b4ffa35bb6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:47:42.436164 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-541d2651-e747-497e-8cb7-40b4ffa35bb6 None None] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:47:42.504791 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG dbcounter [-] [87058] Writing DB stats keystone:SELECT=100 {{(pid=87058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:47:42.707026 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-070a26eb-bbdf-4eea-9f54-7fd753f89050 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6533106d03eb4ca185a8839336d2aff9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b992d6a65df4bf9b27c13192d57c5f0', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:47:42.707786 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070a26eb-bbdf-4eea-9f54-7fd753f89050 service nova] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:47:42.708138 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070a26eb-bbdf-4eea-9f54-7fd753f89050 service nova] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:47:42.708754 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070a26eb-bbdf-4eea-9f54-7fd753f89050 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:47:42.710929 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.fernet_utils [None req-070a26eb-bbdf-4eea-9f54-7fd753f89050 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:47:42.815331 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-070a26eb-bbdf-4eea-9f54-7fd753f89050 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:47:42.816867 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-070a26eb-bbdf-4eea-9f54-7fd753f89050 service nova] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:47:42.835928 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 701/1385] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 1 16:47:42 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 562 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 16:47:44.879987 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c127c1c-95ff-4a51-a5ea-de8daf7b240a None None] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:47:44.881071 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c127c1c-95ff-4a51-a5ea-de8daf7b240a None None] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:47:44.881071 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c127c1c-95ff-4a51-a5ea-de8daf7b240a None None] PATH_INFO: `/` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:47:44.882187 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 685/1386] 10.222.0.20 () {54 vars in 926 bytes} [Mon Aug 1 16:47:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 16:47:44.889145 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e149bd16-3f6f-4c57-9eb8-b08fc725f823 None None] REQUEST_METHOD: `POST` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:47:44.889827 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e149bd16-3f6f-4c57-9eb8-b08fc725f823 None None] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:47:44.889827 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e149bd16-3f6f-4c57-9eb8-b08fc725f823 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:47:44.961232 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.auth.core [None req-e149bd16-3f6f-4c57-9eb8-b08fc725f823 None None] MFA Rules not processed for user `1007f8e379b74e169b98fb7c3c0ba5b4`. Rule list: `[]` (Enabled: `True`). {{(pid=87058) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:47:44.962502 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.fernet_utils [None req-e149bd16-3f6f-4c57-9eb8-b08fc725f823 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:47:44.997134 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG dbcounter [-] [87058] Writing DB stats keystone:SELECT=100 {{(pid=87058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:47:45.068824 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 702/1387] 10.222.0.20 () {58 vars in 1023 bytes} [Mon Aug 1 16:47:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:47:45.123658 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c127c1c-95ff-4a51-a5ea-de8daf7b240a None None] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:47:45.125274 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.fernet_utils [None req-4c127c1c-95ff-4a51-a5ea-de8daf7b240a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87057) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:47:45.215643 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG dbcounter [-] [87057] Writing DB stats keystone:SELECT=100 {{(pid=87057) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:47:45.291152 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c127c1c-95ff-4a51-a5ea-de8daf7b240a None None] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:47:45.411746 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb3da47-7bde-4211-8dfb-4c0644f7b17c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6533106d03eb4ca185a8839336d2aff9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b992d6a65df4bf9b27c13192d57c5f0', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:47:45.412374 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb3da47-7bde-4211-8dfb-4c0644f7b17c service nova] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:47:45.412654 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb3da47-7bde-4211-8dfb-4c0644f7b17c service nova] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:47:45.412927 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb3da47-7bde-4211-8dfb-4c0644f7b17c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:47:45.414581 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.fernet_utils [None req-7cb3da47-7bde-4211-8dfb-4c0644f7b17c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87057) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:47:45.453608 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG dbcounter [-] [87057] Writing DB stats keystone:SELECT=100 {{(pid=87057) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:47:45.509712 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cb3da47-7bde-4211-8dfb-4c0644f7b17c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:47:45.511742 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cb3da47-7bde-4211-8dfb-4c0644f7b17c service nova] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:47:45.531592 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 686/1388] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 1 16:47:45 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 409 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 01 16:47:47.683243 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3207ad-76f6-4a2b-8353-8e98ed94aeb3 None None] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:47:47.687216 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3207ad-76f6-4a2b-8353-8e98ed94aeb3 None None] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:47:47.687216 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3207ad-76f6-4a2b-8353-8e98ed94aeb3 None None] PATH_INFO: `/` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:47:47.687216 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 703/1389] 10.222.0.20 () {54 vars in 926 bytes} [Mon Aug 1 16:47:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 16:47:47.690911 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d126d8-9c4b-478b-a34d-2ea482ee7e85 None None] REQUEST_METHOD: `POST` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:47:47.691336 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d126d8-9c4b-478b-a34d-2ea482ee7e85 None None] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:47:47.691470 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d126d8-9c4b-478b-a34d-2ea482ee7e85 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:47:47.759569 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.auth.core [None req-18d126d8-9c4b-478b-a34d-2ea482ee7e85 None None] MFA Rules not processed for user `1007f8e379b74e169b98fb7c3c0ba5b4`. Rule list: `[]` (Enabled: `True`). {{(pid=87057) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:47:47.760790 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.fernet_utils [None req-18d126d8-9c4b-478b-a34d-2ea482ee7e85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87057) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:47:47.858721 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 687/1390] 10.222.0.20 () {58 vars in 1023 bytes} [Mon Aug 1 16:47:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:47:47.902960 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c3207ad-76f6-4a2b-8353-8e98ed94aeb3 None None] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:47:47.905247 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.fernet_utils [None req-6c3207ad-76f6-4a2b-8353-8e98ed94aeb3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:47:48.055809 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c3207ad-76f6-4a2b-8353-8e98ed94aeb3 None None] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:47:48.072013 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG dbcounter [-] [87058] Writing DB stats keystone:SELECT=100 {{(pid=87058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:47:48.171620 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab9d00c8-3765-4ef2-8a71-70ad9810e843 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6533106d03eb4ca185a8839336d2aff9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b992d6a65df4bf9b27c13192d57c5f0', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:47:48.172234 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9d00c8-3765-4ef2-8a71-70ad9810e843 service nova] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:47:48.172509 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9d00c8-3765-4ef2-8a71-70ad9810e843 service nova] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:47:48.172775 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9d00c8-3765-4ef2-8a71-70ad9810e843 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:47:48.174428 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.fernet_utils [None req-ab9d00c8-3765-4ef2-8a71-70ad9810e843 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:47:48.264824 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab9d00c8-3765-4ef2-8a71-70ad9810e843 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:47:48.266522 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab9d00c8-3765-4ef2-8a71-70ad9810e843 service nova] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:47:48.285233 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 704/1391] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 1 16:47:47 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 383 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 16:47:48.861078 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522b8042-7da0-41e8-b235-b4e11d72ff43 None None] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:47:48.861656 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522b8042-7da0-41e8-b235-b4e11d72ff43 None None] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:47:48.861656 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522b8042-7da0-41e8-b235-b4e11d72ff43 None None] PATH_INFO: `/` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:47:48.863068 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 688/1392] 10.222.0.20 () {54 vars in 926 bytes} [Mon Aug 1 16:47:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 16:47:48.868364 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733df21d-c0e2-41b8-86df-137321a28691 None None] REQUEST_METHOD: `POST` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:47:48.868808 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733df21d-c0e2-41b8-86df-137321a28691 None None] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:47:48.868931 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733df21d-c0e2-41b8-86df-137321a28691 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:47:48.895526 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG dbcounter [-] [87058] Writing DB stats keystone:SELECT=100 {{(pid=87058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:47:48.938697 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.auth.core [None req-733df21d-c0e2-41b8-86df-137321a28691 None None] MFA Rules not processed for user `1007f8e379b74e169b98fb7c3c0ba5b4`. Rule list: `[]` (Enabled: `True`). {{(pid=87058) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:47:48.939918 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.fernet_utils [None req-733df21d-c0e2-41b8-86df-137321a28691 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:47:49.035070 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 705/1393] 10.222.0.20 () {58 vars in 1023 bytes} [Mon Aug 1 16:47:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:47:49.091976 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8e36b1-2590-499a-9c46-b3e30f5e00b6 None None] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:47:49.092265 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8e36b1-2590-499a-9c46-b3e30f5e00b6 None None] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:47:49.092541 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8e36b1-2590-499a-9c46-b3e30f5e00b6 None None] PATH_INFO: `/` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:47:49.093916 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 689/1394] 10.222.0.20 () {54 vars in 966 bytes} [Mon Aug 1 16:47:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 16:47:49.098898 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9899285a-4eff-4542-bfea-fcb4a94a7d57 None None] REQUEST_METHOD: `POST` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:47:49.099188 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9899285a-4eff-4542-bfea-fcb4a94a7d57 None None] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:47:49.099460 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9899285a-4eff-4542-bfea-fcb4a94a7d57 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:47:49.177283 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.auth.core [None req-9899285a-4eff-4542-bfea-fcb4a94a7d57 None None] MFA Rules not processed for user `d80694e439e2401d9491712666409016`. Rule list: `[]` (Enabled: `True`). {{(pid=87058) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:47:49.178443 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.fernet_utils [None req-9899285a-4eff-4542-bfea-fcb4a94a7d57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:47:49.218943 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG dbcounter [-] [87058] Writing DB stats keystone:SELECT=100 {{(pid=87058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:47:49.256245 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 706/1395] 10.222.0.20 () {58 vars in 1063 bytes} [Mon Aug 1 16:47:49 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:47:49.263902 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b8e36b1-2590-499a-9c46-b3e30f5e00b6 None None] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:47:49.265332 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.fernet_utils [None req-9b8e36b1-2590-499a-9c46-b3e30f5e00b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87057) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:47:49.279577 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG dbcounter [-] [87057] Writing DB stats keystone:SELECT=100 {{(pid=87057) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:47:49.368965 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b8e36b1-2590-499a-9c46-b3e30f5e00b6 None None] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:47:49.443567 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e72e2b-ace4-4be9-9836-60bd5159380f service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd80694e439e2401d9491712666409016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b992d6a65df4bf9b27c13192d57c5f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:47:49.444098 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e72e2b-ace4-4be9-9836-60bd5159380f service glance] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:47:49.444373 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e72e2b-ace4-4be9-9836-60bd5159380f service glance] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:47:49.444644 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e72e2b-ace4-4be9-9836-60bd5159380f service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:47:49.446289 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.fernet_utils [None req-c9e72e2b-ace4-4be9-9836-60bd5159380f service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87057) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:47:49.515742 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG dbcounter [-] [87057] Writing DB stats keystone:SELECT=100 {{(pid=87057) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:47:49.559817 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9e72e2b-ace4-4be9-9836-60bd5159380f service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:47:49.561732 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9e72e2b-ace4-4be9-9836-60bd5159380f service glance] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:47:49.580301 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 690/1396] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 1 16:47:49 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 01 16:47:50.463712 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5e5afa-7e7c-41db-b75f-a986e8f50c81 None None] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:47:50.464263 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5e5afa-7e7c-41db-b75f-a986e8f50c81 None None] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:47:50.464263 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5e5afa-7e7c-41db-b75f-a986e8f50c81 None None] PATH_INFO: `/` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:47:50.465624 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 707/1397] 10.222.0.20 () {54 vars in 926 bytes} [Mon Aug 1 16:47:50 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 16:47:50.471841 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6a9c12-b18b-4dce-b56f-94a7ba4bcec1 None None] REQUEST_METHOD: `POST` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:47:50.472381 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6a9c12-b18b-4dce-b56f-94a7ba4bcec1 None None] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:47:50.472381 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6a9c12-b18b-4dce-b56f-94a7ba4bcec1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:47:50.503073 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5a9dd7-d970-438c-8bd7-50ce378f1dcc None None] REQUEST_METHOD: `POST` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:47:50.505995 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5a9dd7-d970-438c-8bd7-50ce378f1dcc None None] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:47:50.505995 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5a9dd7-d970-438c-8bd7-50ce378f1dcc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:47:50.542438 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.auth.core [None req-6d6a9c12-b18b-4dce-b56f-94a7ba4bcec1 None None] MFA Rules not processed for user `1007f8e379b74e169b98fb7c3c0ba5b4`. Rule list: `[]` (Enabled: `True`). {{(pid=87057) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:47:50.543675 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.fernet_utils [None req-6d6a9c12-b18b-4dce-b56f-94a7ba4bcec1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87057) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:47:50.574124 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.auth.core [None req-8a5a9dd7-d970-438c-8bd7-50ce378f1dcc None None] MFA Rules not processed for user `b6a93d1c6c82489099adc8b413280fc0`. Rule list: `[]` (Enabled: `True`). {{(pid=87058) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:47:50.576406 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.fernet_utils [None req-8a5a9dd7-d970-438c-8bd7-50ce378f1dcc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:47:50.645303 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 691/1398] 10.222.0.20 () {58 vars in 1023 bytes} [Mon Aug 1 16:47:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:47:50.667062 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 708/1399] 10.222.0.20 () {58 vars in 1009 bytes} [Mon Aug 1 16:47:50 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 16:47:50.690376 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d6a9c12-b18b-4dce-b56f-94a7ba4bcec1 None None] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:47:50.691607 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.fernet_utils [None req-6d6a9c12-b18b-4dce-b56f-94a7ba4bcec1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87057) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:47:50.711807 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5846ac77-8735-4919-b2d0-0ba75b3bc58d None None] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:47:50.712380 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5846ac77-8735-4919-b2d0-0ba75b3bc58d None None] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:47:50.712380 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5846ac77-8735-4919-b2d0-0ba75b3bc58d None None] PATH_INFO: `/` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:47:50.713843 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 709/1400] 10.222.0.20 () {54 vars in 943 bytes} [Mon Aug 1 16:47:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 01 16:47:50.720043 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477cebde-2ff4-4f3d-85b7-606f720d740f None None] REQUEST_METHOD: `POST` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:47:50.720396 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477cebde-2ff4-4f3d-85b7-606f720d740f None None] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:47:50.720795 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG dbcounter [-] [87057] Writing DB stats keystone:SELECT=100 {{(pid=87057) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:47:50.720898 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477cebde-2ff4-4f3d-85b7-606f720d740f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:47:50.791813 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG dbcounter [-] [87058] Writing DB stats keystone:SELECT=100 {{(pid=87058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:47:50.799946 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.auth.core [None req-477cebde-2ff4-4f3d-85b7-606f720d740f None None] MFA Rules not processed for user `9ab4add1faa04faa811ef7a9341cc35c`. Rule list: `[]` (Enabled: `True`). {{(pid=87058) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:47:50.801199 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.fernet_utils [None req-477cebde-2ff4-4f3d-85b7-606f720d740f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:47:50.814124 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d6a9c12-b18b-4dce-b56f-94a7ba4bcec1 None None] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:47:50.910011 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 710/1401] 10.222.0.20 () {58 vars in 1040 bytes} [Mon Aug 1 16:47:50 2022] POST /identity/v3/auth/tokens => generated 3294 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 16:47:50.919699 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-477cebde-2ff4-4f3d-85b7-606f720d740f None None] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:47:50.929395 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-477cebde-2ff4-4f3d-85b7-606f720d740f None None] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:47:50.934160 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35040ec9-851a-4d87-acd1-e4c45f7e62c2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6533106d03eb4ca185a8839336d2aff9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b992d6a65df4bf9b27c13192d57c5f0', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:47:50.935453 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35040ec9-851a-4d87-acd1-e4c45f7e62c2 service nova] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:47:50.935740 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35040ec9-851a-4d87-acd1-e4c45f7e62c2 service nova] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:47:50.936008 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35040ec9-851a-4d87-acd1-e4c45f7e62c2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:47:50.937619 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.fernet_utils [None req-35040ec9-851a-4d87-acd1-e4c45f7e62c2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87057) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:47:50.955134 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG dbcounter [-] [87057] Writing DB stats keystone:SELECT=100 {{(pid=87057) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:47:51.027781 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35040ec9-851a-4d87-acd1-e4c45f7e62c2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:47:51.029386 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35040ec9-851a-4d87-acd1-e4c45f7e62c2 service nova] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:47:51.035327 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1864871-5a3f-4097-b647-618e42a2f85a service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ab4add1faa04faa811ef7a9341cc35c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b992d6a65df4bf9b27c13192d57c5f0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:47:51.035951 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1864871-5a3f-4097-b647-618e42a2f85a service swift] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:47:51.036228 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1864871-5a3f-4097-b647-618e42a2f85a service swift] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:47:51.036497 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1864871-5a3f-4097-b647-618e42a2f85a service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:47:51.038201 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.fernet_utils [None req-d1864871-5a3f-4097-b647-618e42a2f85a service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:47:51.040694 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG dbcounter [-] [87058] Writing DB stats keystone:SELECT=100 {{(pid=87058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:47:51.048932 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 692/1402] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 1 16:47:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 359 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 16:47:51.124262 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1864871-5a3f-4097-b647-618e42a2f85a service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:47:51.125861 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1864871-5a3f-4097-b647-618e42a2f85a service swift] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:47:51.134530 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 711/1403] 10.222.0.20 () {60 vars in 1384 bytes} [Mon Aug 1 16:47:50 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 216 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Aug 01 16:47:53.224868 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5a0699-e413-4ed6-a425-990b03857aaa None None] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:47:53.225381 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5a0699-e413-4ed6-a425-990b03857aaa None None] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:47:53.225381 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5a0699-e413-4ed6-a425-990b03857aaa None None] PATH_INFO: `/` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:47:53.226723 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 693/1404] 10.222.0.20 () {54 vars in 926 bytes} [Mon Aug 1 16:47:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 16:47:53.232776 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17b56b0-5fa4-4571-9c87-1350cd8683c9 None None] REQUEST_METHOD: `POST` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:47:53.233633 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17b56b0-5fa4-4571-9c87-1350cd8683c9 None None] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:47:53.234094 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17b56b0-5fa4-4571-9c87-1350cd8683c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:47:53.308577 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.auth.core [None req-e17b56b0-5fa4-4571-9c87-1350cd8683c9 None None] MFA Rules not processed for user `1007f8e379b74e169b98fb7c3c0ba5b4`. Rule list: `[]` (Enabled: `True`). {{(pid=87058) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:47:53.309950 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.fernet_utils [None req-e17b56b0-5fa4-4571-9c87-1350cd8683c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:47:53.417038 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG dbcounter [-] [87058] Writing DB stats keystone:SELECT=100 {{(pid=87058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:47:53.426073 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 712/1405] 10.222.0.20 () {58 vars in 1023 bytes} [Mon Aug 1 16:47:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:47:53.470893 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f5a0699-e413-4ed6-a425-990b03857aaa None None] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:47:53.472500 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.fernet_utils [None req-6f5a0699-e413-4ed6-a425-990b03857aaa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87057) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:47:53.646340 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG dbcounter [-] [87057] Writing DB stats keystone:SELECT=100 {{(pid=87057) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:47:53.648991 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f5a0699-e413-4ed6-a425-990b03857aaa None None] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:47:53.781255 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-126b1083-1ea8-460d-8e13-32dc9e7f1ed4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6533106d03eb4ca185a8839336d2aff9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b992d6a65df4bf9b27c13192d57c5f0', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:47:53.781888 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126b1083-1ea8-460d-8e13-32dc9e7f1ed4 service nova] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:47:53.782201 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126b1083-1ea8-460d-8e13-32dc9e7f1ed4 service nova] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:47:53.782572 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126b1083-1ea8-460d-8e13-32dc9e7f1ed4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:47:53.784147 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.fernet_utils [None req-126b1083-1ea8-460d-8e13-32dc9e7f1ed4 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87057) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:47:53.901194 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-126b1083-1ea8-460d-8e13-32dc9e7f1ed4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:47:53.905440 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-126b1083-1ea8-460d-8e13-32dc9e7f1ed4 service nova] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:47:53.926063 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 694/1406] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 1 16:47:53 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 456 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 16:47:56.873890 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7764973a-4f6f-4090-a918-722e73214684 None None] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:47:56.874549 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7764973a-4f6f-4090-a918-722e73214684 None None] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:47:56.874549 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7764973a-4f6f-4090-a918-722e73214684 None None] PATH_INFO: `/` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:47:56.875998 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 713/1407] 10.222.0.20 () {54 vars in 926 bytes} [Mon Aug 1 16:47:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 16:47:56.881259 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e77cb03-59a5-4cee-9508-0ba6a6d99ee0 None None] REQUEST_METHOD: `POST` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:47:56.881683 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e77cb03-59a5-4cee-9508-0ba6a6d99ee0 None None] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:47:56.881813 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e77cb03-59a5-4cee-9508-0ba6a6d99ee0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:47:56.886400 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG dbcounter [-] [87057] Writing DB stats keystone:SELECT=100 {{(pid=87057) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:47:56.956647 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.auth.core [None req-2e77cb03-59a5-4cee-9508-0ba6a6d99ee0 None None] MFA Rules not processed for user `1007f8e379b74e169b98fb7c3c0ba5b4`. Rule list: `[]` (Enabled: `True`). {{(pid=87057) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:47:56.957973 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.fernet_utils [None req-2e77cb03-59a5-4cee-9508-0ba6a6d99ee0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87057) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:47:57.074093 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 695/1408] 10.222.0.20 () {58 vars in 1023 bytes} [Mon Aug 1 16:47:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:47:57.119101 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7764973a-4f6f-4090-a918-722e73214684 None None] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:47:57.120714 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.fernet_utils [None req-7764973a-4f6f-4090-a918-722e73214684 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:47:57.291645 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7764973a-4f6f-4090-a918-722e73214684 None None] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:47:57.371840 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG dbcounter [-] [87058] Writing DB stats keystone:SELECT=100 {{(pid=87058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:47:57.425659 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-693e840d-ee9f-46e7-b8d2-92f7e2f7b512 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6533106d03eb4ca185a8839336d2aff9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b992d6a65df4bf9b27c13192d57c5f0', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:47:57.426433 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693e840d-ee9f-46e7-b8d2-92f7e2f7b512 service nova] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:47:57.426783 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693e840d-ee9f-46e7-b8d2-92f7e2f7b512 service nova] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:47:57.427124 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693e840d-ee9f-46e7-b8d2-92f7e2f7b512 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:47:57.429006 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.fernet_utils [None req-693e840d-ee9f-46e7-b8d2-92f7e2f7b512 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:47:57.533721 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-693e840d-ee9f-46e7-b8d2-92f7e2f7b512 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:47:57.535459 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-693e840d-ee9f-46e7-b8d2-92f7e2f7b512 service nova] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:47:57.555763 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 714/1409] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 1 16:47:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 437 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 16:48:07.075454 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG dbcounter [-] [87057] Writing DB stats keystone:SELECT=64 {{(pid=87057) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:48:07.552658 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG dbcounter [-] [87058] Writing DB stats keystone:SELECT=64 {{(pid=87058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:48:14.443007 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac9fac4-0463-4a73-9e29-8b48955b1a52 None None] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:48:14.443598 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac9fac4-0463-4a73-9e29-8b48955b1a52 None None] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:48:14.443598 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac9fac4-0463-4a73-9e29-8b48955b1a52 None None] PATH_INFO: `/` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:48:14.444902 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 696/1410] 10.222.0.20 () {54 vars in 918 bytes} [Mon Aug 1 16:48:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 16:48:14.450089 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acbfc29c-2ffa-4938-852b-3f792b3f554b None None] REQUEST_METHOD: `POST` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:48:14.451275 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acbfc29c-2ffa-4938-852b-3f792b3f554b None None] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:48:14.451468 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acbfc29c-2ffa-4938-852b-3f792b3f554b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:48:14.526540 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.auth.core [None req-acbfc29c-2ffa-4938-852b-3f792b3f554b None None] MFA Rules not processed for user `8362293261614623bcdabc4120a97a9d`. Rule list: `[]` (Enabled: `True`). {{(pid=87058) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:48:14.527748 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.fernet_utils [None req-acbfc29c-2ffa-4938-852b-3f792b3f554b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:48:14.633460 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 715/1411] 10.222.0.20 () {58 vars in 1015 bytes} [Mon Aug 1 16:48:14 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:48:14.647987 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9d3839-db4a-4e93-bacf-afeeb476c501 None None] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:48:14.648277 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9d3839-db4a-4e93-bacf-afeeb476c501 None None] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:48:14.648553 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9d3839-db4a-4e93-bacf-afeeb476c501 None None] PATH_INFO: `/` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:48:14.649890 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 697/1412] 10.222.0.20 () {54 vars in 918 bytes} [Mon Aug 1 16:48:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 16:48:14.655433 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5ba264-007f-4950-87b0-9391790fddff None None] REQUEST_METHOD: `POST` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:48:14.655711 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5ba264-007f-4950-87b0-9391790fddff None None] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:48:14.655982 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5ba264-007f-4950-87b0-9391790fddff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:48:14.728638 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG dbcounter [-] [87058] Writing DB stats keystone:SELECT=100 {{(pid=87058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:48:14.731270 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.auth.core [None req-9c5ba264-007f-4950-87b0-9391790fddff None None] MFA Rules not processed for user `6533106d03eb4ca185a8839336d2aff9`. Rule list: `[]` (Enabled: `True`). {{(pid=87058) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:48:14.733064 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.fernet_utils [None req-9c5ba264-007f-4950-87b0-9391790fddff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:48:14.847843 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 716/1413] 10.222.0.20 () {58 vars in 1015 bytes} [Mon Aug 1 16:48:14 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:48:17.299534 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8dd398-aa72-495e-a094-f1179917ce80 None None] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:48:17.300110 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8dd398-aa72-495e-a094-f1179917ce80 None None] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:48:17.300110 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8dd398-aa72-495e-a094-f1179917ce80 None None] PATH_INFO: `/` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:48:17.301413 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 698/1414] 127.0.0.1 () {54 vars in 916 bytes} [Mon Aug 1 16:48:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 16:48:17.306833 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e039c4ca-1631-48ab-9c3c-feac5e30e265 None None] REQUEST_METHOD: `POST` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:48:17.307274 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e039c4ca-1631-48ab-9c3c-feac5e30e265 None None] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:48:17.307381 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e039c4ca-1631-48ab-9c3c-feac5e30e265 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:48:17.374162 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.auth.core [None req-e039c4ca-1631-48ab-9c3c-feac5e30e265 None None] MFA Rules not processed for user `1007f8e379b74e169b98fb7c3c0ba5b4`. Rule list: `[]` (Enabled: `True`). {{(pid=87058) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:48:17.375381 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.fernet_utils [None req-e039c4ca-1631-48ab-9c3c-feac5e30e265 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:48:17.429493 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG dbcounter [-] [87058] Writing DB stats keystone:SELECT=100 {{(pid=87058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:48:17.471280 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 717/1415] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Aug 1 16:48:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:48:17.514007 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed8dd398-aa72-495e-a094-f1179917ce80 None None] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:48:17.515624 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.fernet_utils [None req-ed8dd398-aa72-495e-a094-f1179917ce80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87057) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:48:17.660406 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed8dd398-aa72-495e-a094-f1179917ce80 None None] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:48:17.733171 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG dbcounter [-] [87057] Writing DB stats keystone:SELECT=100 {{(pid=87057) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:48:17.773166 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c00f1c-2c08-47c9-9ba3-80b846b4e8e8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6533106d03eb4ca185a8839336d2aff9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b992d6a65df4bf9b27c13192d57c5f0', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:48:17.773778 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c00f1c-2c08-47c9-9ba3-80b846b4e8e8 service nova] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:48:17.774077 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c00f1c-2c08-47c9-9ba3-80b846b4e8e8 service nova] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:48:17.774350 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c00f1c-2c08-47c9-9ba3-80b846b4e8e8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:48:17.776012 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.fernet_utils [None req-f4c00f1c-2c08-47c9-9ba3-80b846b4e8e8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87057) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:48:17.865870 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4c00f1c-2c08-47c9-9ba3-80b846b4e8e8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:48:17.867501 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4c00f1c-2c08-47c9-9ba3-80b846b4e8e8 service nova] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:48:17.886233 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 699/1416] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 1 16:48:17 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 373 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 16:48:18.789079 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5c16dd-18f9-4610-b204-51190ab94639 None None] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:48:18.789684 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5c16dd-18f9-4610-b204-51190ab94639 None None] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:48:18.789684 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5c16dd-18f9-4610-b204-51190ab94639 None None] PATH_INFO: `/` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:48:18.791104 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 718/1417] 127.0.0.1 () {54 vars in 901 bytes} [Mon Aug 1 16:48:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 16:48:18.796564 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4bc51ec-4ff6-44d5-810e-8086b735cf2c None None] REQUEST_METHOD: `POST` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:48:18.797019 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4bc51ec-4ff6-44d5-810e-8086b735cf2c None None] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:48:18.797131 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4bc51ec-4ff6-44d5-810e-8086b735cf2c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:48:18.871997 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.auth.core [None req-a4bc51ec-4ff6-44d5-810e-8086b735cf2c None None] MFA Rules not processed for user `1007f8e379b74e169b98fb7c3c0ba5b4`. Rule list: `[]` (Enabled: `True`). {{(pid=87057) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:48:18.873895 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.fernet_utils [None req-a4bc51ec-4ff6-44d5-810e-8086b735cf2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87057) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:48:18.900920 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG dbcounter [-] [87057] Writing DB stats keystone:SELECT=100 {{(pid=87057) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:48:18.972028 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 700/1418] 127.0.0.1 () {58 vars in 998 bytes} [Mon Aug 1 16:48:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:48:19.121418 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a5c16dd-18f9-4610-b204-51190ab94639 None None] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:48:19.123530 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.fernet_utils [None req-4a5c16dd-18f9-4610-b204-51190ab94639 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:48:19.270267 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a5c16dd-18f9-4610-b204-51190ab94639 None None] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:48:19.309377 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG dbcounter [-] [87058] Writing DB stats keystone:SELECT=100 {{(pid=87058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:48:19.383184 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d0b5947-229c-42af-b72e-2b8088bb8e17 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6533106d03eb4ca185a8839336d2aff9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b992d6a65df4bf9b27c13192d57c5f0', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:48:19.383729 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0b5947-229c-42af-b72e-2b8088bb8e17 service nova] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:48:19.384000 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0b5947-229c-42af-b72e-2b8088bb8e17 service nova] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:48:19.384264 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0b5947-229c-42af-b72e-2b8088bb8e17 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:48:19.385868 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.fernet_utils [None req-3d0b5947-229c-42af-b72e-2b8088bb8e17 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:48:19.473902 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d0b5947-229c-42af-b72e-2b8088bb8e17 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:48:19.475508 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d0b5947-229c-42af-b72e-2b8088bb8e17 service nova] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:48:19.493848 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 719/1419] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 1 16:48:19 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 373 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 16:48:20.373474 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272dee98-0c2b-4ad6-8693-5153788607a5 None None] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:48:20.374074 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272dee98-0c2b-4ad6-8693-5153788607a5 None None] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:48:20.374074 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272dee98-0c2b-4ad6-8693-5153788607a5 None None] PATH_INFO: `/` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:48:20.375388 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 701/1420] 127.0.0.1 () {54 vars in 901 bytes} [Mon Aug 1 16:48:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 16:48:20.380519 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060fa588-441b-4189-beeb-ad399418aea0 None None] REQUEST_METHOD: `POST` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:48:20.380955 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060fa588-441b-4189-beeb-ad399418aea0 None None] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:48:20.381069 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060fa588-441b-4189-beeb-ad399418aea0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:48:20.436648 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG dbcounter [-] [87058] Writing DB stats keystone:SELECT=100 {{(pid=87058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:48:20.458228 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.auth.core [None req-060fa588-441b-4189-beeb-ad399418aea0 None None] MFA Rules not processed for user `1007f8e379b74e169b98fb7c3c0ba5b4`. Rule list: `[]` (Enabled: `True`). {{(pid=87058) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:48:20.459424 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.fernet_utils [None req-060fa588-441b-4189-beeb-ad399418aea0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:48:20.554928 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 720/1421] 127.0.0.1 () {58 vars in 998 bytes} [Mon Aug 1 16:48:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:48:20.709484 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-272dee98-0c2b-4ad6-8693-5153788607a5 None None] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:48:20.711013 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.fernet_utils [None req-272dee98-0c2b-4ad6-8693-5153788607a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87057) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:48:20.857582 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-272dee98-0c2b-4ad6-8693-5153788607a5 None None] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:48:20.869791 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG dbcounter [-] [87057] Writing DB stats keystone:SELECT=100 {{(pid=87057) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:48:20.971017 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35367f5d-0aa0-4dd7-96b7-eb0f73b1cd01 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6533106d03eb4ca185a8839336d2aff9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b992d6a65df4bf9b27c13192d57c5f0', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:48:20.971559 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35367f5d-0aa0-4dd7-96b7-eb0f73b1cd01 service nova] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:48:20.971840 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35367f5d-0aa0-4dd7-96b7-eb0f73b1cd01 service nova] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:48:20.972112 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35367f5d-0aa0-4dd7-96b7-eb0f73b1cd01 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:48:20.973751 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.fernet_utils [None req-35367f5d-0aa0-4dd7-96b7-eb0f73b1cd01 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87057) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:48:21.065461 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35367f5d-0aa0-4dd7-96b7-eb0f73b1cd01 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:48:21.067109 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35367f5d-0aa0-4dd7-96b7-eb0f73b1cd01 service nova] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:48:21.085576 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 702/1422] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 1 16:48:20 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 377 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 16:48:21.982040 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cea3237-aeab-4dd8-8ce2-a1a0b9b978cf None None] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:48:21.982601 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cea3237-aeab-4dd8-8ce2-a1a0b9b978cf None None] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:48:21.982601 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cea3237-aeab-4dd8-8ce2-a1a0b9b978cf None None] PATH_INFO: `/` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:48:21.983989 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 721/1423] 127.0.0.1 () {54 vars in 901 bytes} [Mon Aug 1 16:48:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 16:48:21.989057 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6c701b-329e-4e20-b918-6a0832349bbc None None] REQUEST_METHOD: `POST` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:48:21.989457 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6c701b-329e-4e20-b918-6a0832349bbc None None] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:48:21.989607 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6c701b-329e-4e20-b918-6a0832349bbc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:48:22.010092 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG dbcounter [-] [87057] Writing DB stats keystone:SELECT=100 {{(pid=87057) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:48:22.066082 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.auth.core [None req-0e6c701b-329e-4e20-b918-6a0832349bbc None None] MFA Rules not processed for user `1007f8e379b74e169b98fb7c3c0ba5b4`. Rule list: `[]` (Enabled: `True`). {{(pid=87057) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:48:22.067296 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.fernet_utils [None req-0e6c701b-329e-4e20-b918-6a0832349bbc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87057) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:48:22.163591 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 703/1424] 127.0.0.1 () {58 vars in 998 bytes} [Mon Aug 1 16:48:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:48:22.313061 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cea3237-aeab-4dd8-8ce2-a1a0b9b978cf None None] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:48:22.314618 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.fernet_utils [None req-1cea3237-aeab-4dd8-8ce2-a1a0b9b978cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:48:22.426237 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG dbcounter [-] [87058] Writing DB stats keystone:SELECT=100 {{(pid=87058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:48:22.461451 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cea3237-aeab-4dd8-8ce2-a1a0b9b978cf None None] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:48:22.572782 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed08faf-370d-494c-911e-f9114f062c4a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6533106d03eb4ca185a8839336d2aff9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b992d6a65df4bf9b27c13192d57c5f0', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:48:22.573392 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed08faf-370d-494c-911e-f9114f062c4a service nova] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:48:22.573669 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed08faf-370d-494c-911e-f9114f062c4a service nova] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:48:22.573941 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed08faf-370d-494c-911e-f9114f062c4a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:48:22.575590 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.fernet_utils [None req-0ed08faf-370d-494c-911e-f9114f062c4a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:48:22.650319 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG dbcounter [-] [87058] Writing DB stats keystone:SELECT=100 {{(pid=87058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:48:22.663462 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ed08faf-370d-494c-911e-f9114f062c4a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:48:22.665038 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ed08faf-370d-494c-911e-f9114f062c4a service nova] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:48:22.685443 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 722/1425] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 1 16:48:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 373 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 16:48:23.550645 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f1d6cd-1026-4d61-ad4e-54dbd460a1fb None None] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:48:23.551243 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f1d6cd-1026-4d61-ad4e-54dbd460a1fb None None] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:48:23.551243 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f1d6cd-1026-4d61-ad4e-54dbd460a1fb None None] PATH_INFO: `/` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:48:23.552550 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 704/1426] 127.0.0.1 () {54 vars in 901 bytes} [Mon Aug 1 16:48:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 16:48:23.557425 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c448ca94-0454-401f-aea2-52a859e61108 None None] REQUEST_METHOD: `POST` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:48:23.557867 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c448ca94-0454-401f-aea2-52a859e61108 None None] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:48:23.557979 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c448ca94-0454-401f-aea2-52a859e61108 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:48:23.634526 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.auth.core [None req-c448ca94-0454-401f-aea2-52a859e61108 None None] MFA Rules not processed for user `1007f8e379b74e169b98fb7c3c0ba5b4`. Rule list: `[]` (Enabled: `True`). {{(pid=87058) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:48:23.635658 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.fernet_utils [None req-c448ca94-0454-401f-aea2-52a859e61108 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:48:23.730537 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 723/1427] 127.0.0.1 () {58 vars in 998 bytes} [Mon Aug 1 16:48:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:48:23.884055 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79f1d6cd-1026-4d61-ad4e-54dbd460a1fb None None] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:48:23.885665 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.fernet_utils [None req-79f1d6cd-1026-4d61-ad4e-54dbd460a1fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87057) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:48:23.971964 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG dbcounter [-] [87057] Writing DB stats keystone:SELECT=100 {{(pid=87057) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:48:24.033527 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79f1d6cd-1026-4d61-ad4e-54dbd460a1fb None None] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:48:24.145237 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-059be01c-d6cd-4e21-be34-9fdd21ad8566 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6533106d03eb4ca185a8839336d2aff9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b992d6a65df4bf9b27c13192d57c5f0', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:48:24.145773 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059be01c-d6cd-4e21-be34-9fdd21ad8566 service nova] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:48:24.146065 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059be01c-d6cd-4e21-be34-9fdd21ad8566 service nova] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:48:24.146335 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059be01c-d6cd-4e21-be34-9fdd21ad8566 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:48:24.148013 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.fernet_utils [None req-059be01c-d6cd-4e21-be34-9fdd21ad8566 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87057) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:48:24.195278 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG dbcounter [-] [87057] Writing DB stats keystone:SELECT=100 {{(pid=87057) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:48:24.236425 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-059be01c-d6cd-4e21-be34-9fdd21ad8566 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:48:24.238017 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-059be01c-d6cd-4e21-be34-9fdd21ad8566 service nova] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:48:24.257046 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 705/1428] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 1 16:48:23 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 373 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 16:48:25.145320 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d3156e-672d-43e9-b3c5-dcbdb079ddb1 None None] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:48:25.145937 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d3156e-672d-43e9-b3c5-dcbdb079ddb1 None None] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:48:25.145937 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d3156e-672d-43e9-b3c5-dcbdb079ddb1 None None] PATH_INFO: `/` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:48:25.147181 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 724/1429] 127.0.0.1 () {54 vars in 901 bytes} [Mon Aug 1 16:48:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 16:48:25.152028 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0642d472-00b4-475d-a603-18c2c76d2907 None None] REQUEST_METHOD: `POST` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:48:25.152442 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0642d472-00b4-475d-a603-18c2c76d2907 None None] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:48:25.152565 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0642d472-00b4-475d-a603-18c2c76d2907 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:48:25.229334 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.auth.core [None req-0642d472-00b4-475d-a603-18c2c76d2907 None None] MFA Rules not processed for user `1007f8e379b74e169b98fb7c3c0ba5b4`. Rule list: `[]` (Enabled: `True`). {{(pid=87057) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:48:25.230599 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.fernet_utils [None req-0642d472-00b4-475d-a603-18c2c76d2907 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87057) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:48:25.327409 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 706/1430] 127.0.0.1 () {58 vars in 998 bytes} [Mon Aug 1 16:48:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:48:25.478540 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d3156e-672d-43e9-b3c5-dcbdb079ddb1 None None] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:48:25.479995 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.fernet_utils [None req-12d3156e-672d-43e9-b3c5-dcbdb079ddb1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:48:25.529435 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG dbcounter [-] [87058] Writing DB stats keystone:SELECT=100 {{(pid=87058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:48:25.625613 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d3156e-672d-43e9-b3c5-dcbdb079ddb1 None None] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:48:25.736973 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5df4192-04ef-4dd7-9b9c-fd03f5f6f992 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6533106d03eb4ca185a8839336d2aff9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b992d6a65df4bf9b27c13192d57c5f0', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:48:25.737520 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5df4192-04ef-4dd7-9b9c-fd03f5f6f992 service nova] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:48:25.737801 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5df4192-04ef-4dd7-9b9c-fd03f5f6f992 service nova] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:48:25.738164 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5df4192-04ef-4dd7-9b9c-fd03f5f6f992 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:48:25.739794 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.fernet_utils [None req-f5df4192-04ef-4dd7-9b9c-fd03f5f6f992 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:48:25.746705 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG dbcounter [-] [87058] Writing DB stats keystone:SELECT=100 {{(pid=87058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:48:25.828651 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5df4192-04ef-4dd7-9b9c-fd03f5f6f992 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:48:25.830267 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5df4192-04ef-4dd7-9b9c-fd03f5f6f992 service nova] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:48:25.849165 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 725/1431] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 1 16:48:25 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 371 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 16:48:26.713002 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8c5516-beaf-4400-9329-b12347ee83ae None None] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:48:26.713521 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8c5516-beaf-4400-9329-b12347ee83ae None None] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:48:26.713620 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8c5516-beaf-4400-9329-b12347ee83ae None None] PATH_INFO: `/` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:48:26.714922 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 707/1432] 127.0.0.1 () {54 vars in 901 bytes} [Mon Aug 1 16:48:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 16:48:26.719961 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f453b2-e73d-4e47-ac60-e7dcf7ab6f14 None None] REQUEST_METHOD: `POST` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:48:26.720371 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f453b2-e73d-4e47-ac60-e7dcf7ab6f14 None None] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:48:26.720516 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f453b2-e73d-4e47-ac60-e7dcf7ab6f14 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:48:26.797063 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.auth.core [None req-52f453b2-e73d-4e47-ac60-e7dcf7ab6f14 None None] MFA Rules not processed for user `1007f8e379b74e169b98fb7c3c0ba5b4`. Rule list: `[]` (Enabled: `True`). {{(pid=87058) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:48:26.798289 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.fernet_utils [None req-52f453b2-e73d-4e47-ac60-e7dcf7ab6f14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:48:26.865078 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG dbcounter [-] [87058] Writing DB stats keystone:SELECT=100 {{(pid=87058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:48:26.893719 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 726/1433] 127.0.0.1 () {58 vars in 998 bytes} [Mon Aug 1 16:48:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:48:27.044918 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d8c5516-beaf-4400-9329-b12347ee83ae None None] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:48:27.046516 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.fernet_utils [None req-6d8c5516-beaf-4400-9329-b12347ee83ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87057) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:48:27.064974 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG dbcounter [-] [87057] Writing DB stats keystone:SELECT=100 {{(pid=87057) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:48:27.194624 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d8c5516-beaf-4400-9329-b12347ee83ae None None] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:48:27.281462 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG dbcounter [-] [87057] Writing DB stats keystone:SELECT=100 {{(pid=87057) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:48:27.306901 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-debeea80-c834-489a-9d56-020689c48ccc service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6533106d03eb4ca185a8839336d2aff9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b992d6a65df4bf9b27c13192d57c5f0', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:48:27.307450 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-debeea80-c834-489a-9d56-020689c48ccc service nova] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:48:27.307728 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-debeea80-c834-489a-9d56-020689c48ccc service nova] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:48:27.307995 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-debeea80-c834-489a-9d56-020689c48ccc service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:48:27.310263 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.fernet_utils [None req-debeea80-c834-489a-9d56-020689c48ccc service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87057) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:48:27.398406 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-debeea80-c834-489a-9d56-020689c48ccc service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:48:27.399993 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-debeea80-c834-489a-9d56-020689c48ccc service nova] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:48:27.417547 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 708/1434] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 1 16:48:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 374 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 16:48:28.257425 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baaac8f1-cef3-4d9e-9644-21346b411bf9 None None] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:48:28.257960 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baaac8f1-cef3-4d9e-9644-21346b411bf9 None None] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:48:28.257960 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baaac8f1-cef3-4d9e-9644-21346b411bf9 None None] PATH_INFO: `/` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:48:28.259302 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 727/1435] 127.0.0.1 () {54 vars in 901 bytes} [Mon Aug 1 16:48:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 16:48:28.264745 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeed509a-29be-4c89-a361-139e54296fab None None] REQUEST_METHOD: `POST` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:48:28.265160 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeed509a-29be-4c89-a361-139e54296fab None None] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:48:28.265298 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeed509a-29be-4c89-a361-139e54296fab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:48:28.341905 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.auth.core [None req-eeed509a-29be-4c89-a361-139e54296fab None None] MFA Rules not processed for user `1007f8e379b74e169b98fb7c3c0ba5b4`. Rule list: `[]` (Enabled: `True`). {{(pid=87057) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:48:28.343165 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.fernet_utils [None req-eeed509a-29be-4c89-a361-139e54296fab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87057) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:48:28.386766 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG dbcounter [-] [87057] Writing DB stats keystone:SELECT=100 {{(pid=87057) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:48:28.442728 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 709/1436] 127.0.0.1 () {58 vars in 998 bytes} [Mon Aug 1 16:48:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:48:28.594473 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baaac8f1-cef3-4d9e-9644-21346b411bf9 None None] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:48:28.596000 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.fernet_utils [None req-baaac8f1-cef3-4d9e-9644-21346b411bf9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:48:28.742579 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baaac8f1-cef3-4d9e-9644-21346b411bf9 None None] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:48:28.797302 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG dbcounter [-] [87058] Writing DB stats keystone:SELECT=100 {{(pid=87058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:48:28.855663 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45bb7655-7579-4a38-848b-d10e52cfcbe6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6533106d03eb4ca185a8839336d2aff9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b992d6a65df4bf9b27c13192d57c5f0', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:48:28.856188 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45bb7655-7579-4a38-848b-d10e52cfcbe6 service nova] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:48:28.857041 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45bb7655-7579-4a38-848b-d10e52cfcbe6 service nova] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:48:28.857313 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45bb7655-7579-4a38-848b-d10e52cfcbe6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:48:28.858985 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.fernet_utils [None req-45bb7655-7579-4a38-848b-d10e52cfcbe6 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:48:28.947980 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45bb7655-7579-4a38-848b-d10e52cfcbe6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:48:28.949582 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45bb7655-7579-4a38-848b-d10e52cfcbe6 service nova] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:48:28.968227 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 728/1437] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 1 16:48:28 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 374 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 16:48:29.839854 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf47764-392e-4803-bd2d-d83a0fcc5ac4 None None] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:48:29.840406 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf47764-392e-4803-bd2d-d83a0fcc5ac4 None None] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:48:29.840406 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf47764-392e-4803-bd2d-d83a0fcc5ac4 None None] PATH_INFO: `/` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:48:29.841988 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 710/1438] 127.0.0.1 () {54 vars in 901 bytes} [Mon Aug 1 16:48:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 16:48:29.846822 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f137ee-181c-479c-a62e-3453f5ddcccc None None] REQUEST_METHOD: `POST` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:48:29.847248 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f137ee-181c-479c-a62e-3453f5ddcccc None None] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:48:29.847444 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f137ee-181c-479c-a62e-3453f5ddcccc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:48:29.933767 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.auth.core [None req-68f137ee-181c-479c-a62e-3453f5ddcccc None None] MFA Rules not processed for user `1007f8e379b74e169b98fb7c3c0ba5b4`. Rule list: `[]` (Enabled: `True`). {{(pid=87058) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:48:29.935064 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.fernet_utils [None req-68f137ee-181c-479c-a62e-3453f5ddcccc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:48:29.937114 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG dbcounter [-] [87058] Writing DB stats keystone:SELECT=100 {{(pid=87058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:48:30.032707 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 729/1439] 127.0.0.1 () {58 vars in 998 bytes} [Mon Aug 1 16:48:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 16:48:30.183274 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf47764-392e-4803-bd2d-d83a0fcc5ac4 None None] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:48:30.185409 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.fernet_utils [None req-9cf47764-392e-4803-bd2d-d83a0fcc5ac4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87057) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:48:30.334704 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf47764-392e-4803-bd2d-d83a0fcc5ac4 None None] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:48:30.362378 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG dbcounter [-] [87057] Writing DB stats keystone:SELECT=100 {{(pid=87057) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:48:30.449138 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99abc55d-676b-4b83-acf3-a361dcbb208f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6533106d03eb4ca185a8839336d2aff9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b992d6a65df4bf9b27c13192d57c5f0', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:48:30.449675 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99abc55d-676b-4b83-acf3-a361dcbb208f service nova] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:48:30.449969 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99abc55d-676b-4b83-acf3-a361dcbb208f service nova] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:48:30.450288 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99abc55d-676b-4b83-acf3-a361dcbb208f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:48:30.451959 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.fernet_utils [None req-99abc55d-676b-4b83-acf3-a361dcbb208f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87057) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:48:30.540562 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99abc55d-676b-4b83-acf3-a361dcbb208f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:48:30.542189 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99abc55d-676b-4b83-acf3-a361dcbb208f service nova] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:48:30.561170 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 711/1440] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 1 16:48:30 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 378 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 16:48:32.478203 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc557b7-bcdf-4f52-8795-3c8fd2e46043 None None] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:48:32.478813 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc557b7-bcdf-4f52-8795-3c8fd2e46043 None None] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:48:32.478813 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc557b7-bcdf-4f52-8795-3c8fd2e46043 None None] PATH_INFO: `/` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:48:32.480174 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 730/1441] 127.0.0.1 () {54 vars in 916 bytes} [Mon Aug 1 16:48:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 16:48:32.486178 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b01c26-2c39-4540-82da-78411fb836f8 None None] REQUEST_METHOD: `POST` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:48:32.486626 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b01c26-2c39-4540-82da-78411fb836f8 None None] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:48:32.486740 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b01c26-2c39-4540-82da-78411fb836f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:48:32.526729 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG dbcounter [-] [87057] Writing DB stats keystone:SELECT=100 {{(pid=87057) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:48:32.551568 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.auth.core [None req-c1b01c26-2c39-4540-82da-78411fb836f8 None None] MFA Rules not processed for user `1007f8e379b74e169b98fb7c3c0ba5b4`. Rule list: `[]` (Enabled: `True`). {{(pid=87057) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:48:32.553808 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.fernet_utils [None req-c1b01c26-2c39-4540-82da-78411fb836f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87057) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:48:32.648387 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 712/1442] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Aug 1 16:48:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:48:32.687264 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc557b7-bcdf-4f52-8795-3c8fd2e46043 None None] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:48:32.688831 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.fernet_utils [None req-abc557b7-bcdf-4f52-8795-3c8fd2e46043 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:48:32.793072 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc557b7-bcdf-4f52-8795-3c8fd2e46043 None None] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:48:32.826285 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG dbcounter [-] [87058] Writing DB stats keystone:SELECT=100 {{(pid=87058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:48:32.868011 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b856f151-c580-46a3-8556-f999f42e6e18 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d9bd8d649154c979be3744bfd016f95', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b992d6a65df4bf9b27c13192d57c5f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:48:32.868627 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b856f151-c580-46a3-8556-f999f42e6e18 service neutron] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:48:32.868907 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b856f151-c580-46a3-8556-f999f42e6e18 service neutron] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:48:32.869181 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b856f151-c580-46a3-8556-f999f42e6e18 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:48:32.870844 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.fernet_utils [None req-b856f151-c580-46a3-8556-f999f42e6e18 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:48:32.980719 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b856f151-c580-46a3-8556-f999f42e6e18 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:48:32.982216 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b856f151-c580-46a3-8556-f999f42e6e18 service neutron] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:48:33.000521 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 731/1443] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 1 16:48:32 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 16:48:35.218307 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3d334b-e05f-47b7-bc61-38b60d4b6c2c None None] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:48:35.218836 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3d334b-e05f-47b7-bc61-38b60d4b6c2c None None] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:48:35.218836 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3d334b-e05f-47b7-bc61-38b60d4b6c2c None None] PATH_INFO: `/` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:48:35.220129 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 713/1444] 127.0.0.1 () {54 vars in 916 bytes} [Mon Aug 1 16:48:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 16:48:35.225440 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d0d75a-e8d2-4fe3-906f-0dc6a399d995 None None] REQUEST_METHOD: `POST` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:48:35.225846 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d0d75a-e8d2-4fe3-906f-0dc6a399d995 None None] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:48:35.225993 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d0d75a-e8d2-4fe3-906f-0dc6a399d995 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:48:35.292794 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.auth.core [None req-53d0d75a-e8d2-4fe3-906f-0dc6a399d995 None None] MFA Rules not processed for user `1007f8e379b74e169b98fb7c3c0ba5b4`. Rule list: `[]` (Enabled: `True`). {{(pid=87058) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:48:35.294589 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.fernet_utils [None req-53d0d75a-e8d2-4fe3-906f-0dc6a399d995 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:48:35.307290 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG dbcounter [-] [87058] Writing DB stats keystone:SELECT=100 {{(pid=87058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:48:35.391316 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 732/1445] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Aug 1 16:48:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:48:35.430300 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa3d334b-e05f-47b7-bc61-38b60d4b6c2c None None] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:48:35.431833 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.fernet_utils [None req-aa3d334b-e05f-47b7-bc61-38b60d4b6c2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87057) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:48:35.534566 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa3d334b-e05f-47b7-bc61-38b60d4b6c2c None None] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:48:35.546762 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG dbcounter [-] [87057] Writing DB stats keystone:SELECT=100 {{(pid=87057) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:48:35.607695 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f90254-d3a5-45cf-b863-20055eafb1b1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d9bd8d649154c979be3744bfd016f95', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b992d6a65df4bf9b27c13192d57c5f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:48:35.608313 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f90254-d3a5-45cf-b863-20055eafb1b1 service neutron] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:48:35.608593 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f90254-d3a5-45cf-b863-20055eafb1b1 service neutron] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:48:35.608868 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f90254-d3a5-45cf-b863-20055eafb1b1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:48:35.610531 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.fernet_utils [None req-d0f90254-d3a5-45cf-b863-20055eafb1b1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87057) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:48:35.719991 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0f90254-d3a5-45cf-b863-20055eafb1b1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:48:35.721431 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0f90254-d3a5-45cf-b863-20055eafb1b1 service neutron] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:48:35.740678 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 714/1446] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 1 16:48:35 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 16:48:38.074116 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d282196d-935d-4ff2-84f1-49cf28ab67d0 None None] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:48:38.074800 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d282196d-935d-4ff2-84f1-49cf28ab67d0 None None] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:48:38.074800 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d282196d-935d-4ff2-84f1-49cf28ab67d0 None None] PATH_INFO: `/` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:48:38.076687 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 733/1447] 127.0.0.1 () {54 vars in 916 bytes} [Mon Aug 1 16:48:38 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 16:48:38.081980 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64eed0f7-6607-49e0-aed0-cc25e9a09a9c None None] REQUEST_METHOD: `POST` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:48:38.082431 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64eed0f7-6607-49e0-aed0-cc25e9a09a9c None None] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:48:38.082576 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64eed0f7-6607-49e0-aed0-cc25e9a09a9c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:48:38.130533 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG dbcounter [-] [87057] Writing DB stats keystone:SELECT=100 {{(pid=87057) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:48:38.147820 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.auth.core [None req-64eed0f7-6607-49e0-aed0-cc25e9a09a9c None None] MFA Rules not processed for user `1007f8e379b74e169b98fb7c3c0ba5b4`. Rule list: `[]` (Enabled: `True`). {{(pid=87057) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:48:38.151689 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.fernet_utils [None req-64eed0f7-6607-49e0-aed0-cc25e9a09a9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87057) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:48:38.246328 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 715/1448] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Aug 1 16:48:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:48:38.285178 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d282196d-935d-4ff2-84f1-49cf28ab67d0 None None] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:48:38.286751 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.fernet_utils [None req-d282196d-935d-4ff2-84f1-49cf28ab67d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:48:38.389095 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d282196d-935d-4ff2-84f1-49cf28ab67d0 None None] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:48:38.428188 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG dbcounter [-] [87058] Writing DB stats keystone:SELECT=100 {{(pid=87058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:48:38.462137 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ed9e64-e85b-4420-b407-c0640d95fa98 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d9bd8d649154c979be3744bfd016f95', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b992d6a65df4bf9b27c13192d57c5f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:48:38.462754 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ed9e64-e85b-4420-b407-c0640d95fa98 service neutron] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:48:38.463035 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ed9e64-e85b-4420-b407-c0640d95fa98 service neutron] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:48:38.463307 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ed9e64-e85b-4420-b407-c0640d95fa98 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:48:38.464927 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.fernet_utils [None req-93ed9e64-e85b-4420-b407-c0640d95fa98 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:48:38.575685 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93ed9e64-e85b-4420-b407-c0640d95fa98 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:48:38.577113 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93ed9e64-e85b-4420-b407-c0640d95fa98 service neutron] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:48:38.595499 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 734/1449] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 1 16:48:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 16:48:40.711951 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061e562b-c8d3-4a90-9c12-ba22702fa213 None None] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:48:40.712495 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061e562b-c8d3-4a90-9c12-ba22702fa213 None None] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:48:40.712495 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061e562b-c8d3-4a90-9c12-ba22702fa213 None None] PATH_INFO: `/` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:48:40.713842 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 716/1450] 127.0.0.1 () {54 vars in 916 bytes} [Mon Aug 1 16:48:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 16:48:40.719932 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009e51d9-1427-435c-a359-93eff498b73d None None] REQUEST_METHOD: `POST` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:48:40.720371 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009e51d9-1427-435c-a359-93eff498b73d None None] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:48:40.720468 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009e51d9-1427-435c-a359-93eff498b73d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:48:40.787740 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.auth.core [None req-009e51d9-1427-435c-a359-93eff498b73d None None] MFA Rules not processed for user `1007f8e379b74e169b98fb7c3c0ba5b4`. Rule list: `[]` (Enabled: `True`). {{(pid=87058) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:48:40.788960 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.fernet_utils [None req-009e51d9-1427-435c-a359-93eff498b73d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:48:40.809726 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG dbcounter [-] [87058] Writing DB stats keystone:SELECT=100 {{(pid=87058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:48:40.885794 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 735/1451] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Aug 1 16:48:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:48:40.925871 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-061e562b-c8d3-4a90-9c12-ba22702fa213 None None] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:48:40.927457 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.fernet_utils [None req-061e562b-c8d3-4a90-9c12-ba22702fa213 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87057) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:48:41.030663 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-061e562b-c8d3-4a90-9c12-ba22702fa213 None None] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:48:41.050371 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG dbcounter [-] [87057] Writing DB stats keystone:SELECT=100 {{(pid=87057) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:48:41.103489 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c717bd3-01aa-4fd0-949a-405e5ea680d2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d9bd8d649154c979be3744bfd016f95', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b992d6a65df4bf9b27c13192d57c5f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:48:41.104101 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c717bd3-01aa-4fd0-949a-405e5ea680d2 service neutron] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:48:41.104380 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c717bd3-01aa-4fd0-949a-405e5ea680d2 service neutron] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:48:41.104649 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c717bd3-01aa-4fd0-949a-405e5ea680d2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:48:41.106920 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.fernet_utils [None req-1c717bd3-01aa-4fd0-949a-405e5ea680d2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87057) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:48:41.214536 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c717bd3-01aa-4fd0-949a-405e5ea680d2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:48:41.215987 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c717bd3-01aa-4fd0-949a-405e5ea680d2 service neutron] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:48:41.234160 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 717/1452] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 1 16:48:40 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 16:48:43.280642 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4636058-df7f-4f79-94cf-302dd56ba263 None None] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:48:43.281198 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4636058-df7f-4f79-94cf-302dd56ba263 None None] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:48:43.281198 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4636058-df7f-4f79-94cf-302dd56ba263 None None] PATH_INFO: `/` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:48:43.282632 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 736/1453] 127.0.0.1 () {54 vars in 916 bytes} [Mon Aug 1 16:48:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 16:48:43.287911 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942f27ac-3435-4b7a-9d56-ba5f4df0f5b5 None None] REQUEST_METHOD: `POST` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:48:43.288329 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942f27ac-3435-4b7a-9d56-ba5f4df0f5b5 None None] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:48:43.288464 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942f27ac-3435-4b7a-9d56-ba5f4df0f5b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:48:43.352297 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG dbcounter [-] [87057] Writing DB stats keystone:SELECT=100 {{(pid=87057) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:48:43.355245 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.auth.core [None req-942f27ac-3435-4b7a-9d56-ba5f4df0f5b5 None None] MFA Rules not processed for user `1007f8e379b74e169b98fb7c3c0ba5b4`. Rule list: `[]` (Enabled: `True`). {{(pid=87057) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:48:43.356481 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.fernet_utils [None req-942f27ac-3435-4b7a-9d56-ba5f4df0f5b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87057) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:48:43.453149 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 718/1454] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Aug 1 16:48:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:48:43.491272 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4636058-df7f-4f79-94cf-302dd56ba263 None None] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:48:43.492829 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.fernet_utils [None req-b4636058-df7f-4f79-94cf-302dd56ba263 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:48:43.596335 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4636058-df7f-4f79-94cf-302dd56ba263 None None] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:48:43.644631 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG dbcounter [-] [87058] Writing DB stats keystone:SELECT=100 {{(pid=87058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:48:43.669115 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad3949df-62d7-4d33-bb35-7bfb724ad834 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d9bd8d649154c979be3744bfd016f95', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b992d6a65df4bf9b27c13192d57c5f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:48:43.669727 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3949df-62d7-4d33-bb35-7bfb724ad834 service neutron] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:48:43.670003 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3949df-62d7-4d33-bb35-7bfb724ad834 service neutron] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:48:43.670306 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3949df-62d7-4d33-bb35-7bfb724ad834 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:48:43.671925 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.fernet_utils [None req-ad3949df-62d7-4d33-bb35-7bfb724ad834 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:48:43.779565 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad3949df-62d7-4d33-bb35-7bfb724ad834 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:48:43.780984 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad3949df-62d7-4d33-bb35-7bfb724ad834 service neutron] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:48:43.799162 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 737/1455] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 1 16:48:43 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 16:48:45.869149 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907ed569-d3a8-4690-b7db-304b0bf4f383 None None] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:48:45.869727 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907ed569-d3a8-4690-b7db-304b0bf4f383 None None] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:48:45.869727 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907ed569-d3a8-4690-b7db-304b0bf4f383 None None] PATH_INFO: `/` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:48:45.871101 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 719/1456] 127.0.0.1 () {54 vars in 916 bytes} [Mon Aug 1 16:48:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 16:48:45.876538 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de433699-e2fe-40a0-be2f-7f7b835ba44f None None] REQUEST_METHOD: `POST` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:48:45.876979 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de433699-e2fe-40a0-be2f-7f7b835ba44f None None] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:48:45.877099 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de433699-e2fe-40a0-be2f-7f7b835ba44f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:48:45.945283 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.auth.core [None req-de433699-e2fe-40a0-be2f-7f7b835ba44f None None] MFA Rules not processed for user `1007f8e379b74e169b98fb7c3c0ba5b4`. Rule list: `[]` (Enabled: `True`). {{(pid=87058) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:48:45.946541 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.fernet_utils [None req-de433699-e2fe-40a0-be2f-7f7b835ba44f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:48:45.976577 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG dbcounter [-] [87058] Writing DB stats keystone:SELECT=100 {{(pid=87058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:48:46.045048 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 738/1457] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Aug 1 16:48:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:48:46.083205 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-907ed569-d3a8-4690-b7db-304b0bf4f383 None None] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:48:46.084797 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.fernet_utils [None req-907ed569-d3a8-4690-b7db-304b0bf4f383 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87057) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:48:46.189881 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-907ed569-d3a8-4690-b7db-304b0bf4f383 None None] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:48:46.218530 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG dbcounter [-] [87057] Writing DB stats keystone:SELECT=100 {{(pid=87057) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:48:46.263973 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c1ab4a6-eed4-4893-b783-e9d624e8d7f1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d9bd8d649154c979be3744bfd016f95', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b992d6a65df4bf9b27c13192d57c5f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:48:46.264615 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1ab4a6-eed4-4893-b783-e9d624e8d7f1 service neutron] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:48:46.264894 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1ab4a6-eed4-4893-b783-e9d624e8d7f1 service neutron] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:48:46.265166 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1ab4a6-eed4-4893-b783-e9d624e8d7f1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:48:46.266821 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.fernet_utils [None req-7c1ab4a6-eed4-4893-b783-e9d624e8d7f1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87057) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:48:46.376460 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c1ab4a6-eed4-4893-b783-e9d624e8d7f1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:48:46.377912 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c1ab4a6-eed4-4893-b783-e9d624e8d7f1 service neutron] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:48:46.397672 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 720/1458] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 1 16:48:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 16:48:48.448896 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee7efec-c99d-46c9-8a38-3c5ea3559788 None None] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:48:48.450079 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee7efec-c99d-46c9-8a38-3c5ea3559788 None None] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:48:48.450079 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee7efec-c99d-46c9-8a38-3c5ea3559788 None None] PATH_INFO: `/` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:48:48.451443 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 739/1459] 127.0.0.1 () {54 vars in 916 bytes} [Mon Aug 1 16:48:48 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 16:48:48.456807 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b08cbd-3eec-480c-b1ad-8d5a612cfc7d None None] REQUEST_METHOD: `POST` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:48:48.457207 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b08cbd-3eec-480c-b1ad-8d5a612cfc7d None None] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:48:48.457358 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b08cbd-3eec-480c-b1ad-8d5a612cfc7d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:48:48.522230 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.auth.core [None req-29b08cbd-3eec-480c-b1ad-8d5a612cfc7d None None] MFA Rules not processed for user `1007f8e379b74e169b98fb7c3c0ba5b4`. Rule list: `[]` (Enabled: `True`). {{(pid=87057) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:48:48.523451 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.fernet_utils [None req-29b08cbd-3eec-480c-b1ad-8d5a612cfc7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87057) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:48:48.531586 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG dbcounter [-] [87057] Writing DB stats keystone:SELECT=100 {{(pid=87057) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:48:48.619127 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 721/1460] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Aug 1 16:48:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:48:48.657025 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee7efec-c99d-46c9-8a38-3c5ea3559788 None None] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:48:48.658579 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.fernet_utils [None req-0ee7efec-c99d-46c9-8a38-3c5ea3559788 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:48:48.763882 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee7efec-c99d-46c9-8a38-3c5ea3559788 None None] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:48:48.819031 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG dbcounter [-] [87058] Writing DB stats keystone:SELECT=100 {{(pid=87058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:48:48.836998 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c4d43d-8a3c-4d4a-a860-989d34203697 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d9bd8d649154c979be3744bfd016f95', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b992d6a65df4bf9b27c13192d57c5f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:48:48.837603 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c4d43d-8a3c-4d4a-a860-989d34203697 service neutron] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:48:48.837880 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c4d43d-8a3c-4d4a-a860-989d34203697 service neutron] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:48:48.838177 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c4d43d-8a3c-4d4a-a860-989d34203697 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:48:48.839777 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.fernet_utils [None req-b0c4d43d-8a3c-4d4a-a860-989d34203697 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:48:48.951484 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0c4d43d-8a3c-4d4a-a860-989d34203697 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:48:48.952918 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0c4d43d-8a3c-4d4a-a860-989d34203697 service neutron] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:48:48.971173 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 740/1461] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 1 16:48:48 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 16:48:51.023947 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7b9430-568b-4c3c-925d-e0744a78aeca None None] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:48:51.024482 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7b9430-568b-4c3c-925d-e0744a78aeca None None] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:48:51.024482 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7b9430-568b-4c3c-925d-e0744a78aeca None None] PATH_INFO: `/` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:48:51.025867 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 722/1462] 127.0.0.1 () {54 vars in 916 bytes} [Mon Aug 1 16:48:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 16:48:51.031710 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8773e879-5548-429f-8fe6-7624ff05761a None None] REQUEST_METHOD: `POST` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:48:51.032152 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8773e879-5548-429f-8fe6-7624ff05761a None None] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:48:51.032260 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8773e879-5548-429f-8fe6-7624ff05761a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:48:51.097586 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.auth.core [None req-8773e879-5548-429f-8fe6-7624ff05761a None None] MFA Rules not processed for user `1007f8e379b74e169b98fb7c3c0ba5b4`. Rule list: `[]` (Enabled: `True`). {{(pid=87058) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:48:51.098822 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.fernet_utils [None req-8773e879-5548-429f-8fe6-7624ff05761a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:48:51.136730 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG dbcounter [-] [87058] Writing DB stats keystone:SELECT=100 {{(pid=87058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:48:51.196412 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 741/1463] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Aug 1 16:48:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:48:51.233562 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c7b9430-568b-4c3c-925d-e0744a78aeca None None] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:48:51.235141 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.fernet_utils [None req-6c7b9430-568b-4c3c-925d-e0744a78aeca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87057) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:48:51.365659 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c7b9430-568b-4c3c-925d-e0744a78aeca None None] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:48:51.405055 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG dbcounter [-] [87057] Writing DB stats keystone:SELECT=100 {{(pid=87057) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:48:51.442386 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53eedd21-c7dd-467b-b644-1b746ebf90c4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d9bd8d649154c979be3744bfd016f95', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b992d6a65df4bf9b27c13192d57c5f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:48:51.443007 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53eedd21-c7dd-467b-b644-1b746ebf90c4 service neutron] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:48:51.443287 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53eedd21-c7dd-467b-b644-1b746ebf90c4 service neutron] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:48:51.443560 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53eedd21-c7dd-467b-b644-1b746ebf90c4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:48:51.445208 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.fernet_utils [None req-53eedd21-c7dd-467b-b644-1b746ebf90c4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87057) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:48:51.552949 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53eedd21-c7dd-467b-b644-1b746ebf90c4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:48:51.554425 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53eedd21-c7dd-467b-b644-1b746ebf90c4 service neutron] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:48:51.573150 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 723/1464] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 1 16:48:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 340 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 16:48:53.594999 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3b3269-9a0e-4580-8e33-5ddb22a7d945 None None] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:48:53.595564 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3b3269-9a0e-4580-8e33-5ddb22a7d945 None None] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:48:53.595564 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3b3269-9a0e-4580-8e33-5ddb22a7d945 None None] PATH_INFO: `/` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:48:53.596925 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 742/1465] 127.0.0.1 () {54 vars in 916 bytes} [Mon Aug 1 16:48:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 16:48:53.602243 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20281179-14b6-4e45-bb48-16f178fbdbea None None] REQUEST_METHOD: `POST` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:48:53.602646 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20281179-14b6-4e45-bb48-16f178fbdbea None None] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:48:53.602802 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20281179-14b6-4e45-bb48-16f178fbdbea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:48:53.668955 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.auth.core [None req-20281179-14b6-4e45-bb48-16f178fbdbea None None] MFA Rules not processed for user `1007f8e379b74e169b98fb7c3c0ba5b4`. Rule list: `[]` (Enabled: `True`). {{(pid=87057) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:48:53.670204 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.fernet_utils [None req-20281179-14b6-4e45-bb48-16f178fbdbea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87057) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:48:53.687787 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG dbcounter [-] [87057] Writing DB stats keystone:SELECT=100 {{(pid=87057) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:48:53.767377 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 724/1466] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Aug 1 16:48:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:48:53.804599 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f3b3269-9a0e-4580-8e33-5ddb22a7d945 None None] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:48:53.806194 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.fernet_utils [None req-0f3b3269-9a0e-4580-8e33-5ddb22a7d945 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:48:53.911173 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f3b3269-9a0e-4580-8e33-5ddb22a7d945 None None] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:48:53.979066 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG dbcounter [-] [87058] Writing DB stats keystone:SELECT=100 {{(pid=87058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:48:53.986829 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2518887-d76e-4d37-8d89-80f21d32a71e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d9bd8d649154c979be3744bfd016f95', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b992d6a65df4bf9b27c13192d57c5f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:48:53.987450 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2518887-d76e-4d37-8d89-80f21d32a71e service neutron] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:48:53.987736 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2518887-d76e-4d37-8d89-80f21d32a71e service neutron] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:48:53.988008 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2518887-d76e-4d37-8d89-80f21d32a71e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:48:53.989636 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.fernet_utils [None req-e2518887-d76e-4d37-8d89-80f21d32a71e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:48:54.110866 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2518887-d76e-4d37-8d89-80f21d32a71e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:48:54.112312 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2518887-d76e-4d37-8d89-80f21d32a71e service neutron] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:48:54.131053 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 743/1467] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 1 16:48:53 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 327 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 16:48:56.173704 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b474a3-e946-4364-b405-ac47657e927b None None] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:48:56.174250 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b474a3-e946-4364-b405-ac47657e927b None None] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:48:56.174250 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b474a3-e946-4364-b405-ac47657e927b None None] PATH_INFO: `/` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:48:56.175552 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 725/1468] 127.0.0.1 () {54 vars in 916 bytes} [Mon Aug 1 16:48:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 16:48:56.180934 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ada977-36bc-46c3-904c-fd9d6c4069db None None] REQUEST_METHOD: `POST` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:48:56.181332 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ada977-36bc-46c3-904c-fd9d6c4069db None None] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:48:56.181611 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ada977-36bc-46c3-904c-fd9d6c4069db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:48:56.248448 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.auth.core [None req-b3ada977-36bc-46c3-904c-fd9d6c4069db None None] MFA Rules not processed for user `1007f8e379b74e169b98fb7c3c0ba5b4`. Rule list: `[]` (Enabled: `True`). {{(pid=87058) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:48:56.249666 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.fernet_utils [None req-b3ada977-36bc-46c3-904c-fd9d6c4069db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:48:56.297757 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG dbcounter [-] [87058] Writing DB stats keystone:SELECT=100 {{(pid=87058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:48:56.347037 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 744/1469] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Aug 1 16:48:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:48:56.384837 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0b474a3-e946-4364-b405-ac47657e927b None None] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:48:56.386414 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.fernet_utils [None req-c0b474a3-e946-4364-b405-ac47657e927b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87057) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:48:56.490287 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0b474a3-e946-4364-b405-ac47657e927b None None] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:48:56.538658 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG dbcounter [-] [87057] Writing DB stats keystone:SELECT=100 {{(pid=87057) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:48:56.564655 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a85caaa2-7598-46e0-986a-1ab3739d2591 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d9bd8d649154c979be3744bfd016f95', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b992d6a65df4bf9b27c13192d57c5f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:48:56.565267 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85caaa2-7598-46e0-986a-1ab3739d2591 service neutron] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:48:56.565568 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85caaa2-7598-46e0-986a-1ab3739d2591 service neutron] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:48:56.565823 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85caaa2-7598-46e0-986a-1ab3739d2591 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:48:56.567513 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.fernet_utils [None req-a85caaa2-7598-46e0-986a-1ab3739d2591 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87057) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:48:56.677074 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a85caaa2-7598-46e0-986a-1ab3739d2591 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:48:56.678537 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a85caaa2-7598-46e0-986a-1ab3739d2591 service neutron] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:48:56.696184 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 726/1470] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 1 16:48:56 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 16:49:06.345098 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG dbcounter [-] [87058] Writing DB stats keystone:SELECT=22 {{(pid=87058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:49:06.693971 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG dbcounter [-] [87057] Writing DB stats keystone:SELECT=66 {{(pid=87057) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:49:16.155009 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9488f1fc-f1de-433e-ac35-403456b662cf None None] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:49:16.155570 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9488f1fc-f1de-433e-ac35-403456b662cf None None] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:49:16.155570 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9488f1fc-f1de-433e-ac35-403456b662cf None None] PATH_INFO: `/` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:49:16.156974 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 745/1471] 10.222.0.20 () {54 vars in 926 bytes} [Mon Aug 1 16:49:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 16:49:16.162242 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc466fab-a490-4537-8097-6a7077d1fac2 None None] REQUEST_METHOD: `POST` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:49:16.162695 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc466fab-a490-4537-8097-6a7077d1fac2 None None] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:49:16.162807 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc466fab-a490-4537-8097-6a7077d1fac2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:49:16.230753 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.auth.core [None req-dc466fab-a490-4537-8097-6a7077d1fac2 None None] MFA Rules not processed for user `1007f8e379b74e169b98fb7c3c0ba5b4`. Rule list: `[]` (Enabled: `True`). {{(pid=87057) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:49:16.231966 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.fernet_utils [None req-dc466fab-a490-4537-8097-6a7077d1fac2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87057) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:49:16.328365 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 727/1472] 10.222.0.20 () {58 vars in 1023 bytes} [Mon Aug 1 16:49:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:49:16.412689 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312a5c41-e833-4b20-8bff-641818347ca7 None None] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:49:16.412978 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312a5c41-e833-4b20-8bff-641818347ca7 None None] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:49:16.413253 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312a5c41-e833-4b20-8bff-641818347ca7 None None] PATH_INFO: `/` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:49:16.414636 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 746/1473] 10.222.0.20 () {54 vars in 966 bytes} [Mon Aug 1 16:49:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 16:49:16.420501 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ababc5-14c9-42b8-a633-bd9eacd9bc37 None None] REQUEST_METHOD: `POST` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:49:16.420790 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ababc5-14c9-42b8-a633-bd9eacd9bc37 None None] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:49:16.421066 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ababc5-14c9-42b8-a633-bd9eacd9bc37 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:49:16.499620 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.auth.core [None req-48ababc5-14c9-42b8-a633-bd9eacd9bc37 None None] MFA Rules not processed for user `d80694e439e2401d9491712666409016`. Rule list: `[]` (Enabled: `True`). {{(pid=87057) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:49:16.500819 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.fernet_utils [None req-48ababc5-14c9-42b8-a633-bd9eacd9bc37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87057) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:49:16.521389 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG dbcounter [-] [87057] Writing DB stats keystone:SELECT=100 {{(pid=87057) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:49:16.578431 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 728/1474] 10.222.0.20 () {58 vars in 1063 bytes} [Mon Aug 1 16:49:16 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:49:16.585985 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-312a5c41-e833-4b20-8bff-641818347ca7 None None] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:49:16.587514 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.fernet_utils [None req-312a5c41-e833-4b20-8bff-641818347ca7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:49:16.690572 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-312a5c41-e833-4b20-8bff-641818347ca7 None None] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:49:16.763914 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5514ad88-896e-45b7-ad39-d8d58479bdcc service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd80694e439e2401d9491712666409016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b992d6a65df4bf9b27c13192d57c5f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:49:16.764531 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5514ad88-896e-45b7-ad39-d8d58479bdcc service glance] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:49:16.764812 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5514ad88-896e-45b7-ad39-d8d58479bdcc service glance] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:49:16.765085 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5514ad88-896e-45b7-ad39-d8d58479bdcc service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:49:16.766744 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.fernet_utils [None req-5514ad88-896e-45b7-ad39-d8d58479bdcc service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:49:16.820739 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG dbcounter [-] [87058] Writing DB stats keystone:SELECT=100 {{(pid=87058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:49:16.875147 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5514ad88-896e-45b7-ad39-d8d58479bdcc service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:49:16.876585 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5514ad88-896e-45b7-ad39-d8d58479bdcc service glance] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:49:16.895569 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 747/1475] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 1 16:49:16 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 16:49:18.946799 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bebc6186-d3ca-4e07-ab1a-7b903d02a76c None None] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:49:18.947316 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bebc6186-d3ca-4e07-ab1a-7b903d02a76c None None] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:49:18.947316 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bebc6186-d3ca-4e07-ab1a-7b903d02a76c None None] PATH_INFO: `/` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:49:18.948735 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 729/1476] 10.222.0.20 () {54 vars in 926 bytes} [Mon Aug 1 16:49:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 16:49:18.953858 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d6d8c5-6f23-44e1-a692-4d0444b56ecd None None] REQUEST_METHOD: `POST` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:49:18.954296 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d6d8c5-6f23-44e1-a692-4d0444b56ecd None None] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:49:18.954456 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d6d8c5-6f23-44e1-a692-4d0444b56ecd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:49:19.021599 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.auth.core [None req-07d6d8c5-6f23-44e1-a692-4d0444b56ecd None None] MFA Rules not processed for user `1007f8e379b74e169b98fb7c3c0ba5b4`. Rule list: `[]` (Enabled: `True`). {{(pid=87058) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:49:19.022843 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.fernet_utils [None req-07d6d8c5-6f23-44e1-a692-4d0444b56ecd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:49:19.118041 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 748/1477] 10.222.0.20 () {58 vars in 1023 bytes} [Mon Aug 1 16:49:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:49:19.141616 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bebc6186-d3ca-4e07-ab1a-7b903d02a76c None None] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:49:19.143184 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.fernet_utils [None req-bebc6186-d3ca-4e07-ab1a-7b903d02a76c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87057) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:49:19.268440 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bebc6186-d3ca-4e07-ab1a-7b903d02a76c None None] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:49:19.318335 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG dbcounter [-] [87057] Writing DB stats keystone:SELECT=100 {{(pid=87057) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:49:19.363841 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64793053-6121-4b45-8c75-d25cb518b872 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:49:19.364448 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64793053-6121-4b45-8c75-d25cb518b872 admin admin] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:49:19.364719 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64793053-6121-4b45-8c75-d25cb518b872 admin admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:49:19.364986 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64793053-6121-4b45-8c75-d25cb518b872 admin admin] PATH_INFO: `/v3/projects` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:49:19.365674 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64793053-6121-4b45-8c75-d25cb518b872 admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:49:19.367412 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: WARNING py.warnings [None req-64793053-6121-4b45-8c75-d25cb518b872 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 01 16:49:19.367412 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: warnings.warn(msg) Aug 01 16:49:19.367412 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: Aug 01 16:49:19.368544 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64793053-6121-4b45-8c75-d25cb518b872 admin admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:49:19.388948 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 730/1478] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Aug 1 16:49:19 2022] GET /identity/v3/projects => generated 2553 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 16:49:21.405397 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78c5e0b-5518-4c1f-97c4-b7c003307d1b None None] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:49:21.405990 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78c5e0b-5518-4c1f-97c4-b7c003307d1b None None] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:49:21.405990 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78c5e0b-5518-4c1f-97c4-b7c003307d1b None None] PATH_INFO: `/` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:49:21.407352 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 749/1479] 10.222.0.20 () {54 vars in 926 bytes} [Mon Aug 1 16:49:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 16:49:21.412552 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733f1563-66b2-44d7-96da-baaa2a6adcda None None] REQUEST_METHOD: `POST` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:49:21.413033 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733f1563-66b2-44d7-96da-baaa2a6adcda None None] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:49:21.413133 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733f1563-66b2-44d7-96da-baaa2a6adcda None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:49:21.478225 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.auth.core [None req-733f1563-66b2-44d7-96da-baaa2a6adcda None None] MFA Rules not processed for user `1007f8e379b74e169b98fb7c3c0ba5b4`. Rule list: `[]` (Enabled: `True`). {{(pid=87057) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:49:21.479445 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.fernet_utils [None req-733f1563-66b2-44d7-96da-baaa2a6adcda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87057) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:49:21.575176 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 731/1480] 10.222.0.20 () {58 vars in 1023 bytes} [Mon Aug 1 16:49:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:49:21.617237 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f78c5e0b-5518-4c1f-97c4-b7c003307d1b None None] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:49:21.618811 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.fernet_utils [None req-f78c5e0b-5518-4c1f-97c4-b7c003307d1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:49:21.629607 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG dbcounter [-] [87058] Writing DB stats keystone:SELECT=100 {{(pid=87058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:49:21.770161 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f78c5e0b-5518-4c1f-97c4-b7c003307d1b None None] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:49:21.850001 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG dbcounter [-] [87058] Writing DB stats keystone:SELECT=100 {{(pid=87058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:49:21.883700 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f3f1cff-cc03-4572-93c4-c1980ba99c35 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6533106d03eb4ca185a8839336d2aff9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b992d6a65df4bf9b27c13192d57c5f0', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:49:21.884307 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3f1cff-cc03-4572-93c4-c1980ba99c35 service nova] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:49:21.884582 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3f1cff-cc03-4572-93c4-c1980ba99c35 service nova] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:49:21.884850 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3f1cff-cc03-4572-93c4-c1980ba99c35 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:49:21.887284 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.fernet_utils [None req-7f3f1cff-cc03-4572-93c4-c1980ba99c35 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:49:21.985407 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f3f1cff-cc03-4572-93c4-c1980ba99c35 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:49:21.987073 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f3f1cff-cc03-4572-93c4-c1980ba99c35 service nova] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:49:22.005542 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 750/1481] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 1 16:49:21 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 389 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 16:49:24.280401 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ab5c7a-1757-457e-8523-de8b15be1a0a None None] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:49:24.280947 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ab5c7a-1757-457e-8523-de8b15be1a0a None None] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:49:24.280947 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ab5c7a-1757-457e-8523-de8b15be1a0a None None] PATH_INFO: `/` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:49:24.282333 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 732/1482] 10.222.0.20 () {54 vars in 926 bytes} [Mon Aug 1 16:49:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 16:49:24.287551 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a6db3e-a6df-45d5-8092-a7ec9b0bf63f None None] REQUEST_METHOD: `POST` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:49:24.287967 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a6db3e-a6df-45d5-8092-a7ec9b0bf63f None None] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:49:24.288098 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a6db3e-a6df-45d5-8092-a7ec9b0bf63f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:49:24.354923 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.auth.core [None req-88a6db3e-a6df-45d5-8092-a7ec9b0bf63f None None] MFA Rules not processed for user `1007f8e379b74e169b98fb7c3c0ba5b4`. Rule list: `[]` (Enabled: `True`). {{(pid=87058) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:49:24.356116 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.fernet_utils [None req-88a6db3e-a6df-45d5-8092-a7ec9b0bf63f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:49:24.403114 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG dbcounter [-] [87058] Writing DB stats keystone:SELECT=100 {{(pid=87058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:49:24.453504 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 751/1483] 10.222.0.20 () {58 vars in 1023 bytes} [Mon Aug 1 16:49:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:49:24.490294 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ab5c7a-1757-457e-8523-de8b15be1a0a None None] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:49:24.491848 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.fernet_utils [None req-56ab5c7a-1757-457e-8523-de8b15be1a0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87057) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:49:24.516541 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG dbcounter [-] [87057] Writing DB stats keystone:SELECT=100 {{(pid=87057) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:49:24.597068 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ab5c7a-1757-457e-8523-de8b15be1a0a None None] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:49:24.669225 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3528e6df-63fb-4ad6-a46f-a02b651234b2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d9bd8d649154c979be3744bfd016f95', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b992d6a65df4bf9b27c13192d57c5f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:49:24.669843 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3528e6df-63fb-4ad6-a46f-a02b651234b2 service neutron] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:49:24.670156 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3528e6df-63fb-4ad6-a46f-a02b651234b2 service neutron] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:49:24.670463 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3528e6df-63fb-4ad6-a46f-a02b651234b2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:49:24.672094 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.fernet_utils [None req-3528e6df-63fb-4ad6-a46f-a02b651234b2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87057) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:49:24.746489 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG dbcounter [-] [87057] Writing DB stats keystone:SELECT=100 {{(pid=87057) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:49:24.782107 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3528e6df-63fb-4ad6-a46f-a02b651234b2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:49:24.783562 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3528e6df-63fb-4ad6-a46f-a02b651234b2 service neutron] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:49:24.802126 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 733/1484] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 1 16:49:24 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 16:49:26.847378 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98f06c8-a241-4b4f-9f84-7635b9d59d20 None None] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:49:26.847950 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98f06c8-a241-4b4f-9f84-7635b9d59d20 None None] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:49:26.847950 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98f06c8-a241-4b4f-9f84-7635b9d59d20 None None] PATH_INFO: `/` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:49:26.849300 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 752/1485] 10.222.0.20 () {54 vars in 926 bytes} [Mon Aug 1 16:49:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 16:49:26.854764 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97aee281-d8fb-4107-b74e-6eb7e53ffa6d None None] REQUEST_METHOD: `POST` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:49:26.855173 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97aee281-d8fb-4107-b74e-6eb7e53ffa6d None None] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:49:26.855285 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97aee281-d8fb-4107-b74e-6eb7e53ffa6d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:49:26.923896 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.auth.core [None req-97aee281-d8fb-4107-b74e-6eb7e53ffa6d None None] MFA Rules not processed for user `1007f8e379b74e169b98fb7c3c0ba5b4`. Rule list: `[]` (Enabled: `True`). {{(pid=87057) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:49:26.925117 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.fernet_utils [None req-97aee281-d8fb-4107-b74e-6eb7e53ffa6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87057) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:49:27.021906 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 734/1486] 10.222.0.20 () {58 vars in 1023 bytes} [Mon Aug 1 16:49:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:49:27.063563 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f98f06c8-a241-4b4f-9f84-7635b9d59d20 None None] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:49:27.063563 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.fernet_utils [None req-f98f06c8-a241-4b4f-9f84-7635b9d59d20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:49:27.167753 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f98f06c8-a241-4b4f-9f84-7635b9d59d20 None None] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:49:27.240435 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG dbcounter [-] [87058] Writing DB stats keystone:SELECT=100 {{(pid=87058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:49:27.242197 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb963744-e8e4-417f-a139-d70f9bb1de71 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d9bd8d649154c979be3744bfd016f95', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b992d6a65df4bf9b27c13192d57c5f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:49:27.242819 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb963744-e8e4-417f-a139-d70f9bb1de71 service neutron] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:49:27.243101 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb963744-e8e4-417f-a139-d70f9bb1de71 service neutron] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:49:27.243396 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb963744-e8e4-417f-a139-d70f9bb1de71 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:49:27.245631 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.fernet_utils [None req-eb963744-e8e4-417f-a139-d70f9bb1de71 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:49:27.354103 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb963744-e8e4-417f-a139-d70f9bb1de71 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:49:27.356110 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb963744-e8e4-417f-a139-d70f9bb1de71 service neutron] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:49:27.374854 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 753/1487] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 1 16:49:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 16:49:29.461501 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5400fb1-9ed6-46a9-a12f-186baa4336d2 None None] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:49:29.462067 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5400fb1-9ed6-46a9-a12f-186baa4336d2 None None] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:49:29.462067 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5400fb1-9ed6-46a9-a12f-186baa4336d2 None None] PATH_INFO: `/` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:49:29.463428 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 735/1488] 10.222.0.20 () {54 vars in 926 bytes} [Mon Aug 1 16:49:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 16:49:29.469403 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060fd0d5-4535-4c4e-a54f-267aae52b4b4 None None] REQUEST_METHOD: `POST` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:49:29.469845 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060fd0d5-4535-4c4e-a54f-267aae52b4b4 None None] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:49:29.469941 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060fd0d5-4535-4c4e-a54f-267aae52b4b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:49:29.536147 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.auth.core [None req-060fd0d5-4535-4c4e-a54f-267aae52b4b4 None None] MFA Rules not processed for user `1007f8e379b74e169b98fb7c3c0ba5b4`. Rule list: `[]` (Enabled: `True`). {{(pid=87058) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:49:29.537352 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.fernet_utils [None req-060fd0d5-4535-4c4e-a54f-267aae52b4b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:49:29.590225 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG dbcounter [-] [87058] Writing DB stats keystone:SELECT=100 {{(pid=87058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:49:29.633402 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 754/1489] 10.222.0.20 () {58 vars in 1023 bytes} [Mon Aug 1 16:49:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:49:29.673813 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5400fb1-9ed6-46a9-a12f-186baa4336d2 None None] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:49:29.675432 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.fernet_utils [None req-a5400fb1-9ed6-46a9-a12f-186baa4336d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87057) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:49:29.711268 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG dbcounter [-] [87057] Writing DB stats keystone:SELECT=100 {{(pid=87057) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:49:29.780302 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5400fb1-9ed6-46a9-a12f-186baa4336d2 None None] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:49:29.853117 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb7df1c-5f14-44f0-9cb8-324f9b464f8c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d9bd8d649154c979be3744bfd016f95', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b992d6a65df4bf9b27c13192d57c5f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:49:29.853725 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb7df1c-5f14-44f0-9cb8-324f9b464f8c service neutron] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:49:29.854002 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb7df1c-5f14-44f0-9cb8-324f9b464f8c service neutron] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:49:29.854317 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb7df1c-5f14-44f0-9cb8-324f9b464f8c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:49:29.856234 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.fernet_utils [None req-0cb7df1c-5f14-44f0-9cb8-324f9b464f8c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87057) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:49:29.937233 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG dbcounter [-] [87057] Writing DB stats keystone:SELECT=100 {{(pid=87057) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:49:29.966180 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cb7df1c-5f14-44f0-9cb8-324f9b464f8c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:49:29.968313 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cb7df1c-5f14-44f0-9cb8-324f9b464f8c service neutron] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:49:29.986955 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 736/1490] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 1 16:49:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 16:49:32.481606 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9443c932-cd63-44aa-bc16-f1cc9d2d7d61 None None] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:49:32.482263 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9443c932-cd63-44aa-bc16-f1cc9d2d7d61 None None] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:49:32.482263 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9443c932-cd63-44aa-bc16-f1cc9d2d7d61 None None] PATH_INFO: `/` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:49:32.483571 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 755/1491] 10.222.0.20 () {54 vars in 926 bytes} [Mon Aug 1 16:49:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 01 16:49:32.488937 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e11ff1-f94b-472e-a4aa-f70f4357e093 None None] REQUEST_METHOD: `POST` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:49:32.489380 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e11ff1-f94b-472e-a4aa-f70f4357e093 None None] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:49:32.489475 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e11ff1-f94b-472e-a4aa-f70f4357e093 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:49:32.559698 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.auth.core [None req-04e11ff1-f94b-472e-a4aa-f70f4357e093 None None] MFA Rules not processed for user `1007f8e379b74e169b98fb7c3c0ba5b4`. Rule list: `[]` (Enabled: `True`). {{(pid=87057) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:49:32.561018 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.fernet_utils [None req-04e11ff1-f94b-472e-a4aa-f70f4357e093 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87057) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:49:32.657737 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 737/1492] 10.222.0.20 () {58 vars in 1023 bytes} [Mon Aug 1 16:49:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:49:32.697815 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9443c932-cd63-44aa-bc16-f1cc9d2d7d61 None None] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:49:32.699408 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.fernet_utils [None req-9443c932-cd63-44aa-bc16-f1cc9d2d7d61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:49:32.803902 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9443c932-cd63-44aa-bc16-f1cc9d2d7d61 None None] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:49:32.877548 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7acc1702-68e1-4455-b028-cf5dac46e460 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d9bd8d649154c979be3744bfd016f95', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b992d6a65df4bf9b27c13192d57c5f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:49:32.878461 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7acc1702-68e1-4455-b028-cf5dac46e460 service neutron] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:49:32.878742 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7acc1702-68e1-4455-b028-cf5dac46e460 service neutron] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:49:32.879015 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7acc1702-68e1-4455-b028-cf5dac46e460 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:49:32.880648 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.fernet_utils [None req-7acc1702-68e1-4455-b028-cf5dac46e460 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:49:32.890438 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG dbcounter [-] [87058] Writing DB stats keystone:SELECT=100 {{(pid=87058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:49:32.992263 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7acc1702-68e1-4455-b028-cf5dac46e460 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:49:32.993698 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7acc1702-68e1-4455-b028-cf5dac46e460 service neutron] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:49:33.013179 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 756/1493] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 1 16:49:32 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 16:49:42.655947 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG dbcounter [-] [87057] Writing DB stats keystone:SELECT=82 {{(pid=87057) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:49:43.010648 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG dbcounter [-] [87058] Writing DB stats keystone:SELECT=50 {{(pid=87058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:51:08.435620 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416e8d47-90b0-4fef-9f22-dd7c170dc904 None None] REQUEST_METHOD: `POST` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:51:08.436224 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416e8d47-90b0-4fef-9f22-dd7c170dc904 None None] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:51:08.436224 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416e8d47-90b0-4fef-9f22-dd7c170dc904 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:51:08.513183 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.auth.core [None req-416e8d47-90b0-4fef-9f22-dd7c170dc904 None None] MFA Rules not processed for user `1007f8e379b74e169b98fb7c3c0ba5b4`. Rule list: `[]` (Enabled: `True`). {{(pid=87057) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:51:08.514435 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.fernet_utils [None req-416e8d47-90b0-4fef-9f22-dd7c170dc904 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87057) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:51:08.611316 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 738/1494] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 1 16:51:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:51:08.619424 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f094887-5b66-4399-8909-3f816045dc43 None None] REQUEST_METHOD: `POST` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:51:08.619938 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f094887-5b66-4399-8909-3f816045dc43 None None] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:51:08.620046 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f094887-5b66-4399-8909-3f816045dc43 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:51:08.704962 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.auth.core [None req-9f094887-5b66-4399-8909-3f816045dc43 None None] MFA Rules not processed for user `1007f8e379b74e169b98fb7c3c0ba5b4`. Rule list: `[]` (Enabled: `True`). {{(pid=87058) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:51:08.704962 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.fernet_utils [None req-9f094887-5b66-4399-8909-3f816045dc43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:51:08.794690 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 757/1495] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 1 16:51:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:51:08.799377 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-416e8d47-90b0-4fef-9f22-dd7c170dc904 None None] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:51:08.800578 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.fernet_utils [None req-416e8d47-90b0-4fef-9f22-dd7c170dc904 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87057) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:51:08.811978 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-416e8d47-90b0-4fef-9f22-dd7c170dc904 None None] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:51:08.872263 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG dbcounter [-] [87057] Writing DB stats keystone:SELECT=100 {{(pid=87057) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:51:08.906896 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2c2a668-2747-495d-9f12-a3d9b77e32e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:51:08.907353 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c2a668-2747-495d-9f12-a3d9b77e32e2 admin admin] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:51:08.907694 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c2a668-2747-495d-9f12-a3d9b77e32e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:51:08.907969 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c2a668-2747-495d-9f12-a3d9b77e32e2 admin admin] PATH_INFO: `/v3/domains` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:51:08.908663 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2c2a668-2747-495d-9f12-a3d9b77e32e2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87057) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:51:08.908963 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2c2a668-2747-495d-9f12-a3d9b77e32e2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:51:08.910167 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: WARNING py.warnings [None req-b2c2a668-2747-495d-9f12-a3d9b77e32e2 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 01 16:51:08.910167 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: warnings.warn(msg) Aug 01 16:51:08.910167 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: Aug 01 16:51:08.911059 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2c2a668-2747-495d-9f12-a3d9b77e32e2 admin admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:51:08.922334 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 739/1496] 10.222.0.20 () {58 vars in 1153 bytes} [Mon Aug 1 16:51:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 123 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:51:08.930622 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f094887-5b66-4399-8909-3f816045dc43 None None] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:51:08.939804 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f094887-5b66-4399-8909-3f816045dc43 None None] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:51:09.000872 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG dbcounter [-] [87058] Writing DB stats keystone:SELECT=100 {{(pid=87058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:51:09.035050 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c91ef560-6650-489b-accb-f1ecb5c81500 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:51:09.035665 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91ef560-6650-489b-accb-f1ecb5c81500 admin admin] REQUEST_METHOD: `POST` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:51:09.035946 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91ef560-6650-489b-accb-f1ecb5c81500 admin admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:51:09.036219 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91ef560-6650-489b-accb-f1ecb5c81500 admin admin] PATH_INFO: `/v3/projects` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:51:09.036897 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c91ef560-6650-489b-accb-f1ecb5c81500 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:51:09.038713 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: WARNING py.warnings [None req-c91ef560-6650-489b-accb-f1ecb5c81500 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 01 16:51:09.038713 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: warnings.warn(msg) Aug 01 16:51:09.038713 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: Aug 01 16:51:09.039912 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c91ef560-6650-489b-accb-f1ecb5c81500 admin admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:51:09.153683 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 758/1497] 10.222.0.20 () {60 vars in 1153 bytes} [Mon Aug 1 16:51:08 2022] POST /identity/v3/projects => generated 379 bytes in 224 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 16:51:09.158130 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2c2a668-2747-495d-9f12-a3d9b77e32e2 admin admin] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:51:09.159654 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.fernet_utils [None req-b2c2a668-2747-495d-9f12-a3d9b77e32e2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87057) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:51:09.193360 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2c2a668-2747-495d-9f12-a3d9b77e32e2 admin admin] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:51:09.287890 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5ed1ed8-6d44-4174-bb76-01276f7d7dfe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:51:09.288507 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ed1ed8-6d44-4174-bb76-01276f7d7dfe admin admin] REQUEST_METHOD: `POST` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:51:09.289523 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ed1ed8-6d44-4174-bb76-01276f7d7dfe admin admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:51:09.289862 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ed1ed8-6d44-4174-bb76-01276f7d7dfe admin admin] PATH_INFO: `/v3/users` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:51:09.290575 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5ed1ed8-6d44-4174-bb76-01276f7d7dfe admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:51:09.291610 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: WARNING py.warnings [None req-a5ed1ed8-6d44-4174-bb76-01276f7d7dfe admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 01 16:51:09.291610 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: warnings.warn(msg) Aug 01 16:51:09.291610 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: Aug 01 16:51:09.292830 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5ed1ed8-6d44-4174-bb76-01276f7d7dfe admin admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:51:09.399148 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 740/1498] 10.222.0.20 () {60 vars in 1144 bytes} [Mon Aug 1 16:51:09 2022] POST /identity/v3/users => generated 290 bytes in 241 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 16:51:09.403531 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c91ef560-6650-489b-accb-f1ecb5c81500 admin admin] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:51:09.405076 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.fernet_utils [None req-c91ef560-6650-489b-accb-f1ecb5c81500 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:51:09.467269 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c91ef560-6650-489b-accb-f1ecb5c81500 admin admin] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:51:09.551909 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG dbcounter [-] [87058] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:51:09.562056 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b335b4-a114-4fc7-b703-c6185cad473e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:51:09.564996 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b335b4-a114-4fc7-b703-c6185cad473e admin admin] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:51:09.564996 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b335b4-a114-4fc7-b703-c6185cad473e admin admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:51:09.564996 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b335b4-a114-4fc7-b703-c6185cad473e admin admin] PATH_INFO: `/v3/roles` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:51:09.564996 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85b335b4-a114-4fc7-b703-c6185cad473e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:51:09.564996 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: WARNING py.warnings [None req-85b335b4-a114-4fc7-b703-c6185cad473e admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 01 16:51:09.564996 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: warnings.warn(msg) Aug 01 16:51:09.564996 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: Aug 01 16:51:09.565680 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85b335b4-a114-4fc7-b703-c6185cad473e admin admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:51:09.577876 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 759/1499] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Aug 1 16:51:09 2022] GET /identity/v3/roles => generated 1420 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 16:51:09.582238 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5ed1ed8-6d44-4174-bb76-01276f7d7dfe admin admin] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:51:09.583766 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.fernet_utils [None req-a5ed1ed8-6d44-4174-bb76-01276f7d7dfe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87057) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:51:09.595613 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5ed1ed8-6d44-4174-bb76-01276f7d7dfe admin admin] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:51:09.620051 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG dbcounter [-] [87057] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87057) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:51:09.688898 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4dbe58f-7e1c-4867-adc0-723aed317b57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:51:09.689516 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4dbe58f-7e1c-4867-adc0-723aed317b57 admin admin] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:51:09.689797 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4dbe58f-7e1c-4867-adc0-723aed317b57 admin admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:51:09.690099 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4dbe58f-7e1c-4867-adc0-723aed317b57 admin admin] PATH_INFO: `/v3/roles` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:51:09.690917 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4dbe58f-7e1c-4867-adc0-723aed317b57 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:51:09.691932 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: WARNING py.warnings [None req-b4dbe58f-7e1c-4867-adc0-723aed317b57 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 01 16:51:09.691932 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: warnings.warn(msg) Aug 01 16:51:09.691932 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: Aug 01 16:51:09.692826 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4dbe58f-7e1c-4867-adc0-723aed317b57 admin admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:51:09.706428 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 741/1500] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Aug 1 16:51:09 2022] GET /identity/v3/roles => generated 1420 bytes in 125 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 16:51:09.710727 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b335b4-a114-4fc7-b703-c6185cad473e admin admin] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:51:09.712815 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.fernet_utils [None req-85b335b4-a114-4fc7-b703-c6185cad473e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:51:09.746640 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b335b4-a114-4fc7-b703-c6185cad473e admin admin] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:51:09.845321 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f83b2e37-0c70-4645-8385-2260ad87da4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:51:09.845921 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83b2e37-0c70-4645-8385-2260ad87da4d admin admin] REQUEST_METHOD: `PUT` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:51:09.846229 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83b2e37-0c70-4645-8385-2260ad87da4d admin admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:51:09.846508 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83b2e37-0c70-4645-8385-2260ad87da4d admin admin] PATH_INFO: `/v3/projects/222e09bc3c7744518b6d0e3bdb55f22d/users/baabeaf5688b46d9b29e9c41818d143f/roles/0999d3da63a545f18ff3252afdc90518` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:51:09.868434 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f83b2e37-0c70-4645-8385-2260ad87da4d admin admin] RBAC: Authorizing `identity:create_grant(project_id=222e09bc3c7744518b6d0e3bdb55f22d, user_id=baabeaf5688b46d9b29e9c41818d143f, role_id=0999d3da63a545f18ff3252afdc90518)` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:51:09.869468 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: WARNING py.warnings [None req-f83b2e37-0c70-4645-8385-2260ad87da4d admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 01 16:51:09.869468 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: warnings.warn(msg) Aug 01 16:51:09.869468 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: Aug 01 16:51:09.871479 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f83b2e37-0c70-4645-8385-2260ad87da4d admin admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:51:10.040082 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 760/1501] 10.222.0.20 () {60 vars in 1483 bytes} [Mon Aug 1 16:51:09 2022] PUT /identity/v3/projects/222e09bc3c7744518b6d0e3bdb55f22d/users/baabeaf5688b46d9b29e9c41818d143f/roles/0999d3da63a545f18ff3252afdc90518 => generated 0 bytes in 330 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 16:51:10.052355 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc15136-7feb-4f23-a7c8-5be40f5fcd20 None None] REQUEST_METHOD: `POST` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:51:10.052644 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc15136-7feb-4f23-a7c8-5be40f5fcd20 None None] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:51:10.052917 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc15136-7feb-4f23-a7c8-5be40f5fcd20 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:51:10.108999 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.auth.core [None req-6dc15136-7feb-4f23-a7c8-5be40f5fcd20 None None] MFA Rules not processed for user `baabeaf5688b46d9b29e9c41818d143f`. Rule list: `[]` (Enabled: `True`). {{(pid=87057) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:51:10.110166 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.fernet_utils [None req-6dc15136-7feb-4f23-a7c8-5be40f5fcd20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87057) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:51:10.187556 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 742/1502] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 1 16:51:10 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:51:10.202426 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6560f4a8-e7fc-4f4d-b387-37f2e3b8508f None None] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:51:10.202742 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6560f4a8-e7fc-4f4d-b387-37f2e3b8508f None None] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:51:10.203047 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6560f4a8-e7fc-4f4d-b387-37f2e3b8508f None None] PATH_INFO: `/` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:51:10.204421 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 761/1503] 10.222.0.20 () {54 vars in 964 bytes} [Mon Aug 1 16:51:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 16:51:10.209529 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae795d53-49e1-46c3-9918-29ffbba97715 None None] REQUEST_METHOD: `POST` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:51:10.209827 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae795d53-49e1-46c3-9918-29ffbba97715 None None] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:51:10.210134 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae795d53-49e1-46c3-9918-29ffbba97715 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:51:10.242704 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG dbcounter [-] [87057] Writing DB stats keystone:SELECT=100 {{(pid=87057) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:51:10.288285 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.auth.core [None req-ae795d53-49e1-46c3-9918-29ffbba97715 None None] MFA Rules not processed for user `25970b52ca15448fa1abb1f974d64038`. Rule list: `[]` (Enabled: `True`). {{(pid=87057) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:51:10.290218 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.fernet_utils [None req-ae795d53-49e1-46c3-9918-29ffbba97715 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87057) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:51:10.368756 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 743/1504] 10.222.0.20 () {58 vars in 1061 bytes} [Mon Aug 1 16:51:10 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:51:10.376676 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6560f4a8-e7fc-4f4d-b387-37f2e3b8508f None None] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:51:10.378286 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.fernet_utils [None req-6560f4a8-e7fc-4f4d-b387-37f2e3b8508f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:51:10.428816 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG dbcounter [-] [87058] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:51:10.483562 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6560f4a8-e7fc-4f4d-b387-37f2e3b8508f None None] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:51:10.559805 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9005100f-ba02-4282-a8f2-ff1e128a9e95 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25970b52ca15448fa1abb1f974d64038', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b992d6a65df4bf9b27c13192d57c5f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:51:10.560427 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9005100f-ba02-4282-a8f2-ff1e128a9e95 service cinder] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:51:10.560708 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9005100f-ba02-4282-a8f2-ff1e128a9e95 service cinder] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:51:10.560978 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9005100f-ba02-4282-a8f2-ff1e128a9e95 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:51:10.562628 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.fernet_utils [None req-9005100f-ba02-4282-a8f2-ff1e128a9e95 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:51:10.668873 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9005100f-ba02-4282-a8f2-ff1e128a9e95 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:51:10.670275 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9005100f-ba02-4282-a8f2-ff1e128a9e95 service cinder] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:51:10.673951 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG dbcounter [-] [87058] Writing DB stats keystone:SELECT=100 {{(pid=87058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:51:10.688273 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 762/1505] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 1 16:51:10 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 16:51:10.733762 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3645c6ce-84d0-426e-a587-b79cbab127e6 None None] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:51:10.734192 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3645c6ce-84d0-426e-a587-b79cbab127e6 None None] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:51:10.734345 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3645c6ce-84d0-426e-a587-b79cbab127e6 None None] PATH_INFO: `/` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:51:10.735608 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 744/1506] 10.222.0.20 () {52 vars in 830 bytes} [Mon Aug 1 16:51:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 16:51:10.739266 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9005100f-ba02-4282-a8f2-ff1e128a9e95 service cinder] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:51:10.740489 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.fernet_utils [None req-9005100f-ba02-4282-a8f2-ff1e128a9e95 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:51:10.840915 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9005100f-ba02-4282-a8f2-ff1e128a9e95 service cinder] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:51:10.934815 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b26fb08-8d06-4e55-adf1-e0cb54cbc646 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:51:10.935381 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b26fb08-8d06-4e55-adf1-e0cb54cbc646 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:51:10.935656 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b26fb08-8d06-4e55-adf1-e0cb54cbc646 admin admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:51:10.935924 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b26fb08-8d06-4e55-adf1-e0cb54cbc646 admin admin] PATH_INFO: `/v3/users/baabeaf5688b46d9b29e9c41818d143f` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:51:10.948517 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b26fb08-8d06-4e55-adf1-e0cb54cbc646 admin admin] RBAC: Authorizing `identity:delete_user(user_id=baabeaf5688b46d9b29e9c41818d143f)` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:51:10.949538 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: WARNING py.warnings [None req-6b26fb08-8d06-4e55-adf1-e0cb54cbc646 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 01 16:51:10.949538 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: warnings.warn(msg) Aug 01 16:51:10.949538 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: Aug 01 16:51:10.950778 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b26fb08-8d06-4e55-adf1-e0cb54cbc646 admin admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:51:10.960242 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG dbcounter [-] [87058] Writing DB stats keystone:SELECT=100 {{(pid=87058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:51:11.098057 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.notifications [None req-6b26fb08-8d06-4e55-adf1-e0cb54cbc646 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'baabeaf5688b46d9b29e9c41818d143f'} {{(pid=87058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:51:11.098586 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.notifications [None req-6b26fb08-8d06-4e55-adf1-e0cb54cbc646 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'baabeaf5688b46d9b29e9c41818d143f'} {{(pid=87058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:51:11.099140 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.notifications [None req-6b26fb08-8d06-4e55-adf1-e0cb54cbc646 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'baabeaf5688b46d9b29e9c41818d143f'} {{(pid=87058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:51:11.122816 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.notifications [None req-6b26fb08-8d06-4e55-adf1-e0cb54cbc646 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'baabeaf5688b46d9b29e9c41818d143f'} {{(pid=87058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:51:11.136846 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.notifications [None req-6b26fb08-8d06-4e55-adf1-e0cb54cbc646 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'baabeaf5688b46d9b29e9c41818d143f'} {{(pid=87058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:51:11.198209 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 763/1507] 10.222.0.20 () {58 vars in 1224 bytes} [Mon Aug 1 16:51:10 2022] DELETE /identity/v3/users/baabeaf5688b46d9b29e9c41818d143f => generated 0 bytes in 459 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 16:51:11.202748 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f9b3be-a19f-4067-9d5b-3cd46e51ed2a None None] REQUEST_METHOD: `POST` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:51:11.203038 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f9b3be-a19f-4067-9d5b-3cd46e51ed2a None None] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:51:11.206451 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f9b3be-a19f-4067-9d5b-3cd46e51ed2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:51:11.260342 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.auth.core [None req-91f9b3be-a19f-4067-9d5b-3cd46e51ed2a None None] MFA Rules not processed for user `1007f8e379b74e169b98fb7c3c0ba5b4`. Rule list: `[]` (Enabled: `True`). {{(pid=87057) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:51:11.261562 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.fernet_utils [None req-91f9b3be-a19f-4067-9d5b-3cd46e51ed2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87057) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:51:11.336615 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG dbcounter [-] [87057] Writing DB stats keystone:SELECT=100 {{(pid=87057) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:51:11.349098 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 745/1508] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 1 16:51:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:51:11.359543 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b26fb08-8d06-4e55-adf1-e0cb54cbc646 admin admin] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:51:11.360790 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.fernet_utils [None req-6b26fb08-8d06-4e55-adf1-e0cb54cbc646 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:51:11.459613 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b26fb08-8d06-4e55-adf1-e0cb54cbc646 admin admin] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:51:11.513436 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG dbcounter [-] [87058] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=87058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:51:11.532959 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c437b75-81e2-488a-aa98-d18cbff1f478 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d9bd8d649154c979be3744bfd016f95', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b992d6a65df4bf9b27c13192d57c5f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:51:11.533577 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c437b75-81e2-488a-aa98-d18cbff1f478 service neutron] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:51:11.533852 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c437b75-81e2-488a-aa98-d18cbff1f478 service neutron] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:51:11.534194 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c437b75-81e2-488a-aa98-d18cbff1f478 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:51:11.535754 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.fernet_utils [None req-3c437b75-81e2-488a-aa98-d18cbff1f478 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:51:11.645976 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c437b75-81e2-488a-aa98-d18cbff1f478 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:51:11.647449 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c437b75-81e2-488a-aa98-d18cbff1f478 service neutron] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:51:11.666179 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 764/1509] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 1 16:51:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 16:51:12.087306 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91f9b3be-a19f-4067-9d5b-3cd46e51ed2a None None] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:51:12.088547 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.fernet_utils [None req-91f9b3be-a19f-4067-9d5b-3cd46e51ed2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87057) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:51:12.099997 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91f9b3be-a19f-4067-9d5b-3cd46e51ed2a None None] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:51:12.198777 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de663cd9-5cec-4911-a029-c4058e5e1cfd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:51:12.199419 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de663cd9-5cec-4911-a029-c4058e5e1cfd admin admin] REQUEST_METHOD: `DELETE` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:51:12.199700 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de663cd9-5cec-4911-a029-c4058e5e1cfd admin admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:51:12.199975 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de663cd9-5cec-4911-a029-c4058e5e1cfd admin admin] PATH_INFO: `/v3/projects/222e09bc3c7744518b6d0e3bdb55f22d` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:51:12.209892 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de663cd9-5cec-4911-a029-c4058e5e1cfd admin admin] RBAC: Authorizing `identity:delete_project(project_id=222e09bc3c7744518b6d0e3bdb55f22d)` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:51:12.211542 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: WARNING py.warnings [None req-de663cd9-5cec-4911-a029-c4058e5e1cfd admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 01 16:51:12.211542 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: warnings.warn(msg) Aug 01 16:51:12.211542 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: Aug 01 16:51:12.212814 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de663cd9-5cec-4911-a029-c4058e5e1cfd admin admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:51:12.312913 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.notifications [None req-de663cd9-5cec-4911-a029-c4058e5e1cfd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '222e09bc3c7744518b6d0e3bdb55f22d', 'request_id': 'req-de663cd9-5cec-4911-a029-c4058e5e1cfd'} {{(pid=87057) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:51:12.326622 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.notifications [None req-de663cd9-5cec-4911-a029-c4058e5e1cfd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '222e09bc3c7744518b6d0e3bdb55f22d', 'request_id': 'req-de663cd9-5cec-4911-a029-c4058e5e1cfd'} {{(pid=87057) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:51:12.332587 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.notifications [None req-de663cd9-5cec-4911-a029-c4058e5e1cfd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '222e09bc3c7744518b6d0e3bdb55f22d', 'request_id': 'req-de663cd9-5cec-4911-a029-c4058e5e1cfd'} {{(pid=87057) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:51:12.409771 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.notifications [None req-de663cd9-5cec-4911-a029-c4058e5e1cfd admin admin] The token cache is being invalidate because project 222e09bc3c7744518b6d0e3bdb55f22d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87057) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 16:51:12.410071 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.notifications [None req-de663cd9-5cec-4911-a029-c4058e5e1cfd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87057) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:51:12.410499 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.notifications [None req-de663cd9-5cec-4911-a029-c4058e5e1cfd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87057) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:51:12.411941 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 746/1510] 10.222.0.20 () {58 vars in 1233 bytes} [Mon Aug 1 16:51:12 2022] DELETE /identity/v3/projects/222e09bc3c7744518b6d0e3bdb55f22d => generated 0 bytes in 326 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 16:51:21.663295 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG dbcounter [-] [87058] Writing DB stats keystone:SELECT=62 {{(pid=87058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:51:22.407748 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG dbcounter [-] [87057] Writing DB stats keystone:SELECT=84,keystone:DELETE=7,keystone:INSERT=1 {{(pid=87057) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:52:54.662391 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117dd367-ba88-440a-90a6-2ec8e3e16414 None None] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:52:54.662998 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117dd367-ba88-440a-90a6-2ec8e3e16414 None None] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:52:54.662998 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117dd367-ba88-440a-90a6-2ec8e3e16414 None None] PATH_INFO: `/` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:52:54.664355 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 765/1511] 127.0.0.1 () {54 vars in 901 bytes} [Mon Aug 1 16:52:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 16:52:54.669592 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46998bb-7d68-47f8-a790-000cbf0f3759 None None] REQUEST_METHOD: `POST` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:52:54.670063 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46998bb-7d68-47f8-a790-000cbf0f3759 None None] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:52:54.670190 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46998bb-7d68-47f8-a790-000cbf0f3759 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:52:54.750312 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.auth.core [None req-f46998bb-7d68-47f8-a790-000cbf0f3759 None None] MFA Rules not processed for user `1007f8e379b74e169b98fb7c3c0ba5b4`. Rule list: `[]` (Enabled: `True`). {{(pid=87057) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:52:54.751527 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.fernet_utils [None req-f46998bb-7d68-47f8-a790-000cbf0f3759 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87057) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:52:54.864139 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 747/1512] 127.0.0.1 () {58 vars in 998 bytes} [Mon Aug 1 16:52:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:52:55.023305 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-117dd367-ba88-440a-90a6-2ec8e3e16414 None None] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:52:55.029470 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.fernet_utils [None req-117dd367-ba88-440a-90a6-2ec8e3e16414 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:52:55.144384 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-117dd367-ba88-440a-90a6-2ec8e3e16414 None None] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:52:55.149971 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af75f58-df8a-4c4a-b56d-a37cd86e4abe service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6533106d03eb4ca185a8839336d2aff9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b992d6a65df4bf9b27c13192d57c5f0', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:52:55.150618 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af75f58-df8a-4c4a-b56d-a37cd86e4abe service nova] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:52:55.150936 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af75f58-df8a-4c4a-b56d-a37cd86e4abe service nova] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:52:55.151955 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af75f58-df8a-4c4a-b56d-a37cd86e4abe service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:52:55.160382 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5af75f58-df8a-4c4a-b56d-a37cd86e4abe service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:52:55.161958 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5af75f58-df8a-4c4a-b56d-a37cd86e4abe service nova] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:52:55.171589 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 766/1513] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 1 16:52:55 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 149 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 16:52:56.002570 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7d647c-6322-40e9-8ef5-3c1f46c135e6 None None] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:52:56.003120 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7d647c-6322-40e9-8ef5-3c1f46c135e6 None None] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:52:56.003218 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7d647c-6322-40e9-8ef5-3c1f46c135e6 None None] PATH_INFO: `/` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:52:56.004493 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 748/1514] 127.0.0.1 () {54 vars in 901 bytes} [Mon Aug 1 16:52:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 16:52:56.009511 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00faa48-dbfe-462f-8eef-b8a5931d0781 None None] REQUEST_METHOD: `POST` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:52:56.009935 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00faa48-dbfe-462f-8eef-b8a5931d0781 None None] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:52:56.010104 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00faa48-dbfe-462f-8eef-b8a5931d0781 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:52:56.030083 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.auth.core [None req-c00faa48-dbfe-462f-8eef-b8a5931d0781 None None] MFA Rules not processed for user `1007f8e379b74e169b98fb7c3c0ba5b4`. Rule list: `[]` (Enabled: `True`). {{(pid=87058) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:52:56.031268 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.fernet_utils [None req-c00faa48-dbfe-462f-8eef-b8a5931d0781 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:52:56.039001 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 767/1515] 127.0.0.1 () {58 vars in 998 bytes} [Mon Aug 1 16:52:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 16:52:56.192632 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa7d647c-6322-40e9-8ef5-3c1f46c135e6 None None] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:52:56.207320 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa7d647c-6322-40e9-8ef5-3c1f46c135e6 None None] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:52:56.212639 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5745d38a-dbe5-49d5-94e2-cec58b2f8f29 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6533106d03eb4ca185a8839336d2aff9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b992d6a65df4bf9b27c13192d57c5f0', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:52:56.213238 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5745d38a-dbe5-49d5-94e2-cec58b2f8f29 service nova] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:52:56.214144 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5745d38a-dbe5-49d5-94e2-cec58b2f8f29 service nova] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:52:56.214400 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5745d38a-dbe5-49d5-94e2-cec58b2f8f29 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:52:56.222465 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5745d38a-dbe5-49d5-94e2-cec58b2f8f29 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:52:56.224035 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5745d38a-dbe5-49d5-94e2-cec58b2f8f29 service nova] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:52:56.234004 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 749/1516] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 1 16:52:56 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 16:52:57.576255 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c5a8e1-fbb7-49d1-be9a-c2c4cc97914f None None] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:52:57.576830 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c5a8e1-fbb7-49d1-be9a-c2c4cc97914f None None] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:52:57.576830 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c5a8e1-fbb7-49d1-be9a-c2c4cc97914f None None] PATH_INFO: `/v3` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:52:57.578190 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 768/1517] 10.222.0.128 () {54 vars in 926 bytes} [Mon Aug 1 16:52:57 2022] GET /identity/v3 => generated 255 bytes in 3 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:52:57.584235 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2e1b86-f820-4c12-8535-e25c259ebb1a None None] REQUEST_METHOD: `POST` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:52:57.584733 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2e1b86-f820-4c12-8535-e25c259ebb1a None None] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:52:57.584876 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2e1b86-f820-4c12-8535-e25c259ebb1a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:52:57.643894 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.auth.core [None req-9c2e1b86-f820-4c12-8535-e25c259ebb1a None None] MFA Rules not processed for user `8362293261614623bcdabc4120a97a9d`. Rule list: `[]` (Enabled: `True`). {{(pid=87057) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:52:57.645107 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.fernet_utils [None req-9c2e1b86-f820-4c12-8535-e25c259ebb1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87057) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:52:57.662245 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG dbcounter [-] [87057] Writing DB stats keystone:SELECT=100 {{(pid=87057) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:52:57.717132 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 750/1518] 10.222.0.128 () {58 vars in 1016 bytes} [Mon Aug 1 16:52:57 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 16:53:01.810048 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa0e55c-656c-4926-940c-11ff32b48e8b None None] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:53:01.810580 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa0e55c-656c-4926-940c-11ff32b48e8b None None] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:53:01.810681 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa0e55c-656c-4926-940c-11ff32b48e8b None None] PATH_INFO: `/` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:53:01.811999 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 769/1519] 10.222.0.20 () {54 vars in 961 bytes} [Mon Aug 1 16:53:01 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 16:53:01.817297 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c75df30-8530-43fb-8de6-92dd28256e52 None None] REQUEST_METHOD: `POST` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:53:01.817705 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c75df30-8530-43fb-8de6-92dd28256e52 None None] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:53:01.817875 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c75df30-8530-43fb-8de6-92dd28256e52 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:53:01.838253 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.auth.core [None req-1c75df30-8530-43fb-8de6-92dd28256e52 None None] MFA Rules not processed for user `8362293261614623bcdabc4120a97a9d`. Rule list: `[]` (Enabled: `True`). {{(pid=87057) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:53:01.839452 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.fernet_utils [None req-1c75df30-8530-43fb-8de6-92dd28256e52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87057) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:53:01.847943 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 751/1520] 10.222.0.20 () {58 vars in 1058 bytes} [Mon Aug 1 16:53:01 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:53:01.855935 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faa0e55c-656c-4926-940c-11ff32b48e8b None None] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:53:01.870486 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faa0e55c-656c-4926-940c-11ff32b48e8b None None] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:53:01.876409 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83bb8743-c6fd-4a06-98a2-4cfdd1d11110 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8362293261614623bcdabc4120a97a9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b992d6a65df4bf9b27c13192d57c5f0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:53:01.877006 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83bb8743-c6fd-4a06-98a2-4cfdd1d11110 service placement] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:53:01.877283 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83bb8743-c6fd-4a06-98a2-4cfdd1d11110 service placement] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:53:01.877553 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83bb8743-c6fd-4a06-98a2-4cfdd1d11110 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:53:01.885578 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83bb8743-c6fd-4a06-98a2-4cfdd1d11110 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:53:01.887329 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83bb8743-c6fd-4a06-98a2-4cfdd1d11110 service placement] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:53:01.896803 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 770/1521] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 1 16:53:01 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 01 16:53:07.447452 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbfc0766-f6c8-40aa-9206-7f3e9200aaa2 None None] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:53:07.447985 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbfc0766-f6c8-40aa-9206-7f3e9200aaa2 None None] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:53:07.447985 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbfc0766-f6c8-40aa-9206-7f3e9200aaa2 None None] PATH_INFO: `/` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:53:07.449465 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 752/1522] 127.0.0.1 () {54 vars in 901 bytes} [Mon Aug 1 16:53:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 16:53:07.454275 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01476b1-a927-48bc-b545-c176cd85be83 None None] REQUEST_METHOD: `POST` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:53:07.454834 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01476b1-a927-48bc-b545-c176cd85be83 None None] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:53:07.454834 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01476b1-a927-48bc-b545-c176cd85be83 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:53:07.474650 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.auth.core [None req-e01476b1-a927-48bc-b545-c176cd85be83 None None] MFA Rules not processed for user `1007f8e379b74e169b98fb7c3c0ba5b4`. Rule list: `[]` (Enabled: `True`). {{(pid=87058) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:53:07.475857 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.fernet_utils [None req-e01476b1-a927-48bc-b545-c176cd85be83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:53:07.484483 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 771/1523] 127.0.0.1 () {58 vars in 998 bytes} [Mon Aug 1 16:53:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:53:07.640705 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbfc0766-f6c8-40aa-9206-7f3e9200aaa2 None None] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:53:07.655772 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbfc0766-f6c8-40aa-9206-7f3e9200aaa2 None None] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:53:07.661182 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-604c7de8-587d-4467-9b3a-25af3f153622 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6533106d03eb4ca185a8839336d2aff9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b992d6a65df4bf9b27c13192d57c5f0', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:53:07.661784 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604c7de8-587d-4467-9b3a-25af3f153622 service nova] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:53:07.662087 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604c7de8-587d-4467-9b3a-25af3f153622 service nova] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:53:07.662361 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604c7de8-587d-4467-9b3a-25af3f153622 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:53:07.670404 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-604c7de8-587d-4467-9b3a-25af3f153622 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:53:07.671989 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-604c7de8-587d-4467-9b3a-25af3f153622 service nova] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:53:07.681439 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 753/1524] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 1 16:53:07 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 16:53:08.502170 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be765da7-d088-4fc5-8c5c-6edaa4e5d6f2 None None] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:53:08.502717 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be765da7-d088-4fc5-8c5c-6edaa4e5d6f2 None None] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:53:08.502717 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be765da7-d088-4fc5-8c5c-6edaa4e5d6f2 None None] PATH_INFO: `/` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:53:08.504107 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 772/1525] 127.0.0.1 () {54 vars in 901 bytes} [Mon Aug 1 16:53:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 16:53:08.508795 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd138d37-84d0-4ffb-934f-27d9eb0f28c9 None None] REQUEST_METHOD: `POST` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:53:08.509196 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd138d37-84d0-4ffb-934f-27d9eb0f28c9 None None] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:53:08.509337 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd138d37-84d0-4ffb-934f-27d9eb0f28c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:53:08.528402 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.auth.core [None req-fd138d37-84d0-4ffb-934f-27d9eb0f28c9 None None] MFA Rules not processed for user `1007f8e379b74e169b98fb7c3c0ba5b4`. Rule list: `[]` (Enabled: `True`). {{(pid=87057) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:53:08.529604 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.fernet_utils [None req-fd138d37-84d0-4ffb-934f-27d9eb0f28c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87057) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:53:08.537325 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 754/1526] 127.0.0.1 () {58 vars in 998 bytes} [Mon Aug 1 16:53:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:53:08.687793 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be765da7-d088-4fc5-8c5c-6edaa4e5d6f2 None None] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:53:08.702830 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be765da7-d088-4fc5-8c5c-6edaa4e5d6f2 None None] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:53:08.708216 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1c589e7-a1c2-4d30-b327-45f6af4c9f99 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6533106d03eb4ca185a8839336d2aff9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b992d6a65df4bf9b27c13192d57c5f0', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:53:08.708829 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c589e7-a1c2-4d30-b327-45f6af4c9f99 service nova] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:53:08.709109 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c589e7-a1c2-4d30-b327-45f6af4c9f99 service nova] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:53:08.709382 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c589e7-a1c2-4d30-b327-45f6af4c9f99 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:53:08.717710 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1c589e7-a1c2-4d30-b327-45f6af4c9f99 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:53:08.719331 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1c589e7-a1c2-4d30-b327-45f6af4c9f99 service nova] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:53:08.728652 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 773/1527] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 1 16:53:08 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 16:53:12.493277 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e742cddd-da03-46ae-9fff-cdcb23e7f439 None None] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:53:12.493845 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e742cddd-da03-46ae-9fff-cdcb23e7f439 None None] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:53:12.493845 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e742cddd-da03-46ae-9fff-cdcb23e7f439 None None] PATH_INFO: `/` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:53:12.495560 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 755/1528] 127.0.0.1 () {54 vars in 916 bytes} [Mon Aug 1 16:53:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 16:53:12.501127 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0337211-0e50-4607-8502-4985396bde2e None None] REQUEST_METHOD: `POST` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:53:12.501588 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0337211-0e50-4607-8502-4985396bde2e None None] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:53:12.501686 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0337211-0e50-4607-8502-4985396bde2e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:53:12.520944 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.auth.core [None req-e0337211-0e50-4607-8502-4985396bde2e None None] MFA Rules not processed for user `1007f8e379b74e169b98fb7c3c0ba5b4`. Rule list: `[]` (Enabled: `True`). {{(pid=87058) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:53:12.522246 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.fernet_utils [None req-e0337211-0e50-4607-8502-4985396bde2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:53:12.530603 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 774/1529] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Aug 1 16:53:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:53:12.602182 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e742cddd-da03-46ae-9fff-cdcb23e7f439 None None] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:53:12.603870 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.fernet_utils [None req-e742cddd-da03-46ae-9fff-cdcb23e7f439 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87057) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:53:12.683417 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e742cddd-da03-46ae-9fff-cdcb23e7f439 None None] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:53:12.687068 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbcf6144-5dcd-4cf4-a8bd-36925a88f92a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d9bd8d649154c979be3744bfd016f95', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b992d6a65df4bf9b27c13192d57c5f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:53:12.687803 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbcf6144-5dcd-4cf4-a8bd-36925a88f92a service neutron] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:53:12.687957 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbcf6144-5dcd-4cf4-a8bd-36925a88f92a service neutron] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:53:12.688221 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbcf6144-5dcd-4cf4-a8bd-36925a88f92a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:53:12.697183 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbcf6144-5dcd-4cf4-a8bd-36925a88f92a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:53:12.698659 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbcf6144-5dcd-4cf4-a8bd-36925a88f92a service neutron] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:53:12.708009 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 756/1530] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 1 16:53:12 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 16:53:14.711450 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4dfb41-74f5-403c-b583-ba0b8a6200af None None] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:53:14.712029 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4dfb41-74f5-403c-b583-ba0b8a6200af None None] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:53:14.712029 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4dfb41-74f5-403c-b583-ba0b8a6200af None None] PATH_INFO: `/` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:53:14.713372 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 775/1531] 127.0.0.1 () {54 vars in 916 bytes} [Mon Aug 1 16:53:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 16:53:14.719449 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18aea65f-cb84-4da5-8dd6-a34f47bf1d59 None None] REQUEST_METHOD: `POST` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:53:14.719912 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18aea65f-cb84-4da5-8dd6-a34f47bf1d59 None None] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:53:14.720030 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18aea65f-cb84-4da5-8dd6-a34f47bf1d59 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:53:14.740117 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.auth.core [None req-18aea65f-cb84-4da5-8dd6-a34f47bf1d59 None None] MFA Rules not processed for user `1007f8e379b74e169b98fb7c3c0ba5b4`. Rule list: `[]` (Enabled: `True`). {{(pid=87057) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:53:14.741364 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.fernet_utils [None req-18aea65f-cb84-4da5-8dd6-a34f47bf1d59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87057) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:53:14.749662 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 757/1532] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Aug 1 16:53:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:53:14.785421 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce4dfb41-74f5-403c-b583-ba0b8a6200af None None] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:53:14.796733 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce4dfb41-74f5-403c-b583-ba0b8a6200af None None] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:53:14.800262 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf98c783-2807-4c3a-83e5-4eb71e7f9339 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d9bd8d649154c979be3744bfd016f95', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b992d6a65df4bf9b27c13192d57c5f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:53:14.800863 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf98c783-2807-4c3a-83e5-4eb71e7f9339 service neutron] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:53:14.801143 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf98c783-2807-4c3a-83e5-4eb71e7f9339 service neutron] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:53:14.801414 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf98c783-2807-4c3a-83e5-4eb71e7f9339 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:53:14.810329 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf98c783-2807-4c3a-83e5-4eb71e7f9339 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:53:14.811754 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf98c783-2807-4c3a-83e5-4eb71e7f9339 service neutron] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:53:14.821050 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 776/1533] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 1 16:53:14 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 16:53:18.365757 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6179fb27-2440-4828-8738-60f2b809082d None None] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:53:18.366352 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6179fb27-2440-4828-8738-60f2b809082d None None] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:53:18.366352 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6179fb27-2440-4828-8738-60f2b809082d None None] PATH_INFO: `/` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:53:18.367693 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 758/1534] 127.0.0.1 () {54 vars in 916 bytes} [Mon Aug 1 16:53:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 16:53:18.373387 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5ee034-71b3-4eb8-b111-389919cd8058 None None] REQUEST_METHOD: `POST` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:53:18.373847 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5ee034-71b3-4eb8-b111-389919cd8058 None None] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:53:18.373942 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5ee034-71b3-4eb8-b111-389919cd8058 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:53:18.393644 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.auth.core [None req-ec5ee034-71b3-4eb8-b111-389919cd8058 None None] MFA Rules not processed for user `1007f8e379b74e169b98fb7c3c0ba5b4`. Rule list: `[]` (Enabled: `True`). {{(pid=87058) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:53:18.394880 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.fernet_utils [None req-ec5ee034-71b3-4eb8-b111-389919cd8058 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:53:18.402558 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 777/1535] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Aug 1 16:53:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:53:18.703242 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6179fb27-2440-4828-8738-60f2b809082d None None] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:53:18.708573 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.fernet_utils [None req-6179fb27-2440-4828-8738-60f2b809082d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87057) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:53:18.770675 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG dbcounter [-] [87057] Writing DB stats keystone:SELECT=100 {{(pid=87057) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:53:18.784459 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6179fb27-2440-4828-8738-60f2b809082d None None] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:53:18.788073 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4959c0af-e43b-4266-bea8-6c486d04445e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25970b52ca15448fa1abb1f974d64038', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b992d6a65df4bf9b27c13192d57c5f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:53:18.792114 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4959c0af-e43b-4266-bea8-6c486d04445e service cinder] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:53:18.792114 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4959c0af-e43b-4266-bea8-6c486d04445e service cinder] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:53:18.792114 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4959c0af-e43b-4266-bea8-6c486d04445e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:53:18.798820 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4959c0af-e43b-4266-bea8-6c486d04445e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:53:18.800256 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4959c0af-e43b-4266-bea8-6c486d04445e service cinder] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:53:18.809843 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 759/1536] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 1 16:53:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 16:53:20.811842 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13d38be-580b-48bc-8c85-219e5e0017d0 None None] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:53:20.812419 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13d38be-580b-48bc-8c85-219e5e0017d0 None None] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:53:20.812419 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13d38be-580b-48bc-8c85-219e5e0017d0 None None] PATH_INFO: `/` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:53:20.813742 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 778/1537] 127.0.0.1 () {54 vars in 916 bytes} [Mon Aug 1 16:53:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 16:53:20.818844 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c9ae8d-8960-4a61-923e-77406a530d4e None None] REQUEST_METHOD: `POST` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:53:20.819256 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c9ae8d-8960-4a61-923e-77406a530d4e None None] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:53:20.819401 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c9ae8d-8960-4a61-923e-77406a530d4e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:53:20.839001 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.auth.core [None req-40c9ae8d-8960-4a61-923e-77406a530d4e None None] MFA Rules not processed for user `1007f8e379b74e169b98fb7c3c0ba5b4`. Rule list: `[]` (Enabled: `True`). {{(pid=87057) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:53:20.840743 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.fernet_utils [None req-40c9ae8d-8960-4a61-923e-77406a530d4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87057) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:53:20.848655 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 760/1538] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Aug 1 16:53:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:53:21.151733 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d13d38be-580b-48bc-8c85-219e5e0017d0 None None] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:53:21.153504 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.fernet_utils [None req-d13d38be-580b-48bc-8c85-219e5e0017d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:53:21.166462 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d13d38be-580b-48bc-8c85-219e5e0017d0 None None] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:53:21.170977 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a39516b-378c-4840-8f65-42658426988f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25970b52ca15448fa1abb1f974d64038', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b992d6a65df4bf9b27c13192d57c5f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:53:21.171579 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a39516b-378c-4840-8f65-42658426988f service cinder] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:53:21.171873 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a39516b-378c-4840-8f65-42658426988f service cinder] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:53:21.172125 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a39516b-378c-4840-8f65-42658426988f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:53:21.180954 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a39516b-378c-4840-8f65-42658426988f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:53:21.182396 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a39516b-378c-4840-8f65-42658426988f service cinder] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:53:21.192663 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 779/1539] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 1 16:53:21 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 01 16:53:30.834466 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG dbcounter [-] [87057] Writing DB stats keystone:SELECT=11 {{(pid=87057) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:53:31.188440 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG dbcounter [-] [87058] Writing DB stats keystone:SELECT=89 {{(pid=87058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:54:51.839957 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cae18c6-b275-47c1-96f8-9a2032ef074e None None] REQUEST_METHOD: `POST` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:54:51.840646 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cae18c6-b275-47c1-96f8-9a2032ef074e None None] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:54:51.840646 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cae18c6-b275-47c1-96f8-9a2032ef074e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:54:51.865882 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.auth.core [None req-2cae18c6-b275-47c1-96f8-9a2032ef074e None None] MFA Rules not processed for user `1007f8e379b74e169b98fb7c3c0ba5b4`. Rule list: `[]` (Enabled: `True`). {{(pid=87057) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:54:51.867217 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.fernet_utils [None req-2cae18c6-b275-47c1-96f8-9a2032ef074e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87057) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:54:51.881100 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 761/1540] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 1 16:54:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:54:51.891465 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c272255-3761-481b-bb3f-d46a03fb6e05 None None] REQUEST_METHOD: `POST` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:54:51.892081 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c272255-3761-481b-bb3f-d46a03fb6e05 None None] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:54:51.892081 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c272255-3761-481b-bb3f-d46a03fb6e05 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:54:51.918218 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.auth.core [None req-0c272255-3761-481b-bb3f-d46a03fb6e05 None None] MFA Rules not processed for user `1007f8e379b74e169b98fb7c3c0ba5b4`. Rule list: `[]` (Enabled: `True`). {{(pid=87058) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:54:51.919555 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.fernet_utils [None req-0c272255-3761-481b-bb3f-d46a03fb6e05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:54:51.935990 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 780/1541] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 1 16:54:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:54:51.940860 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cae18c6-b275-47c1-96f8-9a2032ef074e None None] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:54:51.955788 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cae18c6-b275-47c1-96f8-9a2032ef074e None None] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:54:51.963808 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6065b25f-c25c-4f96-a260-ac29bc35885e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:54:51.966002 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6065b25f-c25c-4f96-a260-ac29bc35885e admin admin] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:54:51.966314 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6065b25f-c25c-4f96-a260-ac29bc35885e admin admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:54:51.966587 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6065b25f-c25c-4f96-a260-ac29bc35885e admin admin] PATH_INFO: `/v3/domains` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:54:51.967299 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6065b25f-c25c-4f96-a260-ac29bc35885e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87057) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:54:51.967600 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6065b25f-c25c-4f96-a260-ac29bc35885e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:54:51.969202 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6065b25f-c25c-4f96-a260-ac29bc35885e admin admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:54:51.989258 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 762/1542] 10.222.0.20 () {58 vars in 1153 bytes} [Mon Aug 1 16:54:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:54:51.997887 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c272255-3761-481b-bb3f-d46a03fb6e05 None None] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:54:52.007838 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c272255-3761-481b-bb3f-d46a03fb6e05 None None] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:54:52.016194 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5164c366-b72b-41d5-b6ec-5f9d2381667a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:54:52.017113 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5164c366-b72b-41d5-b6ec-5f9d2381667a admin admin] REQUEST_METHOD: `POST` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:54:52.017397 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5164c366-b72b-41d5-b6ec-5f9d2381667a admin admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:54:52.017665 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5164c366-b72b-41d5-b6ec-5f9d2381667a admin admin] PATH_INFO: `/v3/projects` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:54:52.018370 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5164c366-b72b-41d5-b6ec-5f9d2381667a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:54:52.020272 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5164c366-b72b-41d5-b6ec-5f9d2381667a admin admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:54:52.127446 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8e0202-14b4-44cd-9f73-d8885348ee19 None None] REQUEST_METHOD: `POST` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:54:52.127653 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8e0202-14b4-44cd-9f73-d8885348ee19 None None] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:54:52.127936 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8e0202-14b4-44cd-9f73-d8885348ee19 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:54:52.157590 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.auth.core [None req-bf8e0202-14b4-44cd-9f73-d8885348ee19 None None] MFA Rules not processed for user `1007f8e379b74e169b98fb7c3c0ba5b4`. Rule list: `[]` (Enabled: `True`). {{(pid=87057) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:54:52.157590 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.fernet_utils [None req-bf8e0202-14b4-44cd-9f73-d8885348ee19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87057) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:54:52.167025 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 763/1543] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 1 16:54:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 16:54:52.175991 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc0e9f4-e770-4ae8-8fe0-8df6bcfffa18 None None] REQUEST_METHOD: `POST` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:54:52.176330 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc0e9f4-e770-4ae8-8fe0-8df6bcfffa18 None None] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:54:52.176613 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc0e9f4-e770-4ae8-8fe0-8df6bcfffa18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:54:52.201578 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 781/1544] 10.222.0.20 () {60 vars in 1153 bytes} [Mon Aug 1 16:54:51 2022] POST /identity/v3/projects => generated 379 bytes in 203 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 16:54:52.204517 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.auth.core [None req-fcc0e9f4-e770-4ae8-8fe0-8df6bcfffa18 None None] MFA Rules not processed for user `1007f8e379b74e169b98fb7c3c0ba5b4`. Rule list: `[]` (Enabled: `True`). {{(pid=87057) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:54:52.205581 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5164c366-b72b-41d5-b6ec-5f9d2381667a admin admin] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:54:52.211007 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.fernet_utils [None req-fcc0e9f4-e770-4ae8-8fe0-8df6bcfffa18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87057) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:54:52.309392 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 764/1545] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 1 16:54:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 16:54:52.310983 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5641b5b1-99ba-407b-87e1-6452fac78102 None None] REQUEST_METHOD: `POST` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:54:52.311440 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5641b5b1-99ba-407b-87e1-6452fac78102 None None] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:54:52.311932 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5641b5b1-99ba-407b-87e1-6452fac78102 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:54:52.323349 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5164c366-b72b-41d5-b6ec-5f9d2381667a admin admin] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:54:52.338731 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-421d23de-1cf1-4830-b6e1-95458d9fe30b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:54:52.340248 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421d23de-1cf1-4830-b6e1-95458d9fe30b admin admin] REQUEST_METHOD: `POST` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:54:52.340737 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421d23de-1cf1-4830-b6e1-95458d9fe30b admin admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:54:52.341322 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421d23de-1cf1-4830-b6e1-95458d9fe30b admin admin] PATH_INFO: `/v3/users` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:54:52.342223 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-421d23de-1cf1-4830-b6e1-95458d9fe30b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:54:52.342788 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.auth.core [None req-5641b5b1-99ba-407b-87e1-6452fac78102 None None] MFA Rules not processed for user `1007f8e379b74e169b98fb7c3c0ba5b4`. Rule list: `[]` (Enabled: `True`). {{(pid=87057) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:54:52.343782 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: WARNING py.warnings [None req-421d23de-1cf1-4830-b6e1-95458d9fe30b admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 01 16:54:52.343782 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: warnings.warn(msg) Aug 01 16:54:52.343782 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: Aug 01 16:54:52.344191 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.fernet_utils [None req-5641b5b1-99ba-407b-87e1-6452fac78102 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87057) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:54:52.346015 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-421d23de-1cf1-4830-b6e1-95458d9fe30b admin admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:54:52.363007 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 765/1546] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 1 16:54:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 53 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:54:52.364641 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322ca57f-69d8-4039-b7e6-d8f7cf556d74 None None] REQUEST_METHOD: `POST` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:54:52.364990 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322ca57f-69d8-4039-b7e6-d8f7cf556d74 None None] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:54:52.365340 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322ca57f-69d8-4039-b7e6-d8f7cf556d74 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:54:52.388516 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.auth.core [None req-322ca57f-69d8-4039-b7e6-d8f7cf556d74 None None] MFA Rules not processed for user `1007f8e379b74e169b98fb7c3c0ba5b4`. Rule list: `[]` (Enabled: `True`). {{(pid=87057) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:54:52.389661 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.fernet_utils [None req-322ca57f-69d8-4039-b7e6-d8f7cf556d74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87057) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:54:52.398212 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 766/1547] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 1 16:54:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:54:52.399015 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-322ca57f-69d8-4039-b7e6-d8f7cf556d74 None None] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:54:52.409486 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-322ca57f-69d8-4039-b7e6-d8f7cf556d74 None None] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:54:52.414179 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12769cac-a0df-440c-86f4-02d4c1c42227 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:54:52.414787 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12769cac-a0df-440c-86f4-02d4c1c42227 admin admin] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:54:52.415066 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12769cac-a0df-440c-86f4-02d4c1c42227 admin admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:54:52.415338 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12769cac-a0df-440c-86f4-02d4c1c42227 admin admin] PATH_INFO: `/v3/domains` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:54:52.416031 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12769cac-a0df-440c-86f4-02d4c1c42227 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87057) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:54:52.416334 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12769cac-a0df-440c-86f4-02d4c1c42227 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:54:52.418576 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12769cac-a0df-440c-86f4-02d4c1c42227 admin admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:54:52.432006 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 767/1548] 10.222.0.20 () {58 vars in 1153 bytes} [Mon Aug 1 16:54:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:54:52.432006 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8d4e81-e310-4de1-8b72-5765c7339604 None None] REQUEST_METHOD: `POST` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:54:52.432006 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8d4e81-e310-4de1-8b72-5765c7339604 None None] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:54:52.436450 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8d4e81-e310-4de1-8b72-5765c7339604 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:54:52.436588 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 782/1549] 10.222.0.20 () {60 vars in 1144 bytes} [Mon Aug 1 16:54:52 2022] POST /identity/v3/users => generated 290 bytes in 228 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 16:54:52.436588 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7a06d3-2f18-4ab5-9a1a-908b17955a15 None None] REQUEST_METHOD: `POST` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:54:52.436588 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7a06d3-2f18-4ab5-9a1a-908b17955a15 None None] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:54:52.436588 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7a06d3-2f18-4ab5-9a1a-908b17955a15 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:54:52.455502 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.auth.core [None req-7c7a06d3-2f18-4ab5-9a1a-908b17955a15 None None] MFA Rules not processed for user `1007f8e379b74e169b98fb7c3c0ba5b4`. Rule list: `[]` (Enabled: `True`). {{(pid=87058) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:54:52.456726 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.fernet_utils [None req-7c7a06d3-2f18-4ab5-9a1a-908b17955a15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:54:52.457270 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.auth.core [None req-5c8d4e81-e310-4de1-8b72-5765c7339604 None None] MFA Rules not processed for user `1007f8e379b74e169b98fb7c3c0ba5b4`. Rule list: `[]` (Enabled: `True`). {{(pid=87057) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:54:52.458479 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.fernet_utils [None req-5c8d4e81-e310-4de1-8b72-5765c7339604 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87057) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:54:52.465845 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 783/1550] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 1 16:54:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:54:52.473741 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 768/1551] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 1 16:54:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:54:52.473954 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45300be8-b3c3-481f-98ba-453d4701862d None None] REQUEST_METHOD: `POST` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:54:52.473954 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45300be8-b3c3-481f-98ba-453d4701862d None None] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:54:52.473954 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45300be8-b3c3-481f-98ba-453d4701862d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:54:52.474279 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6edf95-78d5-4ea9-9029-3143730cb745 None None] REQUEST_METHOD: `POST` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:54:52.474279 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6edf95-78d5-4ea9-9029-3143730cb745 None None] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:54:52.474279 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6edf95-78d5-4ea9-9029-3143730cb745 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:54:52.488158 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.auth.core [None req-4b6edf95-78d5-4ea9-9029-3143730cb745 None None] MFA Rules not processed for user `1007f8e379b74e169b98fb7c3c0ba5b4`. Rule list: `[]` (Enabled: `True`). {{(pid=87057) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:54:52.489377 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.fernet_utils [None req-4b6edf95-78d5-4ea9-9029-3143730cb745 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87057) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:54:52.491517 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.auth.core [None req-45300be8-b3c3-481f-98ba-453d4701862d None None] MFA Rules not processed for user `1007f8e379b74e169b98fb7c3c0ba5b4`. Rule list: `[]` (Enabled: `True`). {{(pid=87058) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:54:52.492724 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.fernet_utils [None req-45300be8-b3c3-481f-98ba-453d4701862d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:54:52.497625 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 769/1552] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 1 16:54:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:54:52.498851 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ff38f5-147c-4d86-a863-27575061dbd9 None None] REQUEST_METHOD: `POST` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:54:52.499131 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ff38f5-147c-4d86-a863-27575061dbd9 None None] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:54:52.499399 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ff38f5-147c-4d86-a863-27575061dbd9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:54:52.501178 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 784/1553] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 1 16:54:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:54:52.502667 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f15051-702c-4965-a249-1ab3d3bcb7de None None] REQUEST_METHOD: `POST` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:54:52.502956 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f15051-702c-4965-a249-1ab3d3bcb7de None None] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:54:52.503232 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f15051-702c-4965-a249-1ab3d3bcb7de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:54:52.519950 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.auth.core [None req-a1ff38f5-147c-4d86-a863-27575061dbd9 None None] MFA Rules not processed for user `1007f8e379b74e169b98fb7c3c0ba5b4`. Rule list: `[]` (Enabled: `True`). {{(pid=87057) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:54:52.521151 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.fernet_utils [None req-a1ff38f5-147c-4d86-a863-27575061dbd9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87057) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:54:52.522583 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.auth.core [None req-85f15051-702c-4965-a249-1ab3d3bcb7de None None] MFA Rules not processed for user `1007f8e379b74e169b98fb7c3c0ba5b4`. Rule list: `[]` (Enabled: `True`). {{(pid=87058) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:54:52.524622 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.fernet_utils [None req-85f15051-702c-4965-a249-1ab3d3bcb7de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:54:52.529884 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 770/1554] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 1 16:54:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:54:52.531550 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ff38f5-147c-4d86-a863-27575061dbd9 None None] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:54:52.537162 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 785/1555] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 1 16:54:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:54:52.537162 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f15051-702c-4965-a249-1ab3d3bcb7de None None] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:54:52.541049 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ff38f5-147c-4d86-a863-27575061dbd9 None None] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:54:52.546594 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7070c61d-2faa-4f5e-93a2-7a452abc86a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:54:52.547118 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f15051-702c-4965-a249-1ab3d3bcb7de None None] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:54:52.547250 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7070c61d-2faa-4f5e-93a2-7a452abc86a3 admin admin] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:54:52.547466 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7070c61d-2faa-4f5e-93a2-7a452abc86a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:54:52.547751 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7070c61d-2faa-4f5e-93a2-7a452abc86a3 admin admin] PATH_INFO: `/v3/roles` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:54:52.548428 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7070c61d-2faa-4f5e-93a2-7a452abc86a3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:54:52.550050 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7070c61d-2faa-4f5e-93a2-7a452abc86a3 admin admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:54:52.551691 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81cdd29c-8b71-42f8-9523-9fff51d70403 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:54:52.552287 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81cdd29c-8b71-42f8-9523-9fff51d70403 admin admin] REQUEST_METHOD: `POST` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:54:52.552565 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81cdd29c-8b71-42f8-9523-9fff51d70403 admin admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:54:52.552837 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81cdd29c-8b71-42f8-9523-9fff51d70403 admin admin] PATH_INFO: `/v3/projects` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:54:52.553500 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81cdd29c-8b71-42f8-9523-9fff51d70403 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:54:52.555495 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81cdd29c-8b71-42f8-9523-9fff51d70403 admin admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:54:52.560853 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 771/1556] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Aug 1 16:54:52 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 16:54:52.562058 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522d7e52-fe67-43fd-85a5-85465996c7fd None None] REQUEST_METHOD: `POST` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:54:52.562340 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522d7e52-fe67-43fd-85a5-85465996c7fd None None] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:54:52.562612 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522d7e52-fe67-43fd-85a5-85465996c7fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:54:52.581077 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.auth.core [None req-522d7e52-fe67-43fd-85a5-85465996c7fd None None] MFA Rules not processed for user `1007f8e379b74e169b98fb7c3c0ba5b4`. Rule list: `[]` (Enabled: `True`). {{(pid=87057) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:54:52.584909 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.fernet_utils [None req-522d7e52-fe67-43fd-85a5-85465996c7fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87057) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:54:52.590614 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 772/1557] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 1 16:54:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:54:52.591772 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9797e931-c696-4af4-b010-a9cf033b05c0 None None] REQUEST_METHOD: `POST` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:54:52.592052 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9797e931-c696-4af4-b010-a9cf033b05c0 None None] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:54:52.592320 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9797e931-c696-4af4-b010-a9cf033b05c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:54:52.611367 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.auth.core [None req-9797e931-c696-4af4-b010-a9cf033b05c0 None None] MFA Rules not processed for user `1007f8e379b74e169b98fb7c3c0ba5b4`. Rule list: `[]` (Enabled: `True`). {{(pid=87057) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:54:52.613572 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.fernet_utils [None req-9797e931-c696-4af4-b010-a9cf033b05c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87057) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:54:52.621806 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 773/1558] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 1 16:54:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:54:52.622743 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9797e931-c696-4af4-b010-a9cf033b05c0 None None] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:54:52.631849 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9797e931-c696-4af4-b010-a9cf033b05c0 None None] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:54:52.636643 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5937538-f90d-4138-923a-709e41a767b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:54:52.642343 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5937538-f90d-4138-923a-709e41a767b5 admin admin] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:54:52.642343 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5937538-f90d-4138-923a-709e41a767b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:54:52.642343 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5937538-f90d-4138-923a-709e41a767b5 admin admin] PATH_INFO: `/v3/domains` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:54:52.642343 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5937538-f90d-4138-923a-709e41a767b5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87057) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:54:52.642343 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5937538-f90d-4138-923a-709e41a767b5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:54:52.642343 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5937538-f90d-4138-923a-709e41a767b5 admin admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:54:52.652252 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 774/1559] 10.222.0.20 () {58 vars in 1153 bytes} [Mon Aug 1 16:54:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:54:52.653536 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997e00db-4e05-4e85-b560-4d655b4e8ec2 None None] REQUEST_METHOD: `POST` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:54:52.653828 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997e00db-4e05-4e85-b560-4d655b4e8ec2 None None] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:54:52.654141 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997e00db-4e05-4e85-b560-4d655b4e8ec2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:54:52.667673 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 786/1560] 10.222.0.20 () {60 vars in 1153 bytes} [Mon Aug 1 16:54:52 2022] POST /identity/v3/projects => generated 367 bytes in 133 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 16:54:52.669453 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8439e30b-a245-40a3-99e6-c3a9fcf0822c None None] REQUEST_METHOD: `POST` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:54:52.669911 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8439e30b-a245-40a3-99e6-c3a9fcf0822c None None] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:54:52.670209 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8439e30b-a245-40a3-99e6-c3a9fcf0822c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:54:52.674463 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.auth.core [None req-997e00db-4e05-4e85-b560-4d655b4e8ec2 None None] MFA Rules not processed for user `1007f8e379b74e169b98fb7c3c0ba5b4`. Rule list: `[]` (Enabled: `True`). {{(pid=87057) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:54:52.675705 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.fernet_utils [None req-997e00db-4e05-4e85-b560-4d655b4e8ec2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87057) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:54:52.689933 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.auth.core [None req-8439e30b-a245-40a3-99e6-c3a9fcf0822c None None] MFA Rules not processed for user `1007f8e379b74e169b98fb7c3c0ba5b4`. Rule list: `[]` (Enabled: `True`). {{(pid=87058) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:54:52.694019 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.fernet_utils [None req-8439e30b-a245-40a3-99e6-c3a9fcf0822c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:54:52.721081 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG dbcounter [-] [87057] Writing DB stats keystone:SELECT=100 {{(pid=87057) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:54:52.735685 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 775/1561] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 1 16:54:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:54:52.740795 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-997e00db-4e05-4e85-b560-4d655b4e8ec2 None None] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:54:52.746831 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-997e00db-4e05-4e85-b560-4d655b4e8ec2 None None] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:54:52.751363 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b47b6833-a149-4e9d-8b1c-d17cc2be3d0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:54:52.751962 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47b6833-a149-4e9d-8b1c-d17cc2be3d0c admin admin] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:54:52.752329 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47b6833-a149-4e9d-8b1c-d17cc2be3d0c admin admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:54:52.752655 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47b6833-a149-4e9d-8b1c-d17cc2be3d0c admin admin] PATH_INFO: `/v3/domains` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:54:52.753344 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b47b6833-a149-4e9d-8b1c-d17cc2be3d0c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87057) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:54:52.753672 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b47b6833-a149-4e9d-8b1c-d17cc2be3d0c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:54:52.755287 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b47b6833-a149-4e9d-8b1c-d17cc2be3d0c admin admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:54:52.757201 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 787/1562] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 1 16:54:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:54:52.757201 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8439e30b-a245-40a3-99e6-c3a9fcf0822c None None] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:54:52.766573 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8439e30b-a245-40a3-99e6-c3a9fcf0822c None None] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:54:52.769682 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 776/1563] 10.222.0.20 () {58 vars in 1153 bytes} [Mon Aug 1 16:54:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:54:52.770418 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b47b6833-a149-4e9d-8b1c-d17cc2be3d0c admin admin] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:54:52.778591 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-642a5033-8d84-4ebd-8efe-26315d985d96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:54:52.779251 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642a5033-8d84-4ebd-8efe-26315d985d96 admin admin] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:54:52.779508 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642a5033-8d84-4ebd-8efe-26315d985d96 admin admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:54:52.779800 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642a5033-8d84-4ebd-8efe-26315d985d96 admin admin] PATH_INFO: `/v3/roles` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:54:52.779949 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b47b6833-a149-4e9d-8b1c-d17cc2be3d0c admin admin] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:54:52.780501 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-642a5033-8d84-4ebd-8efe-26315d985d96 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:54:52.782141 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-642a5033-8d84-4ebd-8efe-26315d985d96 admin admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:54:52.785044 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83ff3f1c-1ae0-4370-b3b1-fbb163716bb0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:54:52.785759 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ff3f1c-1ae0-4370-b3b1-fbb163716bb0 admin admin] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:54:52.786177 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ff3f1c-1ae0-4370-b3b1-fbb163716bb0 admin admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:54:52.786612 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ff3f1c-1ae0-4370-b3b1-fbb163716bb0 admin admin] PATH_INFO: `/v3/domains` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:54:52.787457 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83ff3f1c-1ae0-4370-b3b1-fbb163716bb0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87057) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:54:52.787705 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG dbcounter [-] [87058] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=87058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:54:52.787829 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83ff3f1c-1ae0-4370-b3b1-fbb163716bb0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:54:52.789563 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83ff3f1c-1ae0-4370-b3b1-fbb163716bb0 admin admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:54:52.793511 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 788/1564] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Aug 1 16:54:52 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 16:54:52.794325 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-642a5033-8d84-4ebd-8efe-26315d985d96 admin admin] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:54:52.803518 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 777/1565] 10.222.0.20 () {58 vars in 1153 bytes} [Mon Aug 1 16:54:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:54:52.803732 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-642a5033-8d84-4ebd-8efe-26315d985d96 admin admin] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:54:52.804379 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83ff3f1c-1ae0-4370-b3b1-fbb163716bb0 admin admin] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:54:52.815387 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83ff3f1c-1ae0-4370-b3b1-fbb163716bb0 admin admin] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:54:52.820013 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46498462-29bd-41d9-8bc2-8aef957ac6da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:54:52.820013 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46498462-29bd-41d9-8bc2-8aef957ac6da admin admin] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:54:52.820013 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46498462-29bd-41d9-8bc2-8aef957ac6da admin admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:54:52.820013 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46498462-29bd-41d9-8bc2-8aef957ac6da admin admin] PATH_INFO: `/v3/domains` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:54:52.820488 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46498462-29bd-41d9-8bc2-8aef957ac6da admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87058) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:54:52.820488 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46498462-29bd-41d9-8bc2-8aef957ac6da admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:54:52.820488 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: WARNING py.warnings [None req-46498462-29bd-41d9-8bc2-8aef957ac6da admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 01 16:54:52.820488 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: warnings.warn(msg) Aug 01 16:54:52.820488 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: Aug 01 16:54:52.820488 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46498462-29bd-41d9-8bc2-8aef957ac6da admin admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:54:52.820954 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be0f5af9-4cf2-43ea-b6bd-7c25028d79ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:54:52.820954 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0f5af9-4cf2-43ea-b6bd-7c25028d79ed admin admin] REQUEST_METHOD: `POST` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:54:52.820954 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0f5af9-4cf2-43ea-b6bd-7c25028d79ed admin admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:54:52.821236 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0f5af9-4cf2-43ea-b6bd-7c25028d79ed admin admin] PATH_INFO: `/v3/projects` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:54:52.821659 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be0f5af9-4cf2-43ea-b6bd-7c25028d79ed admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:54:52.822693 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: WARNING py.warnings [None req-be0f5af9-4cf2-43ea-b6bd-7c25028d79ed admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 01 16:54:52.822693 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: warnings.warn(msg) Aug 01 16:54:52.822693 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: Aug 01 16:54:52.824025 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be0f5af9-4cf2-43ea-b6bd-7c25028d79ed admin admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:54:52.832053 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 789/1566] 10.222.0.20 () {58 vars in 1153 bytes} [Mon Aug 1 16:54:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:54:52.832841 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46498462-29bd-41d9-8bc2-8aef957ac6da admin admin] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:54:52.842914 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46498462-29bd-41d9-8bc2-8aef957ac6da admin admin] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:54:52.848493 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ed68154-8a35-4d90-8833-b9cd76a795d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:54:52.853696 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed68154-8a35-4d90-8833-b9cd76a795d3 admin admin] REQUEST_METHOD: `POST` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:54:52.853696 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed68154-8a35-4d90-8833-b9cd76a795d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:54:52.853696 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed68154-8a35-4d90-8833-b9cd76a795d3 admin admin] PATH_INFO: `/v3/users` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:54:52.853696 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ed68154-8a35-4d90-8833-b9cd76a795d3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:54:52.853696 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ed68154-8a35-4d90-8833-b9cd76a795d3 admin admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:54:52.953456 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 778/1567] 10.222.0.20 () {60 vars in 1153 bytes} [Mon Aug 1 16:54:52 2022] POST /identity/v3/projects => generated 375 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 16:54:52.954325 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be0f5af9-4cf2-43ea-b6bd-7c25028d79ed admin admin] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:54:53.023101 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be0f5af9-4cf2-43ea-b6bd-7c25028d79ed admin admin] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:54:53.028518 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde6fd6f-ff27-4f14-a2a7-351e9c4191b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:54:53.029124 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde6fd6f-ff27-4f14-a2a7-351e9c4191b8 admin admin] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:54:53.029400 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde6fd6f-ff27-4f14-a2a7-351e9c4191b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:54:53.029666 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde6fd6f-ff27-4f14-a2a7-351e9c4191b8 admin admin] PATH_INFO: `/v3/domains` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:54:53.030438 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fde6fd6f-ff27-4f14-a2a7-351e9c4191b8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87057) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:54:53.030738 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fde6fd6f-ff27-4f14-a2a7-351e9c4191b8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:54:53.032377 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fde6fd6f-ff27-4f14-a2a7-351e9c4191b8 admin admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:54:53.044095 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 779/1568] 10.222.0.20 () {58 vars in 1153 bytes} [Mon Aug 1 16:54:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:54:53.044879 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde6fd6f-ff27-4f14-a2a7-351e9c4191b8 admin admin] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:54:53.054128 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde6fd6f-ff27-4f14-a2a7-351e9c4191b8 admin admin] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:54:53.058687 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae8573c-aaa9-4ec7-84a5-e67429627b08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:54:53.059322 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae8573c-aaa9-4ec7-84a5-e67429627b08 admin admin] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:54:53.059601 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae8573c-aaa9-4ec7-84a5-e67429627b08 admin admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:54:53.059872 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae8573c-aaa9-4ec7-84a5-e67429627b08 admin admin] PATH_INFO: `/v3/domains` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:54:53.060543 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ae8573c-aaa9-4ec7-84a5-e67429627b08 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87057) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:54:53.060840 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ae8573c-aaa9-4ec7-84a5-e67429627b08 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:54:53.062417 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ae8573c-aaa9-4ec7-84a5-e67429627b08 admin admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:54:53.074793 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 780/1569] 10.222.0.20 () {58 vars in 1153 bytes} [Mon Aug 1 16:54:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:54:53.076128 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae8573c-aaa9-4ec7-84a5-e67429627b08 admin admin] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:54:53.084625 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae8573c-aaa9-4ec7-84a5-e67429627b08 admin admin] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:54:53.089676 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-559c1431-d0e5-40b2-895d-5ae836b1de3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:54:53.090291 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559c1431-d0e5-40b2-895d-5ae836b1de3f admin admin] REQUEST_METHOD: `POST` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:54:53.090562 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559c1431-d0e5-40b2-895d-5ae836b1de3f admin admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:54:53.090825 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559c1431-d0e5-40b2-895d-5ae836b1de3f admin admin] PATH_INFO: `/v3/projects` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:54:53.091470 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-559c1431-d0e5-40b2-895d-5ae836b1de3f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:54:53.093333 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-559c1431-d0e5-40b2-895d-5ae836b1de3f admin admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:54:53.104244 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 790/1570] 10.222.0.20 () {60 vars in 1144 bytes} [Mon Aug 1 16:54:52 2022] POST /identity/v3/users => generated 284 bytes in 269 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 16:54:53.104244 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ed68154-8a35-4d90-8833-b9cd76a795d3 admin admin] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:54:53.113698 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ed68154-8a35-4d90-8833-b9cd76a795d3 admin admin] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:54:53.118497 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-211e472d-2f47-4a38-bce4-d9647445bd9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:54:53.121760 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211e472d-2f47-4a38-bce4-d9647445bd9b admin admin] REQUEST_METHOD: `PUT` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:54:53.121760 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211e472d-2f47-4a38-bce4-d9647445bd9b admin admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:54:53.121760 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211e472d-2f47-4a38-bce4-d9647445bd9b admin admin] PATH_INFO: `/v3/projects/704a245a8de14350b5ffe3d1d13ca1b2/users/0d90db97f4334c41ba8e0e54fc14c317/roles/0999d3da63a545f18ff3252afdc90518` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:54:53.134441 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-211e472d-2f47-4a38-bce4-d9647445bd9b admin admin] RBAC: Authorizing `identity:create_grant(project_id=704a245a8de14350b5ffe3d1d13ca1b2, user_id=0d90db97f4334c41ba8e0e54fc14c317, role_id=0999d3da63a545f18ff3252afdc90518)` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:54:53.137323 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-211e472d-2f47-4a38-bce4-d9647445bd9b admin admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:54:53.188055 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 781/1571] 10.222.0.20 () {60 vars in 1153 bytes} [Mon Aug 1 16:54:53 2022] POST /identity/v3/projects => generated 383 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 16:54:53.188934 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-559c1431-d0e5-40b2-895d-5ae836b1de3f admin admin] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:54:53.215868 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 791/1572] 10.222.0.20 () {60 vars in 1483 bytes} [Mon Aug 1 16:54:53 2022] PUT /identity/v3/projects/704a245a8de14350b5ffe3d1d13ca1b2/users/0d90db97f4334c41ba8e0e54fc14c317/roles/0999d3da63a545f18ff3252afdc90518 => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 16:54:53.219375 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-211e472d-2f47-4a38-bce4-d9647445bd9b admin admin] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:54:53.251563 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-559c1431-d0e5-40b2-895d-5ae836b1de3f admin admin] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:54:53.256312 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG dbcounter [-] [87057] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=87057) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:54:53.279495 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-211e472d-2f47-4a38-bce4-d9647445bd9b admin admin] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:54:53.284368 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc0ef04-1fd5-4ef8-bceb-5bd2e1a0afb9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:54:53.284957 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc0ef04-1fd5-4ef8-bceb-5bd2e1a0afb9 admin admin] REQUEST_METHOD: `POST` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:54:53.285236 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc0ef04-1fd5-4ef8-bceb-5bd2e1a0afb9 admin admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:54:53.285527 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc0ef04-1fd5-4ef8-bceb-5bd2e1a0afb9 admin admin] PATH_INFO: `/v3/projects` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:54:53.286917 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdc0ef04-1fd5-4ef8-bceb-5bd2e1a0afb9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:54:53.288140 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdc0ef04-1fd5-4ef8-bceb-5bd2e1a0afb9 admin admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:54:53.309319 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a32ecaea-26d7-4aa8-ba10-f46e9e84d28f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:54:53.309942 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a32ecaea-26d7-4aa8-ba10-f46e9e84d28f admin admin] REQUEST_METHOD: `POST` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:54:53.310249 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a32ecaea-26d7-4aa8-ba10-f46e9e84d28f admin admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:54:53.314303 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a32ecaea-26d7-4aa8-ba10-f46e9e84d28f admin admin] PATH_INFO: `/v3/projects` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:54:53.314303 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a32ecaea-26d7-4aa8-ba10-f46e9e84d28f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:54:53.314303 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a32ecaea-26d7-4aa8-ba10-f46e9e84d28f admin admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:54:53.398516 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 792/1573] 10.222.0.20 () {60 vars in 1153 bytes} [Mon Aug 1 16:54:53 2022] POST /identity/v3/projects => generated 381 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 16:54:53.399453 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc0ef04-1fd5-4ef8-bceb-5bd2e1a0afb9 admin admin] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:54:53.427924 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 782/1574] 10.222.0.20 () {60 vars in 1153 bytes} [Mon Aug 1 16:54:53 2022] POST /identity/v3/projects => generated 393 bytes in 239 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 16:54:53.428743 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a32ecaea-26d7-4aa8-ba10-f46e9e84d28f admin admin] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:54:53.462266 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc0ef04-1fd5-4ef8-bceb-5bd2e1a0afb9 admin admin] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:54:53.494459 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a32ecaea-26d7-4aa8-ba10-f46e9e84d28f admin admin] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:54:53.497387 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG dbcounter [-] [87058] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=87058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:54:53.500421 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e78da8-c04d-4a06-92f1-9c4d0350d63e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:54:53.501354 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e78da8-c04d-4a06-92f1-9c4d0350d63e admin admin] REQUEST_METHOD: `POST` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:54:53.501638 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e78da8-c04d-4a06-92f1-9c4d0350d63e admin admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:54:53.501915 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e78da8-c04d-4a06-92f1-9c4d0350d63e admin admin] PATH_INFO: `/v3/projects` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:54:53.502630 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49e78da8-c04d-4a06-92f1-9c4d0350d63e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:54:53.505690 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49e78da8-c04d-4a06-92f1-9c4d0350d63e admin admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:54:53.519657 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90cd5b52-a39f-47b3-8c85-3e2bb05a49fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:54:53.520285 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90cd5b52-a39f-47b3-8c85-3e2bb05a49fe admin admin] REQUEST_METHOD: `POST` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:54:53.520604 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90cd5b52-a39f-47b3-8c85-3e2bb05a49fe admin admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:54:53.520839 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90cd5b52-a39f-47b3-8c85-3e2bb05a49fe admin admin] PATH_INFO: `/v3/users` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:54:53.521508 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90cd5b52-a39f-47b3-8c85-3e2bb05a49fe admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:54:53.523464 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90cd5b52-a39f-47b3-8c85-3e2bb05a49fe admin admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:54:53.587648 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 783/1575] 10.222.0.20 () {60 vars in 1153 bytes} [Mon Aug 1 16:54:53 2022] POST /identity/v3/projects => generated 403 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 16:54:53.588466 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e78da8-c04d-4a06-92f1-9c4d0350d63e admin admin] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:54:53.617218 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 793/1576] 10.222.0.20 () {60 vars in 1144 bytes} [Mon Aug 1 16:54:53 2022] POST /identity/v3/users => generated 288 bytes in 217 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 16:54:53.617218 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90cd5b52-a39f-47b3-8c85-3e2bb05a49fe admin admin] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:54:53.628869 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90cd5b52-a39f-47b3-8c85-3e2bb05a49fe admin admin] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:54:53.654724 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e78da8-c04d-4a06-92f1-9c4d0350d63e admin admin] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:54:53.659878 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d5fa64d-310b-487a-bb26-fd14657ab2b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:54:53.660518 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5fa64d-310b-487a-bb26-fd14657ab2b5 admin admin] REQUEST_METHOD: `POST` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:54:53.660803 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5fa64d-310b-487a-bb26-fd14657ab2b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:54:53.661077 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5fa64d-310b-487a-bb26-fd14657ab2b5 admin admin] PATH_INFO: `/v3/projects` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:54:53.661754 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d5fa64d-310b-487a-bb26-fd14657ab2b5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:54:53.663824 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d5fa64d-310b-487a-bb26-fd14657ab2b5 admin admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:54:53.688670 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-324a581b-6fa2-409c-be93-ccb0fe0d8104 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:54:53.689474 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324a581b-6fa2-409c-be93-ccb0fe0d8104 admin admin] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:54:53.689778 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324a581b-6fa2-409c-be93-ccb0fe0d8104 admin admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:54:53.690079 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324a581b-6fa2-409c-be93-ccb0fe0d8104 admin admin] PATH_INFO: `/v3/roles` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:54:53.690775 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-324a581b-6fa2-409c-be93-ccb0fe0d8104 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:54:53.692379 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-324a581b-6fa2-409c-be93-ccb0fe0d8104 admin admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:54:53.702393 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 794/1577] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Aug 1 16:54:53 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 16:54:53.703291 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-324a581b-6fa2-409c-be93-ccb0fe0d8104 admin admin] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:54:53.716126 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-324a581b-6fa2-409c-be93-ccb0fe0d8104 admin admin] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:54:53.719681 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0075d2c3-9eee-4a65-8b73-22bb3016e4de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:54:53.720341 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0075d2c3-9eee-4a65-8b73-22bb3016e4de admin admin] REQUEST_METHOD: `POST` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:54:53.720601 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0075d2c3-9eee-4a65-8b73-22bb3016e4de admin admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:54:53.721048 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0075d2c3-9eee-4a65-8b73-22bb3016e4de admin admin] PATH_INFO: `/v3/users` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:54:53.721785 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0075d2c3-9eee-4a65-8b73-22bb3016e4de admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:54:53.723861 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0075d2c3-9eee-4a65-8b73-22bb3016e4de admin admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:54:53.754184 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 784/1578] 10.222.0.20 () {60 vars in 1153 bytes} [Mon Aug 1 16:54:53 2022] POST /identity/v3/projects => generated 367 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 16:54:53.759246 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed55f66-359e-4bc3-b5b8-7c8856d2c702 None None] REQUEST_METHOD: `POST` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:54:53.759246 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed55f66-359e-4bc3-b5b8-7c8856d2c702 None None] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:54:53.759246 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed55f66-359e-4bc3-b5b8-7c8856d2c702 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:54:53.764199 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG dbcounter [-] [87057] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87057) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:54:53.774879 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.auth.core [None req-eed55f66-359e-4bc3-b5b8-7c8856d2c702 None None] MFA Rules not processed for user `1007f8e379b74e169b98fb7c3c0ba5b4`. Rule list: `[]` (Enabled: `True`). {{(pid=87057) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:54:53.776081 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.fernet_utils [None req-eed55f66-359e-4bc3-b5b8-7c8856d2c702 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87057) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:54:53.809501 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 795/1579] 10.222.0.20 () {60 vars in 1144 bytes} [Mon Aug 1 16:54:53 2022] POST /identity/v3/users => generated 292 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 16:54:53.810346 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0075d2c3-9eee-4a65-8b73-22bb3016e4de admin admin] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:54:53.819955 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0075d2c3-9eee-4a65-8b73-22bb3016e4de admin admin] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:54:53.843002 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 785/1580] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 1 16:54:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:54:53.843002 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed55f66-359e-4bc3-b5b8-7c8856d2c702 None None] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:54:53.851480 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed55f66-359e-4bc3-b5b8-7c8856d2c702 None None] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:54:53.857599 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f312f7ca-f18b-4f64-8996-a5c6b9206881 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:54:53.858259 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f312f7ca-f18b-4f64-8996-a5c6b9206881 admin admin] REQUEST_METHOD: `POST` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:54:53.858540 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f312f7ca-f18b-4f64-8996-a5c6b9206881 admin admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:54:53.858818 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f312f7ca-f18b-4f64-8996-a5c6b9206881 admin admin] PATH_INFO: `/v3/users` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:54:53.859492 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f312f7ca-f18b-4f64-8996-a5c6b9206881 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:54:53.861450 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f312f7ca-f18b-4f64-8996-a5c6b9206881 admin admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:54:53.880262 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-151cdab1-2a90-40e3-94a3-8b5586db43ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:54:53.880889 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151cdab1-2a90-40e3-94a3-8b5586db43ac admin admin] REQUEST_METHOD: `POST` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:54:53.881168 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151cdab1-2a90-40e3-94a3-8b5586db43ac admin admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:54:53.881440 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151cdab1-2a90-40e3-94a3-8b5586db43ac admin admin] PATH_INFO: `/v3/users` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:54:53.882173 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-151cdab1-2a90-40e3-94a3-8b5586db43ac admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:54:53.884081 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-151cdab1-2a90-40e3-94a3-8b5586db43ac admin admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:54:53.973134 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 786/1581] 10.222.0.20 () {60 vars in 1144 bytes} [Mon Aug 1 16:54:53 2022] POST /identity/v3/users => generated 297 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 16:54:53.973134 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f312f7ca-f18b-4f64-8996-a5c6b9206881 admin admin] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:54:53.981205 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f312f7ca-f18b-4f64-8996-a5c6b9206881 admin admin] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:54:53.984431 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 796/1582] 10.222.0.20 () {60 vars in 1144 bytes} [Mon Aug 1 16:54:53 2022] POST /identity/v3/users => generated 291 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 16:54:53.985546 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-151cdab1-2a90-40e3-94a3-8b5586db43ac admin admin] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:54:53.986479 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a058554d-6f7e-41b4-922e-01e467cc17e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:54:53.987091 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a058554d-6f7e-41b4-922e-01e467cc17e0 admin admin] REQUEST_METHOD: `POST` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:54:53.987364 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a058554d-6f7e-41b4-922e-01e467cc17e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:54:53.987628 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a058554d-6f7e-41b4-922e-01e467cc17e0 admin admin] PATH_INFO: `/v3/users` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:54:53.988289 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a058554d-6f7e-41b4-922e-01e467cc17e0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:54:53.990493 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a058554d-6f7e-41b4-922e-01e467cc17e0 admin admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:54:53.995595 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-151cdab1-2a90-40e3-94a3-8b5586db43ac admin admin] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:54:54.000327 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-327586dd-8cc9-45ed-9374-e9a3446263b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:54:54.002323 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327586dd-8cc9-45ed-9374-e9a3446263b5 admin admin] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:54:54.002323 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327586dd-8cc9-45ed-9374-e9a3446263b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:54:54.002323 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327586dd-8cc9-45ed-9374-e9a3446263b5 admin admin] PATH_INFO: `/v3/roles` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:54:54.002323 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-327586dd-8cc9-45ed-9374-e9a3446263b5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:54:54.003830 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-327586dd-8cc9-45ed-9374-e9a3446263b5 admin admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:54:54.015594 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 797/1583] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Aug 1 16:54:53 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 16:54:54.016555 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-327586dd-8cc9-45ed-9374-e9a3446263b5 admin admin] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:54:54.026074 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-327586dd-8cc9-45ed-9374-e9a3446263b5 admin admin] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:54:54.031544 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a292d481-6aaf-4241-939b-c6ba9d67861c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:54:54.032972 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a292d481-6aaf-4241-939b-c6ba9d67861c admin admin] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:54:54.032972 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a292d481-6aaf-4241-939b-c6ba9d67861c admin admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:54:54.032972 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a292d481-6aaf-4241-939b-c6ba9d67861c admin admin] PATH_INFO: `/v3/roles` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:54:54.033353 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a292d481-6aaf-4241-939b-c6ba9d67861c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:54:54.034939 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a292d481-6aaf-4241-939b-c6ba9d67861c admin admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:54:54.044840 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 798/1584] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Aug 1 16:54:54 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 16:54:54.045716 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a292d481-6aaf-4241-939b-c6ba9d67861c admin admin] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:54:54.055251 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a292d481-6aaf-4241-939b-c6ba9d67861c admin admin] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:54:54.060065 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f27091e-9600-49d5-a271-2a48d62dfacd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:54:54.060629 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f27091e-9600-49d5-a271-2a48d62dfacd admin admin] REQUEST_METHOD: `POST` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:54:54.060908 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f27091e-9600-49d5-a271-2a48d62dfacd admin admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:54:54.061182 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f27091e-9600-49d5-a271-2a48d62dfacd admin admin] PATH_INFO: `/v3/users` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:54:54.061876 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f27091e-9600-49d5-a271-2a48d62dfacd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:54:54.063810 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f27091e-9600-49d5-a271-2a48d62dfacd admin admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:54:54.075092 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG dbcounter [-] [87058] Writing DB stats keystone:SELECT=88,keystone:INSERT=12 {{(pid=87058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:54:54.095813 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 787/1585] 10.222.0.20 () {60 vars in 1144 bytes} [Mon Aug 1 16:54:53 2022] POST /identity/v3/users => generated 302 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 16:54:54.096746 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a058554d-6f7e-41b4-922e-01e467cc17e0 admin admin] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:54:54.106256 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a058554d-6f7e-41b4-922e-01e467cc17e0 admin admin] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:54:54.110857 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d65bffc6-0c34-4ca4-a7cb-5317aacc901f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:54:54.113381 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65bffc6-0c34-4ca4-a7cb-5317aacc901f admin admin] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:54:54.113381 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65bffc6-0c34-4ca4-a7cb-5317aacc901f admin admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:54:54.113381 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65bffc6-0c34-4ca4-a7cb-5317aacc901f admin admin] PATH_INFO: `/v3/roles` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:54:54.113381 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d65bffc6-0c34-4ca4-a7cb-5317aacc901f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:54:54.114319 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d65bffc6-0c34-4ca4-a7cb-5317aacc901f admin admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:54:54.125161 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 788/1586] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Aug 1 16:54:54 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 16:54:54.125933 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d65bffc6-0c34-4ca4-a7cb-5317aacc901f admin admin] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:54:54.155749 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 799/1587] 10.222.0.20 () {60 vars in 1144 bytes} [Mon Aug 1 16:54:54 2022] POST /identity/v3/users => generated 284 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 16:54:54.155749 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f27091e-9600-49d5-a271-2a48d62dfacd admin admin] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:54:54.164343 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f27091e-9600-49d5-a271-2a48d62dfacd admin admin] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:54:54.169566 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90d67f5a-cb41-4181-9c85-cfd3a939f030 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:54:54.170198 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d67f5a-cb41-4181-9c85-cfd3a939f030 admin admin] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:54:54.170476 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d67f5a-cb41-4181-9c85-cfd3a939f030 admin admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:54:54.170751 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d67f5a-cb41-4181-9c85-cfd3a939f030 admin admin] PATH_INFO: `/v3/roles` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:54:54.171468 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90d67f5a-cb41-4181-9c85-cfd3a939f030 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:54:54.173046 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90d67f5a-cb41-4181-9c85-cfd3a939f030 admin admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:54:54.184499 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d65bffc6-0c34-4ca4-a7cb-5317aacc901f admin admin] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:54:54.188192 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 800/1588] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Aug 1 16:54:54 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 16:54:54.188192 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90d67f5a-cb41-4181-9c85-cfd3a939f030 admin admin] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:54:54.188608 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-474ae298-e6f6-48fb-a31f-1ae37fd79663 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d9bd8d649154c979be3744bfd016f95', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b992d6a65df4bf9b27c13192d57c5f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:54:54.189327 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474ae298-e6f6-48fb-a31f-1ae37fd79663 service neutron] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:54:54.189680 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474ae298-e6f6-48fb-a31f-1ae37fd79663 service neutron] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:54:54.189956 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474ae298-e6f6-48fb-a31f-1ae37fd79663 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:54:54.195601 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90d67f5a-cb41-4181-9c85-cfd3a939f030 admin admin] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:54:54.200387 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-474ae298-e6f6-48fb-a31f-1ae37fd79663 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:54:54.201505 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9360766e-3fcb-47ba-8053-77bff8afd8f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:54:54.201875 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-474ae298-e6f6-48fb-a31f-1ae37fd79663 service neutron] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:54:54.202198 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9360766e-3fcb-47ba-8053-77bff8afd8f7 admin admin] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:54:54.202470 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9360766e-3fcb-47ba-8053-77bff8afd8f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:54:54.202740 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9360766e-3fcb-47ba-8053-77bff8afd8f7 admin admin] PATH_INFO: `/v3/roles` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:54:54.203415 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9360766e-3fcb-47ba-8053-77bff8afd8f7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:54:54.204984 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9360766e-3fcb-47ba-8053-77bff8afd8f7 admin admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:54:54.211693 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 789/1589] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 1 16:54:54 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 16:54:54.212495 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-474ae298-e6f6-48fb-a31f-1ae37fd79663 service neutron] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:54:54.215567 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 801/1590] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Aug 1 16:54:54 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 16:54:54.220209 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9360766e-3fcb-47ba-8053-77bff8afd8f7 admin admin] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:54:54.222835 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-474ae298-e6f6-48fb-a31f-1ae37fd79663 service neutron] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:54:54.228287 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d8e63b6-8e8e-4176-acb4-dafaff5b049e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:54:54.228476 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8e63b6-8e8e-4176-acb4-dafaff5b049e admin admin] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:54:54.228810 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8e63b6-8e8e-4176-acb4-dafaff5b049e admin admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:54:54.229088 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8e63b6-8e8e-4176-acb4-dafaff5b049e admin admin] PATH_INFO: `/v3/roles` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:54:54.229873 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d8e63b6-8e8e-4176-acb4-dafaff5b049e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:54:54.231622 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9360766e-3fcb-47ba-8053-77bff8afd8f7 admin admin] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:54:54.231760 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d8e63b6-8e8e-4176-acb4-dafaff5b049e admin admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:54:54.236795 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8dbd440-abda-4f24-bf1c-0eb7589664c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:54:54.237451 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8dbd440-abda-4f24-bf1c-0eb7589664c1 admin admin] REQUEST_METHOD: `PUT` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:54:54.237809 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8dbd440-abda-4f24-bf1c-0eb7589664c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:54:54.238115 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8dbd440-abda-4f24-bf1c-0eb7589664c1 admin admin] PATH_INFO: `/v3/projects/2c036dda19f64bfca49f657f667ac8f3/users/faebdbd71dfd4dbbbb4daa61de6b9806/roles/0999d3da63a545f18ff3252afdc90518` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:54:54.242658 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 790/1591] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Aug 1 16:54:54 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 16:54:54.245481 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d8e63b6-8e8e-4176-acb4-dafaff5b049e admin admin] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:54:54.253825 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d8e63b6-8e8e-4176-acb4-dafaff5b049e admin admin] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:54:54.254496 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8dbd440-abda-4f24-bf1c-0eb7589664c1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2c036dda19f64bfca49f657f667ac8f3, user_id=faebdbd71dfd4dbbbb4daa61de6b9806, role_id=0999d3da63a545f18ff3252afdc90518)` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:54:54.257187 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8dbd440-abda-4f24-bf1c-0eb7589664c1 admin admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:54:54.258592 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e930f5ce-d13b-49d8-a041-da557913dba1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:54:54.259198 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e930f5ce-d13b-49d8-a041-da557913dba1 admin admin] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:54:54.259479 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e930f5ce-d13b-49d8-a041-da557913dba1 admin admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:54:54.259750 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e930f5ce-d13b-49d8-a041-da557913dba1 admin admin] PATH_INFO: `/v3/roles` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:54:54.260472 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e930f5ce-d13b-49d8-a041-da557913dba1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:54:54.262860 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e930f5ce-d13b-49d8-a041-da557913dba1 admin admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:54:54.272635 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 791/1592] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Aug 1 16:54:54 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 16:54:54.273443 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e930f5ce-d13b-49d8-a041-da557913dba1 admin admin] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:54:54.283587 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e930f5ce-d13b-49d8-a041-da557913dba1 admin admin] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:54:54.288249 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd5bb7bf-4d67-480b-bf55-a7196323cfbe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:54:54.288832 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5bb7bf-4d67-480b-bf55-a7196323cfbe admin admin] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:54:54.289110 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5bb7bf-4d67-480b-bf55-a7196323cfbe admin admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:54:54.289380 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5bb7bf-4d67-480b-bf55-a7196323cfbe admin admin] PATH_INFO: `/v3/roles` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:54:54.290076 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd5bb7bf-4d67-480b-bf55-a7196323cfbe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:54:54.291644 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd5bb7bf-4d67-480b-bf55-a7196323cfbe admin admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:54:54.301709 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 792/1593] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Aug 1 16:54:54 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 16:54:54.302501 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd5bb7bf-4d67-480b-bf55-a7196323cfbe admin admin] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:54:54.311625 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd5bb7bf-4d67-480b-bf55-a7196323cfbe admin admin] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:54:54.316229 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a84d9c31-0de3-46b1-aef6-f8ca37953d30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:54:54.316789 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84d9c31-0de3-46b1-aef6-f8ca37953d30 admin admin] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:54:54.317067 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84d9c31-0de3-46b1-aef6-f8ca37953d30 admin admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:54:54.317337 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84d9c31-0de3-46b1-aef6-f8ca37953d30 admin admin] PATH_INFO: `/v3/roles` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:54:54.318002 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a84d9c31-0de3-46b1-aef6-f8ca37953d30 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:54:54.319593 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a84d9c31-0de3-46b1-aef6-f8ca37953d30 admin admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:54:54.329773 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 793/1594] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Aug 1 16:54:54 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 16:54:54.330569 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a84d9c31-0de3-46b1-aef6-f8ca37953d30 admin admin] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:54:54.340908 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a84d9c31-0de3-46b1-aef6-f8ca37953d30 admin admin] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:54:54.345866 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b82421c1-641f-4da6-9920-91e0010bfe50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:54:54.346490 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82421c1-641f-4da6-9920-91e0010bfe50 admin admin] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:54:54.346771 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82421c1-641f-4da6-9920-91e0010bfe50 admin admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:54:54.347045 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82421c1-641f-4da6-9920-91e0010bfe50 admin admin] PATH_INFO: `/v3/roles` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:54:54.347734 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b82421c1-641f-4da6-9920-91e0010bfe50 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:54:54.349360 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b82421c1-641f-4da6-9920-91e0010bfe50 admin admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:54:54.362505 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 794/1595] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Aug 1 16:54:54 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 16:54:54.362505 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b82421c1-641f-4da6-9920-91e0010bfe50 admin admin] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:54:54.372791 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b82421c1-641f-4da6-9920-91e0010bfe50 admin admin] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:54:54.378541 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da0ca3b2-cdd3-4b0e-bdda-cb1cea9c874a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:54:54.379219 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0ca3b2-cdd3-4b0e-bdda-cb1cea9c874a admin admin] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:54:54.379502 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0ca3b2-cdd3-4b0e-bdda-cb1cea9c874a admin admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:54:54.379774 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0ca3b2-cdd3-4b0e-bdda-cb1cea9c874a admin admin] PATH_INFO: `/v3/roles` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:54:54.380504 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da0ca3b2-cdd3-4b0e-bdda-cb1cea9c874a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:54:54.381454 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 802/1596] 10.222.0.20 () {60 vars in 1483 bytes} [Mon Aug 1 16:54:54 2022] PUT /identity/v3/projects/2c036dda19f64bfca49f657f667ac8f3/users/faebdbd71dfd4dbbbb4daa61de6b9806/roles/0999d3da63a545f18ff3252afdc90518 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 16:54:54.382287 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da0ca3b2-cdd3-4b0e-bdda-cb1cea9c874a admin admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:54:54.384514 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8dbd440-abda-4f24-bf1c-0eb7589664c1 admin admin] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:54:54.384629 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG dbcounter [-] [87057] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=87057) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:54:54.399191 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 795/1597] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Aug 1 16:54:54 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 16:54:54.399191 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da0ca3b2-cdd3-4b0e-bdda-cb1cea9c874a admin admin] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:54:54.409678 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da0ca3b2-cdd3-4b0e-bdda-cb1cea9c874a admin admin] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:54:54.459852 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8dbd440-abda-4f24-bf1c-0eb7589664c1 admin admin] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:54:54.464496 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8a8fc4b-87c4-45f4-8c96-c181a95d4f34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:54:54.465361 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a8fc4b-87c4-45f4-8c96-c181a95d4f34 admin admin] REQUEST_METHOD: `PUT` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:54:54.465636 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a8fc4b-87c4-45f4-8c96-c181a95d4f34 admin admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:54:54.465923 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a8fc4b-87c4-45f4-8c96-c181a95d4f34 admin admin] PATH_INFO: `/v3/projects/ee3d55c8b96e4821bd2592a3ee665533/users/c2079b0fbf7b4a8c88d2522da9f2c73b/roles/0999d3da63a545f18ff3252afdc90518` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:54:54.470196 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b66d951-41be-4db4-974f-ce7c5a6b768c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:54:54.470824 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b66d951-41be-4db4-974f-ce7c5a6b768c admin admin] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:54:54.471102 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b66d951-41be-4db4-974f-ce7c5a6b768c admin admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:54:54.471373 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b66d951-41be-4db4-974f-ce7c5a6b768c admin admin] PATH_INFO: `/v3/roles` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:54:54.472073 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b66d951-41be-4db4-974f-ce7c5a6b768c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:54:54.473665 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b66d951-41be-4db4-974f-ce7c5a6b768c admin admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:54:54.481240 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8a8fc4b-87c4-45f4-8c96-c181a95d4f34 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ee3d55c8b96e4821bd2592a3ee665533, user_id=c2079b0fbf7b4a8c88d2522da9f2c73b, role_id=0999d3da63a545f18ff3252afdc90518)` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:54:54.483842 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 796/1598] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Aug 1 16:54:54 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 16:54:54.487579 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8a8fc4b-87c4-45f4-8c96-c181a95d4f34 admin admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:54:54.487723 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b66d951-41be-4db4-974f-ce7c5a6b768c admin admin] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:54:54.494600 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b66d951-41be-4db4-974f-ce7c5a6b768c admin admin] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:54:54.500288 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d13bf3a-9b5e-43ca-876f-980c43da6209 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:54:54.500929 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d13bf3a-9b5e-43ca-876f-980c43da6209 admin admin] REQUEST_METHOD: `PUT` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:54:54.501211 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d13bf3a-9b5e-43ca-876f-980c43da6209 admin admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:54:54.501489 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d13bf3a-9b5e-43ca-876f-980c43da6209 admin admin] PATH_INFO: `/v3/projects/3730f131740e40af8d6d83d7844df33d/users/3abf76e9664a4723bfbac555c914d50d/roles/0999d3da63a545f18ff3252afdc90518` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:54:54.516916 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d13bf3a-9b5e-43ca-876f-980c43da6209 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3730f131740e40af8d6d83d7844df33d, user_id=3abf76e9664a4723bfbac555c914d50d, role_id=0999d3da63a545f18ff3252afdc90518)` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:54:54.517979 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: WARNING py.warnings [None req-1d13bf3a-9b5e-43ca-876f-980c43da6209 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 01 16:54:54.517979 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: warnings.warn(msg) Aug 01 16:54:54.517979 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: Aug 01 16:54:54.520025 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d13bf3a-9b5e-43ca-876f-980c43da6209 admin admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:54:54.573074 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 803/1599] 10.222.0.20 () {60 vars in 1483 bytes} [Mon Aug 1 16:54:54 2022] PUT /identity/v3/projects/ee3d55c8b96e4821bd2592a3ee665533/users/c2079b0fbf7b4a8c88d2522da9f2c73b/roles/0999d3da63a545f18ff3252afdc90518 => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 16:54:54.573074 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8a8fc4b-87c4-45f4-8c96-c181a95d4f34 admin admin] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:54:54.636622 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 797/1600] 10.222.0.20 () {60 vars in 1483 bytes} [Mon Aug 1 16:54:54 2022] PUT /identity/v3/projects/3730f131740e40af8d6d83d7844df33d/users/3abf76e9664a4723bfbac555c914d50d/roles/0999d3da63a545f18ff3252afdc90518 => generated 0 bytes in 143 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 16:54:54.636622 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d13bf3a-9b5e-43ca-876f-980c43da6209 admin admin] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:54:54.653849 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8a8fc4b-87c4-45f4-8c96-c181a95d4f34 admin admin] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:54:54.690748 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d13bf3a-9b5e-43ca-876f-980c43da6209 admin admin] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:54:54.692169 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG dbcounter [-] [87058] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=87058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:54:54.695934 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfeb983f-1d2a-4d3d-96ee-45afcffa60aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:54:54.696609 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfeb983f-1d2a-4d3d-96ee-45afcffa60aa admin admin] REQUEST_METHOD: `PUT` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:54:54.696891 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfeb983f-1d2a-4d3d-96ee-45afcffa60aa admin admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:54:54.697169 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfeb983f-1d2a-4d3d-96ee-45afcffa60aa admin admin] PATH_INFO: `/v3/projects/64929e340dae4c0f90162702aab5bc21/users/b353fdd302394227987a6ae8c2943173/roles/0999d3da63a545f18ff3252afdc90518` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:54:54.712147 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5de73c1-1d26-4a50-9726-a1d634b6aae4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:54:54.712781 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5de73c1-1d26-4a50-9726-a1d634b6aae4 admin admin] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:54:54.713061 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5de73c1-1d26-4a50-9726-a1d634b6aae4 admin admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:54:54.713336 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5de73c1-1d26-4a50-9726-a1d634b6aae4 admin admin] PATH_INFO: `/v3/roles` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:54:54.714080 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfeb983f-1d2a-4d3d-96ee-45afcffa60aa admin admin] RBAC: Authorizing `identity:create_grant(project_id=64929e340dae4c0f90162702aab5bc21, user_id=b353fdd302394227987a6ae8c2943173, role_id=0999d3da63a545f18ff3252afdc90518)` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:54:54.714257 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5de73c1-1d26-4a50-9726-a1d634b6aae4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:54:54.716238 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5de73c1-1d26-4a50-9726-a1d634b6aae4 admin admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:54:54.717627 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfeb983f-1d2a-4d3d-96ee-45afcffa60aa admin admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:54:54.727960 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 804/1601] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Aug 1 16:54:54 2022] GET /identity/v3/roles => generated 1420 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 16:54:54.729092 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919fb7c5-e46d-4ba6-bf02-ce4c018b1f8b None None] REQUEST_METHOD: `POST` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:54:54.729377 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919fb7c5-e46d-4ba6-bf02-ce4c018b1f8b None None] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:54:54.730442 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919fb7c5-e46d-4ba6-bf02-ce4c018b1f8b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:54:54.762671 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.auth.core [None req-919fb7c5-e46d-4ba6-bf02-ce4c018b1f8b None None] MFA Rules not processed for user `faebdbd71dfd4dbbbb4daa61de6b9806`. Rule list: `[]` (Enabled: `True`). {{(pid=87058) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:54:54.763255 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.fernet_utils [None req-919fb7c5-e46d-4ba6-bf02-ce4c018b1f8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:54:54.796709 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 798/1602] 10.222.0.20 () {60 vars in 1483 bytes} [Mon Aug 1 16:54:54 2022] PUT /identity/v3/projects/64929e340dae4c0f90162702aab5bc21/users/b353fdd302394227987a6ae8c2943173/roles/0999d3da63a545f18ff3252afdc90518 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 16:54:54.797128 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfeb983f-1d2a-4d3d-96ee-45afcffa60aa admin admin] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:54:54.836404 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 805/1603] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 1 16:54:54 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:54:54.837351 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-919fb7c5-e46d-4ba6-bf02-ce4c018b1f8b None None] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:54:54.859702 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-919fb7c5-e46d-4ba6-bf02-ce4c018b1f8b None None] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:54:54.860479 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG dbcounter [-] [87057] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=87057) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:54:54.878599 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfeb983f-1d2a-4d3d-96ee-45afcffa60aa admin admin] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:54:54.885490 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a4e9648-5250-4534-98d4-4138a823fe53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:54:54.886832 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4e9648-5250-4534-98d4-4138a823fe53 admin admin] REQUEST_METHOD: `PUT` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:54:54.887181 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4e9648-5250-4534-98d4-4138a823fe53 admin admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:54:54.887523 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4e9648-5250-4534-98d4-4138a823fe53 admin admin] PATH_INFO: `/v3/projects/204b0c7d4bf843659ec38b79fa7c1815/users/4df0b959497d4d51a308c1db5626be6d/roles/0999d3da63a545f18ff3252afdc90518` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:54:54.907446 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a4e9648-5250-4534-98d4-4138a823fe53 admin admin] RBAC: Authorizing `identity:create_grant(project_id=204b0c7d4bf843659ec38b79fa7c1815, user_id=4df0b959497d4d51a308c1db5626be6d, role_id=0999d3da63a545f18ff3252afdc90518)` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:54:54.910655 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a4e9648-5250-4534-98d4-4138a823fe53 admin admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:54:54.929059 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7676a1c-fd37-41d5-8bc4-210e1f131556 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:54:54.931348 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7676a1c-fd37-41d5-8bc4-210e1f131556 admin admin] REQUEST_METHOD: `PUT` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:54:54.931348 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7676a1c-fd37-41d5-8bc4-210e1f131556 admin admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:54:54.931348 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7676a1c-fd37-41d5-8bc4-210e1f131556 admin admin] PATH_INFO: `/v3/projects/56ae44822d9a49259d08ebc7a35e3a9e/users/9131d19bf929408fb5a194dc615b6b2a/roles/0999d3da63a545f18ff3252afdc90518` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:54:54.944652 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7676a1c-fd37-41d5-8bc4-210e1f131556 admin admin] RBAC: Authorizing `identity:create_grant(project_id=56ae44822d9a49259d08ebc7a35e3a9e, user_id=9131d19bf929408fb5a194dc615b6b2a, role_id=0999d3da63a545f18ff3252afdc90518)` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:54:54.947477 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7676a1c-fd37-41d5-8bc4-210e1f131556 admin admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:54:54.992769 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 799/1604] 10.222.0.20 () {60 vars in 1483 bytes} [Mon Aug 1 16:54:54 2022] PUT /identity/v3/projects/204b0c7d4bf843659ec38b79fa7c1815/users/4df0b959497d4d51a308c1db5626be6d/roles/0999d3da63a545f18ff3252afdc90518 => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 16:54:54.994111 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3373b1e4-fe60-466a-8a96-91be1d16f782 None None] REQUEST_METHOD: `POST` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:54:54.994400 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3373b1e4-fe60-466a-8a96-91be1d16f782 None None] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:54:54.994684 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3373b1e4-fe60-466a-8a96-91be1d16f782 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:54:55.017803 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.auth.core [None req-3373b1e4-fe60-466a-8a96-91be1d16f782 None None] MFA Rules not processed for user `1007f8e379b74e169b98fb7c3c0ba5b4`. Rule list: `[]` (Enabled: `True`). {{(pid=87057) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:54:55.018630 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.fernet_utils [None req-3373b1e4-fe60-466a-8a96-91be1d16f782 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87057) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:54:55.049313 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 806/1605] 10.222.0.20 () {60 vars in 1483 bytes} [Mon Aug 1 16:54:54 2022] PUT /identity/v3/projects/56ae44822d9a49259d08ebc7a35e3a9e/users/9131d19bf929408fb5a194dc615b6b2a/roles/0999d3da63a545f18ff3252afdc90518 => generated 0 bytes in 209 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 16:54:55.049313 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7360ce27-f958-45f4-9a01-9acc2c3d0ef2 None None] REQUEST_METHOD: `POST` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:54:55.049313 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7360ce27-f958-45f4-9a01-9acc2c3d0ef2 None None] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:54:55.049313 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7360ce27-f958-45f4-9a01-9acc2c3d0ef2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:54:55.079639 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.auth.core [None req-7360ce27-f958-45f4-9a01-9acc2c3d0ef2 None None] MFA Rules not processed for user `3abf76e9664a4723bfbac555c914d50d`. Rule list: `[]` (Enabled: `True`). {{(pid=87058) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:54:55.080868 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.fernet_utils [None req-7360ce27-f958-45f4-9a01-9acc2c3d0ef2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:54:55.092048 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 800/1606] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 1 16:54:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:54:55.094199 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3373b1e4-fe60-466a-8a96-91be1d16f782 None None] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:54:55.103453 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3373b1e4-fe60-466a-8a96-91be1d16f782 None None] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:54:55.138464 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG dbcounter [-] [87058] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:54:55.163348 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c3ff87-13a3-4869-9834-5d99da7261a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:54:55.164054 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c3ff87-13a3-4869-9834-5d99da7261a3 admin admin] REQUEST_METHOD: `PUT` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:54:55.164332 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c3ff87-13a3-4869-9834-5d99da7261a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:54:55.164605 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c3ff87-13a3-4869-9834-5d99da7261a3 admin admin] PATH_INFO: `/v3/projects/ee3a7c10d9b748e5a5481446ceb219fb/users/1cf7915cbe9547e0b48706aad57f2bc3/roles/0999d3da63a545f18ff3252afdc90518` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:54:55.166261 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 807/1607] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 1 16:54:55 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:54:55.168117 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c9a720-c9c2-4301-ab5f-36b153807b8c None None] REQUEST_METHOD: `POST` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:54:55.168405 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c9a720-c9c2-4301-ab5f-36b153807b8c None None] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:54:55.168682 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c9a720-c9c2-4301-ab5f-36b153807b8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:54:55.188967 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.auth.core [None req-d2c9a720-c9c2-4301-ab5f-36b153807b8c None None] MFA Rules not processed for user `1007f8e379b74e169b98fb7c3c0ba5b4`. Rule list: `[]` (Enabled: `True`). {{(pid=87058) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:54:55.190210 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.fernet_utils [None req-d2c9a720-c9c2-4301-ab5f-36b153807b8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:54:55.191697 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82c3ff87-13a3-4869-9834-5d99da7261a3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ee3a7c10d9b748e5a5481446ceb219fb, user_id=1cf7915cbe9547e0b48706aad57f2bc3, role_id=0999d3da63a545f18ff3252afdc90518)` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:54:55.194808 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82c3ff87-13a3-4869-9834-5d99da7261a3 admin admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:54:55.199467 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 808/1608] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 1 16:54:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:54:55.200382 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c9a720-c9c2-4301-ab5f-36b153807b8c None None] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:54:55.216045 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c9a720-c9c2-4301-ab5f-36b153807b8c None None] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:54:55.220827 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ebf9a5-df14-4445-add4-caad33550844 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:54:55.221446 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ebf9a5-df14-4445-add4-caad33550844 admin admin] REQUEST_METHOD: `POST` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:54:55.221724 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ebf9a5-df14-4445-add4-caad33550844 admin admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:54:55.221993 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ebf9a5-df14-4445-add4-caad33550844 admin admin] PATH_INFO: `/v3/projects` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:54:55.222690 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43ebf9a5-df14-4445-add4-caad33550844 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:54:55.224645 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43ebf9a5-df14-4445-add4-caad33550844 admin admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:54:55.250111 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 801/1609] 10.222.0.20 () {60 vars in 1483 bytes} [Mon Aug 1 16:54:55 2022] PUT /identity/v3/projects/ee3a7c10d9b748e5a5481446ceb219fb/users/1cf7915cbe9547e0b48706aad57f2bc3/roles/0999d3da63a545f18ff3252afdc90518 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 16:54:55.251539 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306c1304-ab06-4cee-94e8-60ce1674f95c None None] REQUEST_METHOD: `POST` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:54:55.251930 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306c1304-ab06-4cee-94e8-60ce1674f95c None None] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:54:55.252314 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306c1304-ab06-4cee-94e8-60ce1674f95c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:54:55.281265 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.auth.core [None req-306c1304-ab06-4cee-94e8-60ce1674f95c None None] MFA Rules not processed for user `4df0b959497d4d51a308c1db5626be6d`. Rule list: `[]` (Enabled: `True`). {{(pid=87057) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:54:55.282604 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.fernet_utils [None req-306c1304-ab06-4cee-94e8-60ce1674f95c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87057) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:54:55.303055 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 809/1610] 10.222.0.20 () {60 vars in 1153 bytes} [Mon Aug 1 16:54:55 2022] POST /identity/v3/projects => generated 367 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 16:54:55.303055 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eac877f-6185-4a64-837f-d3188067b101 None None] REQUEST_METHOD: `POST` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:54:55.303055 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eac877f-6185-4a64-837f-d3188067b101 None None] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:54:55.304631 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eac877f-6185-4a64-837f-d3188067b101 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:54:55.324584 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.auth.core [None req-5eac877f-6185-4a64-837f-d3188067b101 None None] MFA Rules not processed for user `1007f8e379b74e169b98fb7c3c0ba5b4`. Rule list: `[]` (Enabled: `True`). {{(pid=87058) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:54:55.325819 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.fernet_utils [None req-5eac877f-6185-4a64-837f-d3188067b101 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:54:55.337900 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG dbcounter [-] [87057] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=87057) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:54:55.359914 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 802/1611] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 1 16:54:55 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:54:55.360901 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-306c1304-ab06-4cee-94e8-60ce1674f95c None None] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:54:55.389670 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 810/1612] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 1 16:54:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:54:55.391074 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ce2dae-38ee-487b-b74c-9c9fec1bacda None None] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:54:55.391411 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ce2dae-38ee-487b-b74c-9c9fec1bacda None None] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:54:55.391736 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ce2dae-38ee-487b-b74c-9c9fec1bacda None None] PATH_INFO: `/` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:54:55.394312 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 811/1613] 10.222.0.20 () {54 vars in 962 bytes} [Mon Aug 1 16:54:55 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 16:54:55.394984 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ce2dae-38ee-487b-b74c-9c9fec1bacda None None] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:54:55.422960 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-306c1304-ab06-4cee-94e8-60ce1674f95c None None] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:54:55.451831 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ce2dae-38ee-487b-b74c-9c9fec1bacda None None] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:54:55.455741 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f90d579-285d-4e1b-bfdc-b81983f9aa5d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d9bd8d649154c979be3744bfd016f95', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b992d6a65df4bf9b27c13192d57c5f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:54:55.456420 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f90d579-285d-4e1b-bfdc-b81983f9aa5d service neutron] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:54:55.456702 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f90d579-285d-4e1b-bfdc-b81983f9aa5d service neutron] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:54:55.457003 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f90d579-285d-4e1b-bfdc-b81983f9aa5d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:54:55.467031 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f90d579-285d-4e1b-bfdc-b81983f9aa5d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:54:55.468440 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f90d579-285d-4e1b-bfdc-b81983f9aa5d service neutron] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:54:55.472686 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d8a3044-dee3-4971-a031-abf094f37209 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d9bd8d649154c979be3744bfd016f95', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b992d6a65df4bf9b27c13192d57c5f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:54:55.473310 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8a3044-dee3-4971-a031-abf094f37209 service neutron] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:54:55.476017 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8a3044-dee3-4971-a031-abf094f37209 service neutron] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:54:55.476017 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8a3044-dee3-4971-a031-abf094f37209 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:54:55.478050 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 812/1614] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 1 16:54:55 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 16:54:55.479255 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d10dfb4-6c77-4497-9a9a-4cf196253608 None None] REQUEST_METHOD: `POST` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:54:55.479582 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d10dfb4-6c77-4497-9a9a-4cf196253608 None None] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:54:55.480451 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d10dfb4-6c77-4497-9a9a-4cf196253608 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:54:55.484912 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d8a3044-dee3-4971-a031-abf094f37209 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:54:55.486398 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d8a3044-dee3-4971-a031-abf094f37209 service neutron] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:54:55.499973 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 803/1615] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 1 16:54:55 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 16:54:55.501339 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d8a3044-dee3-4971-a031-abf094f37209 service neutron] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:54:55.514692 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d8a3044-dee3-4971-a031-abf094f37209 service neutron] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:54:55.520713 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a487389-3dc4-4f09-b649-319165273de7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:54:55.521351 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a487389-3dc4-4f09-b649-319165273de7 admin admin] REQUEST_METHOD: `POST` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:54:55.521805 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.auth.core [None req-5d10dfb4-6c77-4497-9a9a-4cf196253608 None None] MFA Rules not processed for user `1cf7915cbe9547e0b48706aad57f2bc3`. Rule list: `[]` (Enabled: `True`). {{(pid=87058) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:54:55.521937 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a487389-3dc4-4f09-b649-319165273de7 admin admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:54:55.522211 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a487389-3dc4-4f09-b649-319165273de7 admin admin] PATH_INFO: `/v3/users` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:54:55.522850 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a487389-3dc4-4f09-b649-319165273de7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:54:55.522978 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.fernet_utils [None req-5d10dfb4-6c77-4497-9a9a-4cf196253608 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:54:55.525331 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a487389-3dc4-4f09-b649-319165273de7 admin admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:54:55.570339 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG dbcounter [-] [87058] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:54:55.596152 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 813/1616] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 1 16:54:55 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:54:55.597354 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d10dfb4-6c77-4497-9a9a-4cf196253608 None None] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:54:55.616365 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d10dfb4-6c77-4497-9a9a-4cf196253608 None None] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:54:55.617113 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 804/1617] 10.222.0.20 () {60 vars in 1144 bytes} [Mon Aug 1 16:54:55 2022] POST /identity/v3/users => generated 290 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 16:54:55.622524 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4d7808-beea-4fd5-90cc-06e595947e64 None None] REQUEST_METHOD: `POST` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:54:55.622907 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4d7808-beea-4fd5-90cc-06e595947e64 None None] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:54:55.623389 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4d7808-beea-4fd5-90cc-06e595947e64 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:54:55.626046 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-672702d8-7efc-4e21-bfaf-924b0dff46e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:54:55.626688 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672702d8-7efc-4e21-bfaf-924b0dff46e0 admin admin] REQUEST_METHOD: `POST` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:54:55.627245 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672702d8-7efc-4e21-bfaf-924b0dff46e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:54:55.627619 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672702d8-7efc-4e21-bfaf-924b0dff46e0 admin admin] PATH_INFO: `/v3/projects` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:54:55.628296 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-672702d8-7efc-4e21-bfaf-924b0dff46e0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:54:55.630521 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-672702d8-7efc-4e21-bfaf-924b0dff46e0 admin admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:54:55.676824 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.auth.core [None req-bd4d7808-beea-4fd5-90cc-06e595947e64 None None] MFA Rules not processed for user `6533106d03eb4ca185a8839336d2aff9`. Rule list: `[]` (Enabled: `True`). {{(pid=87057) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:54:55.678271 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.fernet_utils [None req-bd4d7808-beea-4fd5-90cc-06e595947e64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87057) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:54:55.728581 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 814/1618] 10.222.0.20 () {60 vars in 1153 bytes} [Mon Aug 1 16:54:55 2022] POST /identity/v3/projects => generated 379 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 16:54:55.728581 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-672702d8-7efc-4e21-bfaf-924b0dff46e0 admin admin] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:54:55.773533 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 805/1619] 10.222.0.20 () {58 vars in 1059 bytes} [Mon Aug 1 16:54:55 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:54:55.775139 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd4d7808-beea-4fd5-90cc-06e595947e64 None None] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:54:55.789162 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd4d7808-beea-4fd5-90cc-06e595947e64 None None] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:54:55.798585 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG dbcounter [-] [87057] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87057) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:54:55.861248 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-672702d8-7efc-4e21-bfaf-924b0dff46e0 admin admin] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:54:55.869683 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-932ce547-a417-4c48-a02f-a180860f45a1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d9bd8d649154c979be3744bfd016f95', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b992d6a65df4bf9b27c13192d57c5f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:54:55.872094 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932ce547-a417-4c48-a02f-a180860f45a1 service neutron] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:54:55.872381 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932ce547-a417-4c48-a02f-a180860f45a1 service neutron] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:54:55.872749 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932ce547-a417-4c48-a02f-a180860f45a1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:54:55.887588 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5609d915-6f50-49a5-8aab-d85198205df7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:54:55.889671 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5609d915-6f50-49a5-8aab-d85198205df7 admin admin] REQUEST_METHOD: `POST` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:54:55.889671 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5609d915-6f50-49a5-8aab-d85198205df7 admin admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:54:55.889917 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5609d915-6f50-49a5-8aab-d85198205df7 admin admin] PATH_INFO: `/v3/projects` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:54:55.890465 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5609d915-6f50-49a5-8aab-d85198205df7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:54:55.893885 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5609d915-6f50-49a5-8aab-d85198205df7 admin admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:54:55.982461 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-932ce547-a417-4c48-a02f-a180860f45a1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:54:55.984696 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-932ce547-a417-4c48-a02f-a180860f45a1 service neutron] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:54:56.000081 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 815/1620] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 1 16:54:55 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 274 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 16:54:56.000902 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-932ce547-a417-4c48-a02f-a180860f45a1 service neutron] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:54:56.014513 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-932ce547-a417-4c48-a02f-a180860f45a1 service neutron] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:54:56.020864 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9709ae8-a188-43da-a5cd-b5ff5ff94495 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:54:56.030265 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9709ae8-a188-43da-a5cd-b5ff5ff94495 admin admin] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:54:56.030265 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9709ae8-a188-43da-a5cd-b5ff5ff94495 admin admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:54:56.030265 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9709ae8-a188-43da-a5cd-b5ff5ff94495 admin admin] PATH_INFO: `/v3/roles` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:54:56.030265 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9709ae8-a188-43da-a5cd-b5ff5ff94495 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:54:56.030265 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9709ae8-a188-43da-a5cd-b5ff5ff94495 admin admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:54:56.036199 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 816/1621] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Aug 1 16:54:55 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 16:54:56.039397 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9709ae8-a188-43da-a5cd-b5ff5ff94495 admin admin] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:54:56.063050 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 806/1622] 10.222.0.20 () {60 vars in 1153 bytes} [Mon Aug 1 16:54:55 2022] POST /identity/v3/projects => generated 369 bytes in 289 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 16:54:56.064340 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5609d915-6f50-49a5-8aab-d85198205df7 admin admin] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:54:56.083857 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9709ae8-a188-43da-a5cd-b5ff5ff94495 admin admin] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:54:56.148147 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed8d5d4-8882-4535-96e0-1224e9383624 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:54:56.151040 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed8d5d4-8882-4535-96e0-1224e9383624 admin admin] REQUEST_METHOD: `POST` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:54:56.151040 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed8d5d4-8882-4535-96e0-1224e9383624 admin admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:54:56.151040 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed8d5d4-8882-4535-96e0-1224e9383624 admin admin] PATH_INFO: `/v3/users` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:54:56.151355 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5609d915-6f50-49a5-8aab-d85198205df7 admin admin] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:54:56.151468 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aed8d5d4-8882-4535-96e0-1224e9383624 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:54:56.152379 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aed8d5d4-8882-4535-96e0-1224e9383624 admin admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:54:56.155806 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd64228-3f31-4fd4-ba22-f430b1047b97 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6533106d03eb4ca185a8839336d2aff9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b992d6a65df4bf9b27c13192d57c5f0', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:54:56.156503 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd64228-3f31-4fd4-ba22-f430b1047b97 service nova] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:54:56.156855 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd64228-3f31-4fd4-ba22-f430b1047b97 service nova] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:54:56.157200 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd64228-3f31-4fd4-ba22-f430b1047b97 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:54:56.160396 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG dbcounter [-] [87058] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:54:56.231774 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efd64228-3f31-4fd4-ba22-f430b1047b97 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:54:56.242865 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efd64228-3f31-4fd4-ba22-f430b1047b97 service nova] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:54:56.261564 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 807/1623] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 1 16:54:56 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 198 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 16:54:56.263046 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd64228-3f31-4fd4-ba22-f430b1047b97 service nova] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:54:56.274939 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 817/1624] 10.222.0.20 () {60 vars in 1144 bytes} [Mon Aug 1 16:54:56 2022] POST /identity/v3/users => generated 296 bytes in 235 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 16:54:56.274939 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed8d5d4-8882-4535-96e0-1224e9383624 admin admin] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:54:56.293990 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd64228-3f31-4fd4-ba22-f430b1047b97 service nova] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:54:56.294688 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed8d5d4-8882-4535-96e0-1224e9383624 admin admin] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:54:56.304194 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-653dadd4-901d-4fb4-a985-2da314c69003 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:54:56.304917 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653dadd4-901d-4fb4-a985-2da314c69003 admin admin] REQUEST_METHOD: `POST` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:54:56.305248 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653dadd4-901d-4fb4-a985-2da314c69003 admin admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:54:56.305522 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653dadd4-901d-4fb4-a985-2da314c69003 admin admin] PATH_INFO: `/v3/users` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:54:56.307354 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-653dadd4-901d-4fb4-a985-2da314c69003 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:54:56.311567 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-653dadd4-901d-4fb4-a985-2da314c69003 admin admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:54:56.312159 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e67dcb0e-6e19-42d4-9152-1742b00cd17d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:54:56.312791 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67dcb0e-6e19-42d4-9152-1742b00cd17d admin admin] REQUEST_METHOD: `PUT` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:54:56.313372 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67dcb0e-6e19-42d4-9152-1742b00cd17d admin admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:54:56.313714 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67dcb0e-6e19-42d4-9152-1742b00cd17d admin admin] PATH_INFO: `/v3/projects/2d5994265bf54c1ab513dff4848f63ef/users/00aed21d40604d6088a5e0cdee4ef18f/roles/440bb943dd5b46fea038b903b6eaaf81` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:54:56.343514 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e67dcb0e-6e19-42d4-9152-1742b00cd17d admin admin] RBAC: Authorizing `identity:create_grant(project_id=2d5994265bf54c1ab513dff4848f63ef, user_id=00aed21d40604d6088a5e0cdee4ef18f, role_id=440bb943dd5b46fea038b903b6eaaf81)` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:54:56.346682 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e67dcb0e-6e19-42d4-9152-1742b00cd17d admin admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:54:56.460273 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 818/1625] 10.222.0.20 () {60 vars in 1144 bytes} [Mon Aug 1 16:54:56 2022] POST /identity/v3/users => generated 291 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 16:54:56.461298 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-653dadd4-901d-4fb4-a985-2da314c69003 admin admin] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:54:56.487352 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 808/1626] 10.222.0.20 () {60 vars in 1483 bytes} [Mon Aug 1 16:54:56 2022] PUT /identity/v3/projects/2d5994265bf54c1ab513dff4848f63ef/users/00aed21d40604d6088a5e0cdee4ef18f/roles/440bb943dd5b46fea038b903b6eaaf81 => generated 0 bytes in 222 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 16:54:56.488702 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e67dcb0e-6e19-42d4-9152-1742b00cd17d admin admin] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:54:56.501351 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-653dadd4-901d-4fb4-a985-2da314c69003 admin admin] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:54:56.503721 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG dbcounter [-] [87057] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=87057) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:54:56.619631 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-569ca2fe-89a0-4776-a739-2d0d8f2b9795 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:54:56.621439 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569ca2fe-89a0-4776-a739-2d0d8f2b9795 admin admin] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:54:56.621740 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569ca2fe-89a0-4776-a739-2d0d8f2b9795 admin admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:54:56.622164 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569ca2fe-89a0-4776-a739-2d0d8f2b9795 admin admin] PATH_INFO: `/v3/roles` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:54:56.622934 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-569ca2fe-89a0-4776-a739-2d0d8f2b9795 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:54:56.624700 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-569ca2fe-89a0-4776-a739-2d0d8f2b9795 admin admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:54:56.643789 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 819/1627] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Aug 1 16:54:56 2022] GET /identity/v3/roles => generated 1420 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 16:54:56.643789 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-569ca2fe-89a0-4776-a739-2d0d8f2b9795 admin admin] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:54:56.646113 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e67dcb0e-6e19-42d4-9152-1742b00cd17d admin admin] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:54:56.658338 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-569ca2fe-89a0-4776-a739-2d0d8f2b9795 admin admin] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:54:56.660779 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e342d458-33c2-4b38-8065-2f6ce0533fb9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:54:56.661434 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e342d458-33c2-4b38-8065-2f6ce0533fb9 admin admin] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:54:56.662399 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e342d458-33c2-4b38-8065-2f6ce0533fb9 admin admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:54:56.662686 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e342d458-33c2-4b38-8065-2f6ce0533fb9 admin admin] PATH_INFO: `/v3/roles` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:54:56.663990 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e342d458-33c2-4b38-8065-2f6ce0533fb9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:54:56.665626 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e342d458-33c2-4b38-8065-2f6ce0533fb9 admin admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:54:56.676274 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d256b3-0012-4148-90ca-f3d7050109e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:54:56.677257 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d256b3-0012-4148-90ca-f3d7050109e8 admin admin] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:54:56.681252 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d256b3-0012-4148-90ca-f3d7050109e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:54:56.681252 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d256b3-0012-4148-90ca-f3d7050109e8 admin admin] PATH_INFO: `/v3/roles` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:54:56.681252 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80d256b3-0012-4148-90ca-f3d7050109e8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:54:56.682882 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80d256b3-0012-4148-90ca-f3d7050109e8 admin admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:54:56.686465 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 809/1628] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Aug 1 16:54:56 2022] GET /identity/v3/roles => generated 1420 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 16:54:56.691961 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e342d458-33c2-4b38-8065-2f6ce0533fb9 admin admin] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:54:56.711489 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 820/1629] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Aug 1 16:54:56 2022] GET /identity/v3/roles => generated 1420 bytes in 70 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 16:54:56.712354 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d256b3-0012-4148-90ca-f3d7050109e8 admin admin] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:54:56.718156 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e342d458-33c2-4b38-8065-2f6ce0533fb9 admin admin] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:54:56.739620 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76d4bdb8-0e6b-4edb-bacf-920f23c83a7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:54:56.742761 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d4bdb8-0e6b-4edb-bacf-920f23c83a7a admin admin] REQUEST_METHOD: `PUT` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:54:56.742761 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d4bdb8-0e6b-4edb-bacf-920f23c83a7a admin admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:54:56.742761 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d4bdb8-0e6b-4edb-bacf-920f23c83a7a admin admin] PATH_INFO: `/v3/projects/7267007bed914c8091878bd1f4773616/users/9e7dd0b9bb5743f4918fca51d577bcaa/roles/440bb943dd5b46fea038b903b6eaaf81` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:54:56.747653 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d256b3-0012-4148-90ca-f3d7050109e8 admin admin] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:54:56.759691 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de89ff2-aba9-46f2-bdba-6049936813fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:54:56.761084 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de89ff2-aba9-46f2-bdba-6049936813fc admin admin] REQUEST_METHOD: `PUT` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:54:56.761738 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de89ff2-aba9-46f2-bdba-6049936813fc admin admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:54:56.763017 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de89ff2-aba9-46f2-bdba-6049936813fc admin admin] PATH_INFO: `/v3/projects/acef9cbdf0714ec9ab8d08a0e5f915c7/users/f431c3af83eb482d91de9ec171831fed/roles/440bb943dd5b46fea038b903b6eaaf81` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:54:56.779518 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76d4bdb8-0e6b-4edb-bacf-920f23c83a7a admin admin] RBAC: Authorizing `identity:create_grant(project_id=7267007bed914c8091878bd1f4773616, user_id=9e7dd0b9bb5743f4918fca51d577bcaa, role_id=440bb943dd5b46fea038b903b6eaaf81)` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:54:56.782252 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76d4bdb8-0e6b-4edb-bacf-920f23c83a7a admin admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:54:56.786531 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1de89ff2-aba9-46f2-bdba-6049936813fc admin admin] RBAC: Authorizing `identity:create_grant(project_id=acef9cbdf0714ec9ab8d08a0e5f915c7, user_id=f431c3af83eb482d91de9ec171831fed, role_id=440bb943dd5b46fea038b903b6eaaf81)` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:54:56.789337 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1de89ff2-aba9-46f2-bdba-6049936813fc admin admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:54:56.892332 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 810/1630] 10.222.0.20 () {60 vars in 1483 bytes} [Mon Aug 1 16:54:56 2022] PUT /identity/v3/projects/7267007bed914c8091878bd1f4773616/users/9e7dd0b9bb5743f4918fca51d577bcaa/roles/440bb943dd5b46fea038b903b6eaaf81 => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 16:54:56.893914 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc4185f-5984-4c3a-976f-d943996fc0d8 None None] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:54:56.894249 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc4185f-5984-4c3a-976f-d943996fc0d8 None None] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:54:56.894526 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc4185f-5984-4c3a-976f-d943996fc0d8 None None] PATH_INFO: `/` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:54:56.895905 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 811/1631] 10.222.0.20 () {54 vars in 912 bytes} [Mon Aug 1 16:54:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 16:54:56.896471 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc4185f-5984-4c3a-976f-d943996fc0d8 None None] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:54:56.922566 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 821/1632] 10.222.0.20 () {60 vars in 1483 bytes} [Mon Aug 1 16:54:56 2022] PUT /identity/v3/projects/acef9cbdf0714ec9ab8d08a0e5f915c7/users/f431c3af83eb482d91de9ec171831fed/roles/440bb943dd5b46fea038b903b6eaaf81 => generated 0 bytes in 210 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 16:54:56.923311 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de89ff2-aba9-46f2-bdba-6049936813fc admin admin] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:54:57.096196 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de89ff2-aba9-46f2-bdba-6049936813fc admin admin] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:54:57.103960 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc4185f-5984-4c3a-976f-d943996fc0d8 None None] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:54:57.110365 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-562c0756-d127-4e4d-93b3-403ffb2fa29b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:54:57.111145 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562c0756-d127-4e4d-93b3-403ffb2fa29b admin admin] REQUEST_METHOD: `PUT` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:54:57.111489 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562c0756-d127-4e4d-93b3-403ffb2fa29b admin admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:54:57.117201 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562c0756-d127-4e4d-93b3-403ffb2fa29b admin admin] PATH_INFO: `/v3/projects/2d5994265bf54c1ab513dff4848f63ef/users/00aed21d40604d6088a5e0cdee4ef18f/roles/440bb943dd5b46fea038b903b6eaaf81` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:54:57.117368 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71999001-6ae6-491d-aa13-750db9b49124 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:54:57.117368 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71999001-6ae6-491d-aa13-750db9b49124 admin admin] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:54:57.117368 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71999001-6ae6-491d-aa13-750db9b49124 admin admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:54:57.117368 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71999001-6ae6-491d-aa13-750db9b49124 admin admin] PATH_INFO: `/v3/roles` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:54:57.117711 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71999001-6ae6-491d-aa13-750db9b49124 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:54:57.117797 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-562c0756-d127-4e4d-93b3-403ffb2fa29b admin admin] RBAC: Authorizing `identity:create_grant(project_id=2d5994265bf54c1ab513dff4848f63ef, user_id=00aed21d40604d6088a5e0cdee4ef18f, role_id=440bb943dd5b46fea038b903b6eaaf81)` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:54:57.118893 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-562c0756-d127-4e4d-93b3-403ffb2fa29b admin admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:54:57.121259 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71999001-6ae6-491d-aa13-750db9b49124 admin admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:54:57.141110 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 822/1633] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Aug 1 16:54:56 2022] GET /identity/v3/roles => generated 1420 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 16:54:57.152199 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 812/1634] 10.222.0.20 () {60 vars in 1483 bytes} [Mon Aug 1 16:54:56 2022] PUT /identity/v3/projects/2d5994265bf54c1ab513dff4848f63ef/users/00aed21d40604d6088a5e0cdee4ef18f/roles/440bb943dd5b46fea038b903b6eaaf81 => generated 0 bytes in 247 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 16:54:57.152569 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71999001-6ae6-491d-aa13-750db9b49124 admin admin] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:54:57.152675 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f1fa47-c2a5-42f3-84cb-1c2e16e16579 None None] REQUEST_METHOD: `POST` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:54:57.152675 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f1fa47-c2a5-42f3-84cb-1c2e16e16579 None None] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:54:57.152675 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f1fa47-c2a5-42f3-84cb-1c2e16e16579 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:54:57.162669 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71999001-6ae6-491d-aa13-750db9b49124 admin admin] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:54:57.186505 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.auth.core [None req-58f1fa47-c2a5-42f3-84cb-1c2e16e16579 None None] MFA Rules not processed for user `6533106d03eb4ca185a8839336d2aff9`. Rule list: `[]` (Enabled: `True`). {{(pid=87057) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:54:57.188090 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.fernet_utils [None req-58f1fa47-c2a5-42f3-84cb-1c2e16e16579 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87057) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:54:57.218869 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG dbcounter [-] [87058] Writing DB stats keystone:INSERT=7,keystone:SELECT=93 {{(pid=87058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:54:57.274709 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 813/1635] 10.222.0.20 () {58 vars in 1009 bytes} [Mon Aug 1 16:54:57 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:54:57.281813 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ec25b8-50eb-4302-9687-97ad0ac67a57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:54:57.282739 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f1fa47-c2a5-42f3-84cb-1c2e16e16579 None None] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:54:57.288441 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ec25b8-50eb-4302-9687-97ad0ac67a57 admin admin] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:54:57.288441 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ec25b8-50eb-4302-9687-97ad0ac67a57 admin admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:54:57.288441 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ec25b8-50eb-4302-9687-97ad0ac67a57 admin admin] PATH_INFO: `/v3/roles` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:54:57.288441 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43ec25b8-50eb-4302-9687-97ad0ac67a57 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:54:57.288441 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43ec25b8-50eb-4302-9687-97ad0ac67a57 admin admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:54:57.296557 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f1fa47-c2a5-42f3-84cb-1c2e16e16579 None None] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:54:57.301909 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a5f650a-cafb-4f8d-a4b5-afb07047dbe1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:54:57.303001 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5f650a-cafb-4f8d-a4b5-afb07047dbe1 admin admin] REQUEST_METHOD: `PUT` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:54:57.303281 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5f650a-cafb-4f8d-a4b5-afb07047dbe1 admin admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:54:57.303551 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5f650a-cafb-4f8d-a4b5-afb07047dbe1 admin admin] PATH_INFO: `/v3/projects/7267007bed914c8091878bd1f4773616/users/9e7dd0b9bb5743f4918fca51d577bcaa/roles/440bb943dd5b46fea038b903b6eaaf81` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:54:57.304340 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 823/1636] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Aug 1 16:54:57 2022] GET /identity/v3/roles => generated 1420 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 16:54:57.305854 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51757f40-62b6-4a65-ab1f-fa141b236d3f None None] REQUEST_METHOD: `POST` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:54:57.306428 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51757f40-62b6-4a65-ab1f-fa141b236d3f None None] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:54:57.308467 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51757f40-62b6-4a65-ab1f-fa141b236d3f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:54:57.310841 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a5f650a-cafb-4f8d-a4b5-afb07047dbe1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7267007bed914c8091878bd1f4773616, user_id=9e7dd0b9bb5743f4918fca51d577bcaa, role_id=440bb943dd5b46fea038b903b6eaaf81)` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:54:57.315882 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a5f650a-cafb-4f8d-a4b5-afb07047dbe1 admin admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:54:57.328586 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 814/1637] 10.222.0.20 () {60 vars in 1483 bytes} [Mon Aug 1 16:54:57 2022] PUT /identity/v3/projects/7267007bed914c8091878bd1f4773616/users/9e7dd0b9bb5743f4918fca51d577bcaa/roles/440bb943dd5b46fea038b903b6eaaf81 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 16:54:57.329590 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a5f650a-cafb-4f8d-a4b5-afb07047dbe1 admin admin] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:54:57.344065 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.auth.core [None req-51757f40-62b6-4a65-ab1f-fa141b236d3f None None] MFA Rules not processed for user `00aed21d40604d6088a5e0cdee4ef18f`. Rule list: `[]` (Enabled: `True`). {{(pid=87058) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:54:57.345459 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.fernet_utils [None req-51757f40-62b6-4a65-ab1f-fa141b236d3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:54:57.446071 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a5f650a-cafb-4f8d-a4b5-afb07047dbe1 admin admin] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:54:57.451382 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4959fd38-7a74-4ed1-8702-611ae9c96f89 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25970b52ca15448fa1abb1f974d64038', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b992d6a65df4bf9b27c13192d57c5f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:54:57.452841 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4959fd38-7a74-4ed1-8702-611ae9c96f89 service cinder] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:54:57.453452 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4959fd38-7a74-4ed1-8702-611ae9c96f89 service cinder] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:54:57.454317 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4959fd38-7a74-4ed1-8702-611ae9c96f89 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:54:57.463214 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 824/1638] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 1 16:54:57 2022] POST /identity/v3/auth/tokens => generated 3301 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:54:57.463214 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51757f40-62b6-4a65-ab1f-fa141b236d3f None None] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:54:57.509481 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG dbcounter [-] [87057] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=87057) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:54:57.564066 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4959fd38-7a74-4ed1-8702-611ae9c96f89 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:54:57.565528 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4959fd38-7a74-4ed1-8702-611ae9c96f89 service cinder] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:54:57.574746 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51757f40-62b6-4a65-ab1f-fa141b236d3f None None] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:54:57.590491 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20770f39-0fdd-4cca-8599-b80d46fa4d89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:54:57.590491 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20770f39-0fdd-4cca-8599-b80d46fa4d89 admin admin] REQUEST_METHOD: `PUT` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:54:57.590491 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20770f39-0fdd-4cca-8599-b80d46fa4d89 admin admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:54:57.590491 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20770f39-0fdd-4cca-8599-b80d46fa4d89 admin admin] PATH_INFO: `/v3/projects/acef9cbdf0714ec9ab8d08a0e5f915c7/users/f431c3af83eb482d91de9ec171831fed/roles/440bb943dd5b46fea038b903b6eaaf81` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:54:57.590868 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 815/1639] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 1 16:54:57 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 255 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 16:54:57.590868 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72d3f3d-27e1-4210-914c-09ea30158f38 None None] REQUEST_METHOD: `POST` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:54:57.591065 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20770f39-0fdd-4cca-8599-b80d46fa4d89 admin admin] RBAC: Authorizing `identity:create_grant(project_id=acef9cbdf0714ec9ab8d08a0e5f915c7, user_id=f431c3af83eb482d91de9ec171831fed, role_id=440bb943dd5b46fea038b903b6eaaf81)` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:54:57.591257 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72d3f3d-27e1-4210-914c-09ea30158f38 None None] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:54:57.591561 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20770f39-0fdd-4cca-8599-b80d46fa4d89 admin admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:54:57.591932 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72d3f3d-27e1-4210-914c-09ea30158f38 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:54:57.624973 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 825/1640] 10.222.0.20 () {60 vars in 1483 bytes} [Mon Aug 1 16:54:57 2022] PUT /identity/v3/projects/acef9cbdf0714ec9ab8d08a0e5f915c7/users/f431c3af83eb482d91de9ec171831fed/roles/440bb943dd5b46fea038b903b6eaaf81 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 16:54:57.624973 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20770f39-0fdd-4cca-8599-b80d46fa4d89 admin admin] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:54:57.664687 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.auth.core [None req-d72d3f3d-27e1-4210-914c-09ea30158f38 None None] MFA Rules not processed for user `9e7dd0b9bb5743f4918fca51d577bcaa`. Rule list: `[]` (Enabled: `True`). {{(pid=87057) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:54:57.666379 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.fernet_utils [None req-d72d3f3d-27e1-4210-914c-09ea30158f38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87057) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:54:57.758086 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20770f39-0fdd-4cca-8599-b80d46fa4d89 admin admin] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:54:57.764661 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da0770e-9f39-46b7-bb27-5b19144f4926 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25970b52ca15448fa1abb1f974d64038', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b992d6a65df4bf9b27c13192d57c5f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:54:57.765364 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da0770e-9f39-46b7-bb27-5b19144f4926 service cinder] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:54:57.765657 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da0770e-9f39-46b7-bb27-5b19144f4926 service cinder] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:54:57.765937 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da0770e-9f39-46b7-bb27-5b19144f4926 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:54:57.769294 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 816/1641] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 1 16:54:57 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:54:57.772443 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-543048a9-aa72-4529-ab92-179ed63b0bd4 None None] REQUEST_METHOD: `POST` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:54:57.772958 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-543048a9-aa72-4529-ab92-179ed63b0bd4 None None] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:54:57.773737 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-543048a9-aa72-4529-ab92-179ed63b0bd4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:54:57.829642 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.auth.core [None req-543048a9-aa72-4529-ab92-179ed63b0bd4 None None] MFA Rules not processed for user `f431c3af83eb482d91de9ec171831fed`. Rule list: `[]` (Enabled: `True`). {{(pid=87057) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:54:57.830803 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.fernet_utils [None req-543048a9-aa72-4529-ab92-179ed63b0bd4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87057) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:54:57.898871 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5da0770e-9f39-46b7-bb27-5b19144f4926 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:54:57.899228 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5da0770e-9f39-46b7-bb27-5b19144f4926 service cinder] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:54:57.920409 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 826/1642] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 1 16:54:57 2022] GET /identity/v3/auth/tokens => generated 3301 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 16:54:58.000348 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 817/1643] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 1 16:54:57 2022] POST /identity/v3/auth/tokens => generated 3303 bytes in 230 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:54:58.109962 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da0770e-9f39-46b7-bb27-5b19144f4926 service cinder] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:54:58.138666 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da0770e-9f39-46b7-bb27-5b19144f4926 service cinder] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:54:58.150563 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1df89684-a660-420c-aacb-34755935d45d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25970b52ca15448fa1abb1f974d64038', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b992d6a65df4bf9b27c13192d57c5f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:54:58.152002 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df89684-a660-420c-aacb-34755935d45d service cinder] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:54:58.152956 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df89684-a660-420c-aacb-34755935d45d service cinder] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:54:58.153330 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df89684-a660-420c-aacb-34755935d45d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:54:58.190517 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1df89684-a660-420c-aacb-34755935d45d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:54:58.194894 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1df89684-a660-420c-aacb-34755935d45d service cinder] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:54:58.214850 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 827/1644] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 1 16:54:58 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 16:54:58.375181 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG dbcounter [-] [87058] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:54:58.385040 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-543048a9-aa72-4529-ab92-179ed63b0bd4 None None] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:54:58.396482 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-543048a9-aa72-4529-ab92-179ed63b0bd4 None None] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:54:58.407592 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8440b7d-838e-4c2d-87ed-5dc0579af90d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25970b52ca15448fa1abb1f974d64038', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b992d6a65df4bf9b27c13192d57c5f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:54:58.409615 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8440b7d-838e-4c2d-87ed-5dc0579af90d service cinder] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:54:58.409905 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8440b7d-838e-4c2d-87ed-5dc0579af90d service cinder] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:54:58.410205 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8440b7d-838e-4c2d-87ed-5dc0579af90d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:54:58.427228 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8440b7d-838e-4c2d-87ed-5dc0579af90d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:54:58.428790 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8440b7d-838e-4c2d-87ed-5dc0579af90d service cinder] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:54:58.453306 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 818/1645] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 1 16:54:58 2022] GET /identity/v3/auth/tokens => generated 3303 bytes in 72 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 16:54:58.713802 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG dbcounter [-] [87057] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87057) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:54:59.156769 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1df89684-a660-420c-aacb-34755935d45d service cinder] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:54:59.156769 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1df89684-a660-420c-aacb-34755935d45d service cinder] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:54:59.156769 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-583bc380-139a-4810-8b2e-15404cb6c978 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25970b52ca15448fa1abb1f974d64038', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b992d6a65df4bf9b27c13192d57c5f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:54:59.156769 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583bc380-139a-4810-8b2e-15404cb6c978 service cinder] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:54:59.159944 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583bc380-139a-4810-8b2e-15404cb6c978 service cinder] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:54:59.159944 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583bc380-139a-4810-8b2e-15404cb6c978 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:54:59.161098 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31519bf9-f3cc-4df3-bdb9-0c759049f23c None None] REQUEST_METHOD: `POST` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:54:59.161098 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31519bf9-f3cc-4df3-bdb9-0c759049f23c None None] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:54:59.161098 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31519bf9-f3cc-4df3-bdb9-0c759049f23c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:54:59.164376 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-583bc380-139a-4810-8b2e-15404cb6c978 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:54:59.164376 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-583bc380-139a-4810-8b2e-15404cb6c978 service cinder] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:54:59.165060 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.auth.core [None req-31519bf9-f3cc-4df3-bdb9-0c759049f23c None None] MFA Rules not processed for user `9131d19bf929408fb5a194dc615b6b2a`. Rule list: `[]` (Enabled: `True`). {{(pid=87057) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:54:59.165060 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.fernet_utils [None req-31519bf9-f3cc-4df3-bdb9-0c759049f23c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87057) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:54:59.165274 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 828/1646] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 1 16:54:58 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 227 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 16:54:59.245978 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-583bc380-139a-4810-8b2e-15404cb6c978 service cinder] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:54:59.261741 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-583bc380-139a-4810-8b2e-15404cb6c978 service cinder] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:54:59.282272 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26def03d-aa36-43fd-b31b-db5a5d9171fb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25970b52ca15448fa1abb1f974d64038', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b992d6a65df4bf9b27c13192d57c5f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:54:59.283234 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26def03d-aa36-43fd-b31b-db5a5d9171fb service cinder] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:54:59.283520 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26def03d-aa36-43fd-b31b-db5a5d9171fb service cinder] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:54:59.283794 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26def03d-aa36-43fd-b31b-db5a5d9171fb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:54:59.427909 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 819/1647] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 1 16:54:59 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 371 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:54:59.432112 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31519bf9-f3cc-4df3-bdb9-0c759049f23c None None] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:54:59.485203 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26def03d-aa36-43fd-b31b-db5a5d9171fb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:54:59.487801 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26def03d-aa36-43fd-b31b-db5a5d9171fb service cinder] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:54:59.524277 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 829/1648] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 1 16:54:59 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 279 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 16:54:59.643773 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31519bf9-f3cc-4df3-bdb9-0c759049f23c None None] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:54:59.662402 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d256300a-b8d1-4a82-a209-cc3ae5b401fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:54:59.663218 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d256300a-b8d1-4a82-a209-cc3ae5b401fe admin admin] REQUEST_METHOD: `POST` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:54:59.663638 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d256300a-b8d1-4a82-a209-cc3ae5b401fe admin admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:54:59.664035 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d256300a-b8d1-4a82-a209-cc3ae5b401fe admin admin] PATH_INFO: `/v3/projects` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:54:59.664836 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d256300a-b8d1-4a82-a209-cc3ae5b401fe admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:54:59.667041 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d256300a-b8d1-4a82-a209-cc3ae5b401fe admin admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:54:59.841469 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a876a7-d326-43c1-baf7-d055816b43ef None None] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:54:59.842061 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a876a7-d326-43c1-baf7-d055816b43ef None None] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:54:59.842483 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a876a7-d326-43c1-baf7-d055816b43ef None None] PATH_INFO: `/` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:54:59.843974 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 830/1649] 10.222.0.20 () {54 vars in 962 bytes} [Mon Aug 1 16:54:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 01 16:54:59.852062 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 820/1650] 10.222.0.20 () {60 vars in 1153 bytes} [Mon Aug 1 16:54:59 2022] POST /identity/v3/projects => generated 405 bytes in 420 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 16:54:59.857139 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a876a7-d326-43c1-baf7-d055816b43ef None None] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:54:59.873874 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837b0f9d-ab26-4ec2-8ac6-88868e489dbd None None] REQUEST_METHOD: `POST` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:54:59.874684 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837b0f9d-ab26-4ec2-8ac6-88868e489dbd None None] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:54:59.880827 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837b0f9d-ab26-4ec2-8ac6-88868e489dbd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:54:59.892265 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG dbcounter [-] [87057] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87057) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:54:59.906682 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG dbcounter [-] [87058] Writing DB stats keystone:SELECT=100 {{(pid=87058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:54:59.937031 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.auth.core [None req-837b0f9d-ab26-4ec2-8ac6-88868e489dbd None None] MFA Rules not processed for user `6533106d03eb4ca185a8839336d2aff9`. Rule list: `[]` (Enabled: `True`). {{(pid=87057) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:54:59.939837 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.fernet_utils [None req-837b0f9d-ab26-4ec2-8ac6-88868e489dbd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87057) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:54:59.999703 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a876a7-d326-43c1-baf7-d055816b43ef None None] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:55:00.009280 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef276515-5ead-486f-8170-0673bf33ce03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:55:00.009919 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef276515-5ead-486f-8170-0673bf33ce03 admin admin] REQUEST_METHOD: `POST` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:55:00.010293 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef276515-5ead-486f-8170-0673bf33ce03 admin admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:55:00.010565 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef276515-5ead-486f-8170-0673bf33ce03 admin admin] PATH_INFO: `/v3/users` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:55:00.013648 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef276515-5ead-486f-8170-0673bf33ce03 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:55:00.016080 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef276515-5ead-486f-8170-0673bf33ce03 admin admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:55:00.101994 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 821/1651] 10.222.0.20 () {58 vars in 1059 bytes} [Mon Aug 1 16:54:59 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 229 msecs (HTTP/1.1 201) 6 headers in 385 bytes (3 switches on core 0) Aug 01 16:55:00.107103 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed11782-9879-4228-8863-8bbc0f8ed618 None None] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:55:00.107463 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed11782-9879-4228-8863-8bbc0f8ed618 None None] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:55:00.108299 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed11782-9879-4228-8863-8bbc0f8ed618 None None] PATH_INFO: `/` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:55:00.109551 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 822/1652] 10.222.0.20 () {54 vars in 912 bytes} [Mon Aug 1 16:55:00 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 16:55:00.115220 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1b2e1a-206c-4120-bcf2-d87bf6a20326 None None] REQUEST_METHOD: `POST` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:55:00.115509 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1b2e1a-206c-4120-bcf2-d87bf6a20326 None None] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:55:00.116151 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1b2e1a-206c-4120-bcf2-d87bf6a20326 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:55:00.144545 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 831/1653] 10.222.0.20 () {60 vars in 1144 bytes} [Mon Aug 1 16:54:59 2022] POST /identity/v3/users => generated 309 bytes in 286 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 16:55:00.145975 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef276515-5ead-486f-8170-0673bf33ce03 admin admin] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:55:00.147974 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.auth.core [None req-ef1b2e1a-206c-4120-bcf2-d87bf6a20326 None None] MFA Rules not processed for user `6533106d03eb4ca185a8839336d2aff9`. Rule list: `[]` (Enabled: `True`). {{(pid=87057) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:55:00.166251 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.fernet_utils [None req-ef1b2e1a-206c-4120-bcf2-d87bf6a20326 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87057) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:55:00.169502 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef276515-5ead-486f-8170-0673bf33ce03 admin admin] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:55:00.180270 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7179222-1976-4159-9275-ba78d786655a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:55:00.180913 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7179222-1976-4159-9275-ba78d786655a admin admin] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:55:00.186563 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7179222-1976-4159-9275-ba78d786655a admin admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:55:00.186563 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7179222-1976-4159-9275-ba78d786655a admin admin] PATH_INFO: `/v3/roles` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:55:00.186563 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7179222-1976-4159-9275-ba78d786655a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:55:00.186563 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7179222-1976-4159-9275-ba78d786655a admin admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:55:00.205414 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 832/1654] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Aug 1 16:55:00 2022] GET /identity/v3/roles => generated 1420 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 16:55:00.210290 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7179222-1976-4159-9275-ba78d786655a admin admin] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:55:00.228002 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 823/1655] 10.222.0.20 () {58 vars in 1009 bytes} [Mon Aug 1 16:55:00 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 16:55:00.236063 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7179222-1976-4159-9275-ba78d786655a admin admin] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:55:00.247723 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88e813e1-94bd-4c86-a0c7-fdd1bd272261 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:55:00.248512 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e813e1-94bd-4c86-a0c7-fdd1bd272261 admin admin] REQUEST_METHOD: `PUT` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:55:00.251964 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e813e1-94bd-4c86-a0c7-fdd1bd272261 admin admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:55:00.251964 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e813e1-94bd-4c86-a0c7-fdd1bd272261 admin admin] PATH_INFO: `/v3/projects/c90d55d32e114f11a69daef0107627aa/users/f56f528f94af4bdda6d150fa32be5194/roles/440bb943dd5b46fea038b903b6eaaf81` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:55:00.295586 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88e813e1-94bd-4c86-a0c7-fdd1bd272261 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c90d55d32e114f11a69daef0107627aa, user_id=f56f528f94af4bdda6d150fa32be5194, role_id=440bb943dd5b46fea038b903b6eaaf81)` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:55:00.301434 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88e813e1-94bd-4c86-a0c7-fdd1bd272261 admin admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:55:00.375149 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef1b2e1a-206c-4120-bcf2-d87bf6a20326 None None] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:55:00.431320 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 833/1656] 10.222.0.20 () {60 vars in 1483 bytes} [Mon Aug 1 16:55:00 2022] PUT /identity/v3/projects/c90d55d32e114f11a69daef0107627aa/users/f56f528f94af4bdda6d150fa32be5194/roles/440bb943dd5b46fea038b903b6eaaf81 => generated 0 bytes in 222 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 16:55:00.436040 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88e813e1-94bd-4c86-a0c7-fdd1bd272261 admin admin] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:55:00.558349 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef1b2e1a-206c-4120-bcf2-d87bf6a20326 None None] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:55:00.614241 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88e813e1-94bd-4c86-a0c7-fdd1bd272261 admin admin] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:55:00.629818 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eeda2aa-2cc0-4cbc-a52f-740d073eefad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:55:00.633212 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eeda2aa-2cc0-4cbc-a52f-740d073eefad admin admin] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:55:00.633641 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eeda2aa-2cc0-4cbc-a52f-740d073eefad admin admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:55:00.636018 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eeda2aa-2cc0-4cbc-a52f-740d073eefad admin admin] PATH_INFO: `/v3/roles` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:55:00.636018 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eeda2aa-2cc0-4cbc-a52f-740d073eefad admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:55:00.639526 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eeda2aa-2cc0-4cbc-a52f-740d073eefad admin admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:55:00.664472 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 834/1657] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Aug 1 16:55:00 2022] GET /identity/v3/roles => generated 1420 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 16:55:00.669479 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eeda2aa-2cc0-4cbc-a52f-740d073eefad admin admin] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:55:00.688982 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca87d45e-086f-4796-8af1-1d1ed76a0c2c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25970b52ca15448fa1abb1f974d64038', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b992d6a65df4bf9b27c13192d57c5f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:55:00.690425 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca87d45e-086f-4796-8af1-1d1ed76a0c2c service cinder] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:55:00.690425 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca87d45e-086f-4796-8af1-1d1ed76a0c2c service cinder] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:55:00.690811 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca87d45e-086f-4796-8af1-1d1ed76a0c2c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:55:00.697380 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eeda2aa-2cc0-4cbc-a52f-740d073eefad admin admin] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:55:00.716955 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca9e3ec-3730-44b1-93bd-a4a5ec9eb34e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:55:00.723414 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca9e3ec-3730-44b1-93bd-a4a5ec9eb34e admin admin] REQUEST_METHOD: `PUT` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:55:00.723808 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca9e3ec-3730-44b1-93bd-a4a5ec9eb34e admin admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:55:00.725137 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca9e3ec-3730-44b1-93bd-a4a5ec9eb34e admin admin] PATH_INFO: `/v3/projects/c90d55d32e114f11a69daef0107627aa/users/f56f528f94af4bdda6d150fa32be5194/roles/440bb943dd5b46fea038b903b6eaaf81` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:55:00.734497 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fca9e3ec-3730-44b1-93bd-a4a5ec9eb34e admin admin] RBAC: Authorizing `identity:create_grant(project_id=c90d55d32e114f11a69daef0107627aa, user_id=f56f528f94af4bdda6d150fa32be5194, role_id=440bb943dd5b46fea038b903b6eaaf81)` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:55:00.737425 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fca9e3ec-3730-44b1-93bd-a4a5ec9eb34e admin admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:55:00.778743 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 835/1658] 10.222.0.20 () {60 vars in 1483 bytes} [Mon Aug 1 16:55:00 2022] PUT /identity/v3/projects/c90d55d32e114f11a69daef0107627aa/users/f56f528f94af4bdda6d150fa32be5194/roles/440bb943dd5b46fea038b903b6eaaf81 => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 16:55:00.961481 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca87d45e-086f-4796-8af1-1d1ed76a0c2c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:55:00.963122 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca87d45e-086f-4796-8af1-1d1ed76a0c2c service cinder] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:55:01.004121 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 824/1659] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 1 16:55:00 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 630 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 16:55:01.421022 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG dbcounter [-] [87057] Writing DB stats keystone:SELECT=100 {{(pid=87057) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:55:04.258278 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29374ef-2ac1-48e7-a40f-4ef732f5dd13 None None] REQUEST_METHOD: `POST` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:55:04.258756 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29374ef-2ac1-48e7-a40f-4ef732f5dd13 None None] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:55:04.259474 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29374ef-2ac1-48e7-a40f-4ef732f5dd13 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:55:04.359109 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.auth.core [None req-f29374ef-2ac1-48e7-a40f-4ef732f5dd13 None None] MFA Rules not processed for user `f56f528f94af4bdda6d150fa32be5194`. Rule list: `[]` (Enabled: `True`). {{(pid=87058) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:55:04.360716 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.fernet_utils [None req-f29374ef-2ac1-48e7-a40f-4ef732f5dd13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:55:04.536178 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG dbcounter [-] [87058] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=87058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:55:04.609299 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 836/1660] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 1 16:55:04 2022] POST /identity/v3/auth/tokens => generated 3339 bytes in 352 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:55:05.280168 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ad4194-3177-4438-8dae-6fa912f101b7 None None] REQUEST_METHOD: `POST` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:55:05.280764 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ad4194-3177-4438-8dae-6fa912f101b7 None None] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:55:05.280764 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ad4194-3177-4438-8dae-6fa912f101b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:55:05.326663 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.auth.core [None req-86ad4194-3177-4438-8dae-6fa912f101b7 None None] MFA Rules not processed for user `0d90db97f4334c41ba8e0e54fc14c317`. Rule list: `[]` (Enabled: `True`). {{(pid=87057) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:55:05.327969 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.fernet_utils [None req-86ad4194-3177-4438-8dae-6fa912f101b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87057) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:55:05.411910 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 825/1661] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 1 16:55:05 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:55:05.414391 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be389083-30f6-4aa1-af6e-1ab83f58df85 None None] REQUEST_METHOD: `POST` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:55:05.414871 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be389083-30f6-4aa1-af6e-1ab83f58df85 None None] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:55:05.415066 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be389083-30f6-4aa1-af6e-1ab83f58df85 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:55:05.449704 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.auth.core [None req-be389083-30f6-4aa1-af6e-1ab83f58df85 None None] MFA Rules not processed for user `0d90db97f4334c41ba8e0e54fc14c317`. Rule list: `[]` (Enabled: `True`). {{(pid=87058) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:55:05.453456 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.fernet_utils [None req-be389083-30f6-4aa1-af6e-1ab83f58df85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:55:05.480194 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 837/1662] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 1 16:55:05 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 67 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:55:05.485861 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86ad4194-3177-4438-8dae-6fa912f101b7 None None] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:55:05.728896 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86ad4194-3177-4438-8dae-6fa912f101b7 None None] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:55:05.763009 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa88e1c8-6dfe-4c34-9752-1925c8441f8c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6533106d03eb4ca185a8839336d2aff9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b992d6a65df4bf9b27c13192d57c5f0', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:55:05.763839 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa88e1c8-6dfe-4c34-9752-1925c8441f8c service nova] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:55:05.764230 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa88e1c8-6dfe-4c34-9752-1925c8441f8c service nova] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:55:05.764607 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa88e1c8-6dfe-4c34-9752-1925c8441f8c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:55:05.796083 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa88e1c8-6dfe-4c34-9752-1925c8441f8c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:55:05.801000 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa88e1c8-6dfe-4c34-9752-1925c8441f8c service nova] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:55:05.840709 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 826/1663] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 1 16:55:05 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 354 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 16:55:06.569580 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446e1d34-87b8-4e74-b6f3-13f3788692c7 None None] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:55:06.570153 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446e1d34-87b8-4e74-b6f3-13f3788692c7 None None] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:55:06.570227 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446e1d34-87b8-4e74-b6f3-13f3788692c7 None None] PATH_INFO: `/` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:55:06.571617 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 838/1664] 10.222.0.20 () {54 vars in 921 bytes} [Mon Aug 1 16:55:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 16:55:06.581187 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d48270-4c5a-4e16-8cc3-86e38467d45d None None] REQUEST_METHOD: `POST` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:55:06.582429 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d48270-4c5a-4e16-8cc3-86e38467d45d None None] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:55:06.585641 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d48270-4c5a-4e16-8cc3-86e38467d45d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:55:06.620446 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.auth.core [None req-27d48270-4c5a-4e16-8cc3-86e38467d45d None None] MFA Rules not processed for user `8362293261614623bcdabc4120a97a9d`. Rule list: `[]` (Enabled: `True`). {{(pid=87057) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:55:06.621109 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.fernet_utils [None req-27d48270-4c5a-4e16-8cc3-86e38467d45d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87057) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:55:06.707367 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG dbcounter [-] [87057] Writing DB stats keystone:SELECT=100 {{(pid=87057) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:55:06.738286 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 827/1665] 10.222.0.20 () {58 vars in 1018 bytes} [Mon Aug 1 16:55:06 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:55:06.777885 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e2c65f-d3d7-4297-8ec2-11bea2e0a8a8 None None] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:55:06.778321 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e2c65f-d3d7-4297-8ec2-11bea2e0a8a8 None None] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:55:06.779230 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e2c65f-d3d7-4297-8ec2-11bea2e0a8a8 None None] PATH_INFO: `/` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:55:06.781031 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 839/1666] 10.222.0.20 () {54 vars in 961 bytes} [Mon Aug 1 16:55:06 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 16:55:06.788047 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b966c9c-4a16-4212-8142-d81a68dd9050 None None] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:55:06.788340 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b966c9c-4a16-4212-8142-d81a68dd9050 None None] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:55:06.789317 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da7d4ef9-96fc-45bf-b545-a287e338c2ae None None] REQUEST_METHOD: `POST` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:55:06.789600 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da7d4ef9-96fc-45bf-b545-a287e338c2ae None None] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:55:06.789755 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b966c9c-4a16-4212-8142-d81a68dd9050 None None] PATH_INFO: `/` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:55:06.791191 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da7d4ef9-96fc-45bf-b545-a287e338c2ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:55:06.793381 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 828/1667] 10.222.0.20 () {54 vars in 912 bytes} [Mon Aug 1 16:55:06 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 16:55:06.799557 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b48b1e-177d-4ce1-bda8-6047af4941b4 None None] REQUEST_METHOD: `POST` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:55:06.799855 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b48b1e-177d-4ce1-bda8-6047af4941b4 None None] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:55:06.800140 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b48b1e-177d-4ce1-bda8-6047af4941b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:55:06.850065 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.auth.core [None req-da7d4ef9-96fc-45bf-b545-a287e338c2ae None None] MFA Rules not processed for user `8362293261614623bcdabc4120a97a9d`. Rule list: `[]` (Enabled: `True`). {{(pid=87058) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:55:06.857407 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.fernet_utils [None req-da7d4ef9-96fc-45bf-b545-a287e338c2ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:55:06.906941 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 840/1668] 10.222.0.20 () {58 vars in 1058 bytes} [Mon Aug 1 16:55:06 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:55:06.911747 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.auth.core [None req-77b48b1e-177d-4ce1-bda8-6047af4941b4 None None] MFA Rules not processed for user `3d9bd8d649154c979be3744bfd016f95`. Rule list: `[]` (Enabled: `True`). {{(pid=87057) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:55:06.912823 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da7d4ef9-96fc-45bf-b545-a287e338c2ae None None] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:55:06.912921 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.fernet_utils [None req-77b48b1e-177d-4ce1-bda8-6047af4941b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87057) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:55:07.002882 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 829/1669] 10.222.0.20 () {58 vars in 1009 bytes} [Mon Aug 1 16:55:06 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 16:55:07.003793 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77b48b1e-177d-4ce1-bda8-6047af4941b4 None None] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:55:07.033806 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77b48b1e-177d-4ce1-bda8-6047af4941b4 None None] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:55:07.053615 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c33ffef-5fe4-437e-aac4-77774ca66fa9 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8362293261614623bcdabc4120a97a9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b992d6a65df4bf9b27c13192d57c5f0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:55:07.055726 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c33ffef-5fe4-437e-aac4-77774ca66fa9 service placement] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:55:07.058050 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c33ffef-5fe4-437e-aac4-77774ca66fa9 service placement] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:55:07.059008 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c33ffef-5fe4-437e-aac4-77774ca66fa9 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:55:07.074681 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c33ffef-5fe4-437e-aac4-77774ca66fa9 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:55:07.076351 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c33ffef-5fe4-437e-aac4-77774ca66fa9 service placement] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:55:07.078623 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da7d4ef9-96fc-45bf-b545-a287e338c2ae None None] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:55:07.095726 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f690462e-0d5d-40f4-8fe1-8785dcd91397 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:55:07.101384 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f690462e-0d5d-40f4-8fe1-8785dcd91397 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:55:07.101384 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f690462e-0d5d-40f4-8fe1-8785dcd91397 admin admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:55:07.101384 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f690462e-0d5d-40f4-8fe1-8785dcd91397 admin admin] PATH_INFO: `/v3/users/1cf7915cbe9547e0b48706aad57f2bc3` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:55:07.101384 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f690462e-0d5d-40f4-8fe1-8785dcd91397 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1cf7915cbe9547e0b48706aad57f2bc3)` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:55:07.105385 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f690462e-0d5d-40f4-8fe1-8785dcd91397 admin admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:55:07.105889 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 830/1670] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 1 16:55:07 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 16:55:07.106836 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c33ffef-5fe4-437e-aac4-77774ca66fa9 service placement] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:55:07.108798 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.fernet_utils [None req-7c33ffef-5fe4-437e-aac4-77774ca66fa9 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87057) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:55:07.146865 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c33ffef-5fe4-437e-aac4-77774ca66fa9 service placement] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:55:07.154239 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a14a07-8edc-4372-894e-8e40df69c94e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d9bd8d649154c979be3744bfd016f95', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b992d6a65df4bf9b27c13192d57c5f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:55:07.155108 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a14a07-8edc-4372-894e-8e40df69c94e service neutron] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:55:07.155534 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a14a07-8edc-4372-894e-8e40df69c94e service neutron] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:55:07.159886 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a14a07-8edc-4372-894e-8e40df69c94e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:55:07.179635 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86a14a07-8edc-4372-894e-8e40df69c94e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:55:07.181495 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86a14a07-8edc-4372-894e-8e40df69c94e service neutron] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:55:07.198134 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 831/1671] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 1 16:55:07 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 16:55:07.410783 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.notifications [None req-f690462e-0d5d-40f4-8fe1-8785dcd91397 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1cf7915cbe9547e0b48706aad57f2bc3'} {{(pid=87058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:55:07.411543 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.notifications [None req-f690462e-0d5d-40f4-8fe1-8785dcd91397 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1cf7915cbe9547e0b48706aad57f2bc3'} {{(pid=87058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:55:07.412173 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.notifications [None req-f690462e-0d5d-40f4-8fe1-8785dcd91397 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1cf7915cbe9547e0b48706aad57f2bc3'} {{(pid=87058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:55:07.442443 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.notifications [None req-f690462e-0d5d-40f4-8fe1-8785dcd91397 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1cf7915cbe9547e0b48706aad57f2bc3'} {{(pid=87058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:55:07.454369 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.notifications [None req-f690462e-0d5d-40f4-8fe1-8785dcd91397 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1cf7915cbe9547e0b48706aad57f2bc3'} {{(pid=87058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:55:07.561387 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 841/1672] 10.222.0.20 () {58 vars in 1224 bytes} [Mon Aug 1 16:55:06 2022] DELETE /identity/v3/users/1cf7915cbe9547e0b48706aad57f2bc3 => generated 0 bytes in 654 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 16:55:07.566329 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81795e78-9567-40d5-a211-530e7ea0ac35 None None] REQUEST_METHOD: `POST` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:55:07.566621 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81795e78-9567-40d5-a211-530e7ea0ac35 None None] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:55:07.568490 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81795e78-9567-40d5-a211-530e7ea0ac35 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:55:07.663304 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.auth.core [None req-81795e78-9567-40d5-a211-530e7ea0ac35 None None] MFA Rules not processed for user `1007f8e379b74e169b98fb7c3c0ba5b4`. Rule list: `[]` (Enabled: `True`). {{(pid=87057) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:55:07.664543 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.fernet_utils [None req-81795e78-9567-40d5-a211-530e7ea0ac35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87057) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:55:07.838777 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 832/1673] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 1 16:55:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 270 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:55:07.860634 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f690462e-0d5d-40f4-8fe1-8785dcd91397 admin admin] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:55:07.863274 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.fernet_utils [None req-f690462e-0d5d-40f4-8fe1-8785dcd91397 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:55:07.928069 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81795e78-9567-40d5-a211-530e7ea0ac35 None None] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:55:07.929967 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.fernet_utils [None req-81795e78-9567-40d5-a211-530e7ea0ac35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87057) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:55:07.951588 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81795e78-9567-40d5-a211-530e7ea0ac35 None None] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:55:07.957260 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0386fe12-aa65-4bb4-9a68-926214f403f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:55:07.962898 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0386fe12-aa65-4bb4-9a68-926214f403f5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:55:07.962898 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0386fe12-aa65-4bb4-9a68-926214f403f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:55:07.962898 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0386fe12-aa65-4bb4-9a68-926214f403f5 admin admin] PATH_INFO: `/v3/users/9131d19bf929408fb5a194dc615b6b2a` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:55:07.962898 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0386fe12-aa65-4bb4-9a68-926214f403f5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9131d19bf929408fb5a194dc615b6b2a)` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:55:07.962898 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: WARNING py.warnings [None req-0386fe12-aa65-4bb4-9a68-926214f403f5 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 01 16:55:07.962898 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: warnings.warn(msg) Aug 01 16:55:07.962898 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: Aug 01 16:55:07.963342 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0386fe12-aa65-4bb4-9a68-926214f403f5 admin admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:55:08.014127 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG dbcounter [-] [87058] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:55:08.023382 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f690462e-0d5d-40f4-8fe1-8785dcd91397 admin admin] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:55:08.029445 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44296287-396a-4f2d-9011-282b8c5ac46f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d9bd8d649154c979be3744bfd016f95', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b992d6a65df4bf9b27c13192d57c5f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:55:08.030636 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44296287-396a-4f2d-9011-282b8c5ac46f service neutron] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:55:08.031097 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44296287-396a-4f2d-9011-282b8c5ac46f service neutron] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:55:08.031861 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44296287-396a-4f2d-9011-282b8c5ac46f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:55:08.044161 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44296287-396a-4f2d-9011-282b8c5ac46f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:55:08.045993 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44296287-396a-4f2d-9011-282b8c5ac46f service neutron] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:55:08.061564 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 842/1674] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 1 16:55:07 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 201 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 16:55:08.171475 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.notifications [None req-0386fe12-aa65-4bb4-9a68-926214f403f5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9131d19bf929408fb5a194dc615b6b2a'} {{(pid=87057) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:55:08.174162 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.notifications [None req-0386fe12-aa65-4bb4-9a68-926214f403f5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9131d19bf929408fb5a194dc615b6b2a'} {{(pid=87057) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:55:08.368852 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.notifications [None req-0386fe12-aa65-4bb4-9a68-926214f403f5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9131d19bf929408fb5a194dc615b6b2a'} {{(pid=87057) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:55:08.369521 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.notifications [None req-0386fe12-aa65-4bb4-9a68-926214f403f5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9131d19bf929408fb5a194dc615b6b2a'} {{(pid=87057) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:55:08.382299 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44296287-396a-4f2d-9011-282b8c5ac46f service neutron] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:55:08.383468 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG dbcounter [-] [87057] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=87057) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:55:08.384485 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.fernet_utils [None req-44296287-396a-4f2d-9011-282b8c5ac46f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:55:08.388500 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.notifications [None req-0386fe12-aa65-4bb4-9a68-926214f403f5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9131d19bf929408fb5a194dc615b6b2a'} {{(pid=87057) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:55:08.446774 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 833/1675] 10.222.0.20 () {58 vars in 1224 bytes} [Mon Aug 1 16:55:07 2022] DELETE /identity/v3/users/9131d19bf929408fb5a194dc615b6b2a => generated 0 bytes in 516 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 16:55:08.572141 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44296287-396a-4f2d-9011-282b8c5ac46f service neutron] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:55:08.641899 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be46450-f1f2-4af7-98f1-4e84b2f6c91b None None] REQUEST_METHOD: `POST` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:55:08.642214 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be46450-f1f2-4af7-98f1-4e84b2f6c91b None None] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:55:08.642540 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be46450-f1f2-4af7-98f1-4e84b2f6c91b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:55:08.726885 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.auth.core [None req-4be46450-f1f2-4af7-98f1-4e84b2f6c91b None None] MFA Rules not processed for user `c2079b0fbf7b4a8c88d2522da9f2c73b`. Rule list: `[]` (Enabled: `True`). {{(pid=87057) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:55:08.730344 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.fernet_utils [None req-4be46450-f1f2-4af7-98f1-4e84b2f6c91b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87057) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:55:08.832354 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 834/1676] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 1 16:55:08 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:55:08.834266 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d61ff9-751e-4c6f-a095-376ef9ba0c8c service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8362293261614623bcdabc4120a97a9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b992d6a65df4bf9b27c13192d57c5f0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:55:08.835359 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d61ff9-751e-4c6f-a095-376ef9ba0c8c service placement] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:55:08.836044 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d61ff9-751e-4c6f-a095-376ef9ba0c8c service placement] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:55:08.841667 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d61ff9-751e-4c6f-a095-376ef9ba0c8c service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:55:08.846796 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.fernet_utils [None req-06d61ff9-751e-4c6f-a095-376ef9ba0c8c service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:55:08.860408 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be46450-f1f2-4af7-98f1-4e84b2f6c91b None None] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:55:08.862475 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.fernet_utils [None req-4be46450-f1f2-4af7-98f1-4e84b2f6c91b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87057) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:55:08.901451 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06d61ff9-751e-4c6f-a095-376ef9ba0c8c service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:55:08.917672 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06d61ff9-751e-4c6f-a095-376ef9ba0c8c service placement] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:55:08.964167 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 843/1677] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 1 16:55:08 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 582 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 01 16:55:09.047550 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be46450-f1f2-4af7-98f1-4e84b2f6c91b None None] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:55:09.071363 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2646678b-b652-407e-9eaa-c985af74961f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6533106d03eb4ca185a8839336d2aff9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b992d6a65df4bf9b27c13192d57c5f0', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:55:09.071363 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2646678b-b652-407e-9eaa-c985af74961f service nova] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:55:09.071827 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2646678b-b652-407e-9eaa-c985af74961f service nova] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:55:09.072042 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2646678b-b652-407e-9eaa-c985af74961f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:55:09.107924 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2646678b-b652-407e-9eaa-c985af74961f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:55:09.109831 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2646678b-b652-407e-9eaa-c985af74961f service nova] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:55:09.133331 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 835/1678] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 1 16:55:08 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 273 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 16:55:09.235552 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d61ff9-751e-4c6f-a095-376ef9ba0c8c service placement] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:55:09.237333 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.fernet_utils [None req-06d61ff9-751e-4c6f-a095-376ef9ba0c8c service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:55:09.408946 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2646678b-b652-407e-9eaa-c985af74961f service nova] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:55:09.411976 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.fernet_utils [None req-2646678b-b652-407e-9eaa-c985af74961f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87057) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:55:09.420773 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d61ff9-751e-4c6f-a095-376ef9ba0c8c service placement] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:55:09.444709 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c12eb159-6f38-49d0-afde-d8369e99f946 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:55:09.445409 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12eb159-6f38-49d0-afde-d8369e99f946 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:55:09.445691 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12eb159-6f38-49d0-afde-d8369e99f946 admin admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:55:09.446943 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12eb159-6f38-49d0-afde-d8369e99f946 admin admin] PATH_INFO: `/v3/projects/ee3a7c10d9b748e5a5481446ceb219fb` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:55:09.451085 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c12eb159-6f38-49d0-afde-d8369e99f946 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ee3a7c10d9b748e5a5481446ceb219fb)` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:55:09.451345 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2646678b-b652-407e-9eaa-c985af74961f service nova] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:55:09.461716 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: WARNING py.warnings [None req-c12eb159-6f38-49d0-afde-d8369e99f946 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 01 16:55:09.461716 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: warnings.warn(msg) Aug 01 16:55:09.461716 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: Aug 01 16:55:09.461914 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c8a5919-1213-4edf-9fc6-9e6291860213 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:55:09.461914 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8a5919-1213-4edf-9fc6-9e6291860213 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:55:09.461914 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8a5919-1213-4edf-9fc6-9e6291860213 admin admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:55:09.461914 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8a5919-1213-4edf-9fc6-9e6291860213 admin admin] PATH_INFO: `/v3/projects/56ae44822d9a49259d08ebc7a35e3a9e` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:55:09.462995 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c8a5919-1213-4edf-9fc6-9e6291860213 admin admin] RBAC: Authorizing `identity:delete_project(project_id=56ae44822d9a49259d08ebc7a35e3a9e)` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:55:09.467240 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c8a5919-1213-4edf-9fc6-9e6291860213 admin admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:55:09.477519 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c12eb159-6f38-49d0-afde-d8369e99f946 admin admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:55:09.580899 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG dbcounter [-] [87058] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=87058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:55:09.717340 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG dbcounter [-] [87057] Writing DB stats keystone:SELECT=94,keystone:DELETE=6 {{(pid=87057) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:55:09.720526 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.notifications [None req-1c8a5919-1213-4edf-9fc6-9e6291860213 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '56ae44822d9a49259d08ebc7a35e3a9e', 'request_id': 'req-1c8a5919-1213-4edf-9fc6-9e6291860213'} {{(pid=87057) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:55:09.751802 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.notifications [None req-c12eb159-6f38-49d0-afde-d8369e99f946 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ee3a7c10d9b748e5a5481446ceb219fb', 'request_id': 'req-c12eb159-6f38-49d0-afde-d8369e99f946'} {{(pid=87058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:55:09.754026 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.notifications [None req-1c8a5919-1213-4edf-9fc6-9e6291860213 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '56ae44822d9a49259d08ebc7a35e3a9e', 'request_id': 'req-1c8a5919-1213-4edf-9fc6-9e6291860213'} {{(pid=87057) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:55:09.775419 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.notifications [None req-1c8a5919-1213-4edf-9fc6-9e6291860213 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '56ae44822d9a49259d08ebc7a35e3a9e', 'request_id': 'req-1c8a5919-1213-4edf-9fc6-9e6291860213'} {{(pid=87057) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:55:09.803009 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.notifications [None req-c12eb159-6f38-49d0-afde-d8369e99f946 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ee3a7c10d9b748e5a5481446ceb219fb', 'request_id': 'req-c12eb159-6f38-49d0-afde-d8369e99f946'} {{(pid=87058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:55:09.826338 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.notifications [None req-c12eb159-6f38-49d0-afde-d8369e99f946 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ee3a7c10d9b748e5a5481446ceb219fb', 'request_id': 'req-c12eb159-6f38-49d0-afde-d8369e99f946'} {{(pid=87058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:55:09.881931 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.notifications [None req-1c8a5919-1213-4edf-9fc6-9e6291860213 admin admin] The token cache is being invalidate because project 56ae44822d9a49259d08ebc7a35e3a9e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87057) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 16:55:09.882330 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.notifications [None req-1c8a5919-1213-4edf-9fc6-9e6291860213 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87057) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:55:09.886142 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.notifications [None req-1c8a5919-1213-4edf-9fc6-9e6291860213 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87057) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:55:09.886142 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 836/1679] 10.222.0.20 () {58 vars in 1233 bytes} [Mon Aug 1 16:55:09 2022] DELETE /identity/v3/projects/56ae44822d9a49259d08ebc7a35e3a9e => generated 0 bytes in 478 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 16:55:09.889319 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c8a5919-1213-4edf-9fc6-9e6291860213 admin admin] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:55:09.891280 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.fernet_utils [None req-1c8a5919-1213-4edf-9fc6-9e6291860213 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87057) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:55:09.977491 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.notifications [None req-c12eb159-6f38-49d0-afde-d8369e99f946 admin admin] The token cache is being invalidate because project ee3a7c10d9b748e5a5481446ceb219fb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87058) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 16:55:09.978069 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.notifications [None req-c12eb159-6f38-49d0-afde-d8369e99f946 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:55:09.978862 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.notifications [None req-c12eb159-6f38-49d0-afde-d8369e99f946 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:55:09.985336 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 844/1680] 10.222.0.20 () {58 vars in 1233 bytes} [Mon Aug 1 16:55:09 2022] DELETE /identity/v3/projects/ee3a7c10d9b748e5a5481446ceb219fb => generated 0 bytes in 747 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 16:55:09.988481 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5beece-cc30-4c71-904f-30b4a1e4693a None None] REQUEST_METHOD: `POST` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:55:09.989692 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5beece-cc30-4c71-904f-30b4a1e4693a None None] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:55:09.990172 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5beece-cc30-4c71-904f-30b4a1e4693a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:55:10.036654 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c8a5919-1213-4edf-9fc6-9e6291860213 admin admin] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:55:10.068712 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d77169f2-fa42-430d-8155-9a49f1a7a178 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:55:10.068712 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77169f2-fa42-430d-8155-9a49f1a7a178 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:55:10.068904 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.auth.core [None req-3a5beece-cc30-4c71-904f-30b4a1e4693a None None] MFA Rules not processed for user `b353fdd302394227987a6ae8c2943173`. Rule list: `[]` (Enabled: `True`). {{(pid=87058) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:55:10.068985 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77169f2-fa42-430d-8155-9a49f1a7a178 admin admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:55:10.068985 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77169f2-fa42-430d-8155-9a49f1a7a178 admin admin] PATH_INFO: `/v3/users/f56f528f94af4bdda6d150fa32be5194` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:55:10.069650 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d77169f2-fa42-430d-8155-9a49f1a7a178 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f56f528f94af4bdda6d150fa32be5194)` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:55:10.072794 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d77169f2-fa42-430d-8155-9a49f1a7a178 admin admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:55:10.074560 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.fernet_utils [None req-3a5beece-cc30-4c71-904f-30b4a1e4693a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:55:10.246454 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 845/1681] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 1 16:55:09 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 260 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:55:10.248663 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a5beece-cc30-4c71-904f-30b4a1e4693a None None] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:55:10.251265 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.fernet_utils [None req-3a5beece-cc30-4c71-904f-30b4a1e4693a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:55:10.305679 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a5beece-cc30-4c71-904f-30b4a1e4693a None None] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:55:10.312680 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c1c5dfc-4d98-495e-8f68-5f2e86093f43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:55:10.313455 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1c5dfc-4d98-495e-8f68-5f2e86093f43 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:55:10.313776 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1c5dfc-4d98-495e-8f68-5f2e86093f43 admin admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:55:10.314121 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1c5dfc-4d98-495e-8f68-5f2e86093f43 admin admin] PATH_INFO: `/v3/users/f431c3af83eb482d91de9ec171831fed` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:55:10.315433 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c1c5dfc-4d98-495e-8f68-5f2e86093f43 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f431c3af83eb482d91de9ec171831fed)` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:55:10.317420 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c1c5dfc-4d98-495e-8f68-5f2e86093f43 admin admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:55:10.473712 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.notifications [None req-d77169f2-fa42-430d-8155-9a49f1a7a178 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f56f528f94af4bdda6d150fa32be5194'} {{(pid=87057) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:55:10.476942 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.notifications [None req-d77169f2-fa42-430d-8155-9a49f1a7a178 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f56f528f94af4bdda6d150fa32be5194'} {{(pid=87057) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:55:10.549796 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.notifications [None req-4c1c5dfc-4d98-495e-8f68-5f2e86093f43 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f431c3af83eb482d91de9ec171831fed'} {{(pid=87058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:55:10.554640 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.notifications [None req-4c1c5dfc-4d98-495e-8f68-5f2e86093f43 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f431c3af83eb482d91de9ec171831fed'} {{(pid=87058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:55:10.558449 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.notifications [None req-4c1c5dfc-4d98-495e-8f68-5f2e86093f43 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f431c3af83eb482d91de9ec171831fed'} {{(pid=87058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:55:10.563337 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.notifications [None req-d77169f2-fa42-430d-8155-9a49f1a7a178 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f56f528f94af4bdda6d150fa32be5194'} {{(pid=87057) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:55:10.570112 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.notifications [None req-d77169f2-fa42-430d-8155-9a49f1a7a178 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f56f528f94af4bdda6d150fa32be5194'} {{(pid=87057) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:55:10.575368 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.notifications [None req-d77169f2-fa42-430d-8155-9a49f1a7a178 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f56f528f94af4bdda6d150fa32be5194'} {{(pid=87057) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:55:10.611381 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.notifications [None req-4c1c5dfc-4d98-495e-8f68-5f2e86093f43 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f431c3af83eb482d91de9ec171831fed'} {{(pid=87058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:55:10.629099 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 837/1682] 10.222.0.20 () {58 vars in 1224 bytes} [Mon Aug 1 16:55:09 2022] DELETE /identity/v3/users/f56f528f94af4bdda6d150fa32be5194 => generated 0 bytes in 740 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 16:55:10.630607 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64fa5d1-eafe-474b-92d2-98525c8b57a1 None None] REQUEST_METHOD: `POST` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:55:10.631529 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64fa5d1-eafe-474b-92d2-98525c8b57a1 None None] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:55:10.631529 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64fa5d1-eafe-474b-92d2-98525c8b57a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:55:10.641847 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.notifications [None req-4c1c5dfc-4d98-495e-8f68-5f2e86093f43 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f431c3af83eb482d91de9ec171831fed'} {{(pid=87058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:55:10.680322 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.auth.core [None req-b64fa5d1-eafe-474b-92d2-98525c8b57a1 None None] MFA Rules not processed for user `b353fdd302394227987a6ae8c2943173`. Rule list: `[]` (Enabled: `True`). {{(pid=87057) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:55:10.683187 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.fernet_utils [None req-b64fa5d1-eafe-474b-92d2-98525c8b57a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87057) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:55:10.776916 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 846/1683] 10.222.0.20 () {58 vars in 1224 bytes} [Mon Aug 1 16:55:10 2022] DELETE /identity/v3/users/f431c3af83eb482d91de9ec171831fed => generated 0 bytes in 530 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 16:55:10.829659 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG dbcounter [-] [87057] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=87057) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:55:10.856001 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 838/1684] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 1 16:55:10 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 226 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:55:10.862385 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c1c5dfc-4d98-495e-8f68-5f2e86093f43 admin admin] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:55:10.864028 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.fernet_utils [None req-4c1c5dfc-4d98-495e-8f68-5f2e86093f43 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:55:10.919352 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG dbcounter [-] [87058] Writing DB stats keystone:SELECT=82,keystone:DELETE=16,keystone:INSERT=2 {{(pid=87058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:55:11.074356 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c1c5dfc-4d98-495e-8f68-5f2e86093f43 admin admin] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:55:11.097577 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db3987a-2b03-4dad-b835-878fa9ee4c50 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6533106d03eb4ca185a8839336d2aff9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b992d6a65df4bf9b27c13192d57c5f0', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:55:11.098348 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db3987a-2b03-4dad-b835-878fa9ee4c50 service nova] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:55:11.098746 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db3987a-2b03-4dad-b835-878fa9ee4c50 service nova] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:55:11.099116 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db3987a-2b03-4dad-b835-878fa9ee4c50 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:55:11.108183 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.fernet_utils [None req-5db3987a-2b03-4dad-b835-878fa9ee4c50 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:55:11.264452 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5db3987a-2b03-4dad-b835-878fa9ee4c50 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:55:11.267727 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5db3987a-2b03-4dad-b835-878fa9ee4c50 service nova] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:55:11.292564 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 847/1685] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 1 16:55:10 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 431 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 16:55:11.682681 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b64fa5d1-eafe-474b-92d2-98525c8b57a1 None None] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:55:11.685254 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.fernet_utils [None req-b64fa5d1-eafe-474b-92d2-98525c8b57a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87057) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:55:11.928034 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b64fa5d1-eafe-474b-92d2-98525c8b57a1 None None] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:55:12.100740 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a906ea29-73f8-4840-8279-d9b18a7516b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:55:12.102708 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a906ea29-73f8-4840-8279-d9b18a7516b0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:55:12.102999 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a906ea29-73f8-4840-8279-d9b18a7516b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:55:12.103276 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a906ea29-73f8-4840-8279-d9b18a7516b0 admin admin] PATH_INFO: `/v3/projects/c90d55d32e114f11a69daef0107627aa` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:55:12.112772 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a906ea29-73f8-4840-8279-d9b18a7516b0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c90d55d32e114f11a69daef0107627aa)` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:55:12.120139 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a906ea29-73f8-4840-8279-d9b18a7516b0 admin admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:55:12.326367 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.notifications [None req-a906ea29-73f8-4840-8279-d9b18a7516b0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c90d55d32e114f11a69daef0107627aa', 'request_id': 'req-a906ea29-73f8-4840-8279-d9b18a7516b0'} {{(pid=87057) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:55:12.344818 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.notifications [None req-a906ea29-73f8-4840-8279-d9b18a7516b0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c90d55d32e114f11a69daef0107627aa', 'request_id': 'req-a906ea29-73f8-4840-8279-d9b18a7516b0'} {{(pid=87057) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:55:12.354886 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.notifications [None req-a906ea29-73f8-4840-8279-d9b18a7516b0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c90d55d32e114f11a69daef0107627aa', 'request_id': 'req-a906ea29-73f8-4840-8279-d9b18a7516b0'} {{(pid=87057) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:55:12.408353 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db3987a-2b03-4dad-b835-878fa9ee4c50 service nova] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:55:12.415292 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.fernet_utils [None req-5db3987a-2b03-4dad-b835-878fa9ee4c50 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:55:12.432135 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db3987a-2b03-4dad-b835-878fa9ee4c50 service nova] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:55:12.480097 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.notifications [None req-a906ea29-73f8-4840-8279-d9b18a7516b0 admin admin] The token cache is being invalidate because project c90d55d32e114f11a69daef0107627aa was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87057) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 16:55:12.480388 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.notifications [None req-a906ea29-73f8-4840-8279-d9b18a7516b0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87057) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:55:12.483510 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.notifications [None req-a906ea29-73f8-4840-8279-d9b18a7516b0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87057) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:55:12.486447 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 839/1686] 10.222.0.20 () {58 vars in 1233 bytes} [Mon Aug 1 16:55:11 2022] DELETE /identity/v3/projects/c90d55d32e114f11a69daef0107627aa => generated 0 bytes in 808 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 16:55:12.491680 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1660513e-cbc3-40e6-95d5-6297bd3991f7 None None] REQUEST_METHOD: `POST` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:55:12.491961 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1660513e-cbc3-40e6-95d5-6297bd3991f7 None None] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:55:12.494164 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1660513e-cbc3-40e6-95d5-6297bd3991f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:55:12.574147 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.auth.core [None req-1660513e-cbc3-40e6-95d5-6297bd3991f7 None None] MFA Rules not processed for user `1007f8e379b74e169b98fb7c3c0ba5b4`. Rule list: `[]` (Enabled: `True`). {{(pid=87057) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:55:12.575402 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.fernet_utils [None req-1660513e-cbc3-40e6-95d5-6297bd3991f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87057) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:55:12.641409 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55a62710-b16f-4e7f-a421-4b0d7df3fad2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:55:12.643118 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a62710-b16f-4e7f-a421-4b0d7df3fad2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:55:12.643400 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a62710-b16f-4e7f-a421-4b0d7df3fad2 admin admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:55:12.644564 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a62710-b16f-4e7f-a421-4b0d7df3fad2 admin admin] PATH_INFO: `/v3/projects/acef9cbdf0714ec9ab8d08a0e5f915c7` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:55:12.649014 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55a62710-b16f-4e7f-a421-4b0d7df3fad2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=acef9cbdf0714ec9ab8d08a0e5f915c7)` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:55:12.654528 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55a62710-b16f-4e7f-a421-4b0d7df3fad2 admin admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:55:12.685162 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG dbcounter [-] [87057] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=87057) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:55:12.727319 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 840/1687] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 1 16:55:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 237 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:55:12.729718 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f78b38a-b785-4bbe-bfc7-ea38e7548614 None None] REQUEST_METHOD: `POST` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:55:12.730188 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f78b38a-b785-4bbe-bfc7-ea38e7548614 None None] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:55:12.730612 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f78b38a-b785-4bbe-bfc7-ea38e7548614 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:55:12.820861 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.auth.core [None req-4f78b38a-b785-4bbe-bfc7-ea38e7548614 None None] MFA Rules not processed for user `1007f8e379b74e169b98fb7c3c0ba5b4`. Rule list: `[]` (Enabled: `True`). {{(pid=87057) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:55:12.835558 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.notifications [None req-55a62710-b16f-4e7f-a421-4b0d7df3fad2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'acef9cbdf0714ec9ab8d08a0e5f915c7', 'request_id': 'req-55a62710-b16f-4e7f-a421-4b0d7df3fad2'} {{(pid=87058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:55:12.849275 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.fernet_utils [None req-4f78b38a-b785-4bbe-bfc7-ea38e7548614 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87057) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:55:12.860031 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.notifications [None req-55a62710-b16f-4e7f-a421-4b0d7df3fad2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'acef9cbdf0714ec9ab8d08a0e5f915c7', 'request_id': 'req-55a62710-b16f-4e7f-a421-4b0d7df3fad2'} {{(pid=87058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:55:12.877787 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG dbcounter [-] [87058] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=87058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:55:12.885835 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.notifications [None req-55a62710-b16f-4e7f-a421-4b0d7df3fad2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'acef9cbdf0714ec9ab8d08a0e5f915c7', 'request_id': 'req-55a62710-b16f-4e7f-a421-4b0d7df3fad2'} {{(pid=87058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:55:12.988876 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 841/1688] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 1 16:55:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 261 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:55:12.990250 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f303024b-b4fe-4516-b53b-cc3bbac3ae25 None None] REQUEST_METHOD: `POST` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:55:12.990542 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f303024b-b4fe-4516-b53b-cc3bbac3ae25 None None] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:55:12.990817 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f303024b-b4fe-4516-b53b-cc3bbac3ae25 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:55:12.998940 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.notifications [None req-55a62710-b16f-4e7f-a421-4b0d7df3fad2 admin admin] The token cache is being invalidate because project acef9cbdf0714ec9ab8d08a0e5f915c7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87058) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 16:55:12.999232 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.notifications [None req-55a62710-b16f-4e7f-a421-4b0d7df3fad2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:55:13.000343 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.notifications [None req-55a62710-b16f-4e7f-a421-4b0d7df3fad2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:55:13.002504 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 848/1689] 10.222.0.20 () {58 vars in 1233 bytes} [Mon Aug 1 16:55:12 2022] DELETE /identity/v3/projects/acef9cbdf0714ec9ab8d08a0e5f915c7 => generated 0 bytes in 594 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 16:55:13.003340 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55a62710-b16f-4e7f-a421-4b0d7df3fad2 admin admin] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:55:13.004918 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.fernet_utils [None req-55a62710-b16f-4e7f-a421-4b0d7df3fad2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:55:13.059350 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.auth.core [None req-f303024b-b4fe-4516-b53b-cc3bbac3ae25 None None] MFA Rules not processed for user `b353fdd302394227987a6ae8c2943173`. Rule list: `[]` (Enabled: `True`). {{(pid=87057) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:55:13.060603 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.fernet_utils [None req-f303024b-b4fe-4516-b53b-cc3bbac3ae25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87057) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:55:13.087482 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55a62710-b16f-4e7f-a421-4b0d7df3fad2 admin admin] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:55:13.103493 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5382276-a8b4-4a2a-97ba-2827c4bfad9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:55:13.110348 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5382276-a8b4-4a2a-97ba-2827c4bfad9c admin admin] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:55:13.110698 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5382276-a8b4-4a2a-97ba-2827c4bfad9c admin admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:55:13.110985 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5382276-a8b4-4a2a-97ba-2827c4bfad9c admin admin] PATH_INFO: `/v3/domains` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:55:13.111752 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5382276-a8b4-4a2a-97ba-2827c4bfad9c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87058) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:55:13.114654 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5382276-a8b4-4a2a-97ba-2827c4bfad9c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:55:13.118115 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5382276-a8b4-4a2a-97ba-2827c4bfad9c admin admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:55:13.167792 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 849/1690] 10.222.0.20 () {58 vars in 1153 bytes} [Mon Aug 1 16:55:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:55:13.169379 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac945f0-ec5a-41e5-9893-5aca2b1f5a14 None None] REQUEST_METHOD: `POST` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:55:13.169618 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac945f0-ec5a-41e5-9893-5aca2b1f5a14 None None] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:55:13.169890 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac945f0-ec5a-41e5-9893-5aca2b1f5a14 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:55:13.222558 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.auth.core [None req-cac945f0-ec5a-41e5-9893-5aca2b1f5a14 None None] MFA Rules not processed for user `1007f8e379b74e169b98fb7c3c0ba5b4`. Rule list: `[]` (Enabled: `True`). {{(pid=87058) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:55:13.222558 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.fernet_utils [None req-cac945f0-ec5a-41e5-9893-5aca2b1f5a14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:55:13.230282 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 850/1691] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 1 16:55:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 62 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:55:13.238340 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac945f0-ec5a-41e5-9893-5aca2b1f5a14 None None] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:55:13.249947 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac945f0-ec5a-41e5-9893-5aca2b1f5a14 None None] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:55:13.255314 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff6a456-f9e8-4e69-8926-45ea1aa91763 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:55:13.255959 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff6a456-f9e8-4e69-8926-45ea1aa91763 admin admin] REQUEST_METHOD: `POST` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:55:13.256238 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff6a456-f9e8-4e69-8926-45ea1aa91763 admin admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:55:13.256577 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff6a456-f9e8-4e69-8926-45ea1aa91763 admin admin] PATH_INFO: `/v3/projects` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:55:13.257257 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ff6a456-f9e8-4e69-8926-45ea1aa91763 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:55:13.259505 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ff6a456-f9e8-4e69-8926-45ea1aa91763 admin admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:55:13.286468 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 842/1692] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 1 16:55:12 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 297 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:55:13.288354 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7507a5-8942-458c-835b-53486a6c0439 None None] REQUEST_METHOD: `POST` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:55:13.289136 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7507a5-8942-458c-835b-53486a6c0439 None None] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:55:13.289411 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7507a5-8942-458c-835b-53486a6c0439 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:55:13.324365 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.auth.core [None req-ed7507a5-8942-458c-835b-53486a6c0439 None None] MFA Rules not processed for user `1007f8e379b74e169b98fb7c3c0ba5b4`. Rule list: `[]` (Enabled: `True`). {{(pid=87057) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:55:13.325779 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.fernet_utils [None req-ed7507a5-8942-458c-835b-53486a6c0439 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87057) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:55:13.370735 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 843/1693] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 1 16:55:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:55:13.374175 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed7507a5-8942-458c-835b-53486a6c0439 None None] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:55:13.395854 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed7507a5-8942-458c-835b-53486a6c0439 None None] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:55:13.402745 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b565fee2-7592-4e6d-a0bf-5f0000e8a414 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:55:13.403429 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b565fee2-7592-4e6d-a0bf-5f0000e8a414 admin admin] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:55:13.403745 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b565fee2-7592-4e6d-a0bf-5f0000e8a414 admin admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:55:13.404136 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b565fee2-7592-4e6d-a0bf-5f0000e8a414 admin admin] PATH_INFO: `/v3/domains` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:55:13.405038 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b565fee2-7592-4e6d-a0bf-5f0000e8a414 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87057) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:55:13.405409 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b565fee2-7592-4e6d-a0bf-5f0000e8a414 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:55:13.407241 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b565fee2-7592-4e6d-a0bf-5f0000e8a414 admin admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:55:13.432963 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 844/1694] 10.222.0.20 () {58 vars in 1153 bytes} [Mon Aug 1 16:55:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:55:13.435372 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b565fee2-7592-4e6d-a0bf-5f0000e8a414 admin admin] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:55:13.457710 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 851/1695] 10.222.0.20 () {60 vars in 1153 bytes} [Mon Aug 1 16:55:13 2022] POST /identity/v3/projects => generated 371 bytes in 227 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 16:55:13.462421 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff6a456-f9e8-4e69-8926-45ea1aa91763 admin admin] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:55:13.489323 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b565fee2-7592-4e6d-a0bf-5f0000e8a414 admin admin] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:55:13.662344 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff6a456-f9e8-4e69-8926-45ea1aa91763 admin admin] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:55:13.672910 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f79a360-d9fd-4526-ad1e-993d76cc866d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:55:13.674017 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f79a360-d9fd-4526-ad1e-993d76cc866d admin admin] REQUEST_METHOD: `POST` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:55:13.674658 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f79a360-d9fd-4526-ad1e-993d76cc866d admin admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:55:13.676253 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f79a360-d9fd-4526-ad1e-993d76cc866d admin admin] PATH_INFO: `/v3/users` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:55:13.676503 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5300ef8e-e871-49f8-b7c2-00d478f7dcef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:55:13.677117 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5300ef8e-e871-49f8-b7c2-00d478f7dcef admin admin] REQUEST_METHOD: `POST` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:55:13.677402 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5300ef8e-e871-49f8-b7c2-00d478f7dcef admin admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:55:13.677681 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5300ef8e-e871-49f8-b7c2-00d478f7dcef admin admin] PATH_INFO: `/v3/projects` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:55:13.677960 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f79a360-d9fd-4526-ad1e-993d76cc866d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:55:13.683464 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5300ef8e-e871-49f8-b7c2-00d478f7dcef admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:55:13.683464 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5300ef8e-e871-49f8-b7c2-00d478f7dcef admin admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:55:13.683649 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f79a360-d9fd-4526-ad1e-993d76cc866d admin admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:55:13.796732 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 845/1696] 10.222.0.20 () {60 vars in 1153 bytes} [Mon Aug 1 16:55:13 2022] POST /identity/v3/projects => generated 375 bytes in 362 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 16:55:13.801991 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5300ef8e-e871-49f8-b7c2-00d478f7dcef admin admin] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:55:13.910576 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 852/1697] 10.222.0.20 () {60 vars in 1144 bytes} [Mon Aug 1 16:55:13 2022] POST /identity/v3/users => generated 286 bytes in 448 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 16:55:13.914271 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f79a360-d9fd-4526-ad1e-993d76cc866d admin admin] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:55:13.931374 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5300ef8e-e871-49f8-b7c2-00d478f7dcef admin admin] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:55:13.938662 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65d8df41-2351-482f-8a1f-29291f072c65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:55:13.941180 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d8df41-2351-482f-8a1f-29291f072c65 admin admin] REQUEST_METHOD: `POST` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:55:13.941673 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d8df41-2351-482f-8a1f-29291f072c65 admin admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:55:13.942877 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d8df41-2351-482f-8a1f-29291f072c65 admin admin] PATH_INFO: `/v3/users` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:55:13.943349 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65d8df41-2351-482f-8a1f-29291f072c65 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:55:13.948498 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65d8df41-2351-482f-8a1f-29291f072c65 admin admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:55:13.950746 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f79a360-d9fd-4526-ad1e-993d76cc866d admin admin] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:55:13.969514 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97633eb1-b85a-42ec-9410-cf2dcfb7230d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:55:13.970762 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97633eb1-b85a-42ec-9410-cf2dcfb7230d admin admin] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:55:13.972389 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97633eb1-b85a-42ec-9410-cf2dcfb7230d admin admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:55:13.972899 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97633eb1-b85a-42ec-9410-cf2dcfb7230d admin admin] PATH_INFO: `/v3/roles` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:55:13.973881 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97633eb1-b85a-42ec-9410-cf2dcfb7230d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:55:13.978475 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97633eb1-b85a-42ec-9410-cf2dcfb7230d admin admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:55:14.006069 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 853/1698] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Aug 1 16:55:13 2022] GET /identity/v3/roles => generated 1420 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 16:55:14.008642 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97633eb1-b85a-42ec-9410-cf2dcfb7230d admin admin] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:55:14.031399 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97633eb1-b85a-42ec-9410-cf2dcfb7230d admin admin] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:55:14.046208 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35840c08-3bfc-4ff8-8fd3-ed542863aeab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:55:14.047056 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35840c08-3bfc-4ff8-8fd3-ed542863aeab admin admin] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:55:14.047435 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35840c08-3bfc-4ff8-8fd3-ed542863aeab admin admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:55:14.047808 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35840c08-3bfc-4ff8-8fd3-ed542863aeab admin admin] PATH_INFO: `/v3/roles` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:55:14.048742 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35840c08-3bfc-4ff8-8fd3-ed542863aeab admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:55:14.051777 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35840c08-3bfc-4ff8-8fd3-ed542863aeab admin admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:55:14.075812 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 854/1699] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Aug 1 16:55:14 2022] GET /identity/v3/roles => generated 1420 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 16:55:14.080969 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35840c08-3bfc-4ff8-8fd3-ed542863aeab admin admin] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:55:14.093552 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35840c08-3bfc-4ff8-8fd3-ed542863aeab admin admin] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:55:14.100283 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-640fb1ca-b1ed-4179-9ca4-b9d1482f2e69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:55:14.101521 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640fb1ca-b1ed-4179-9ca4-b9d1482f2e69 admin admin] REQUEST_METHOD: `PUT` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:55:14.102260 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640fb1ca-b1ed-4179-9ca4-b9d1482f2e69 admin admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:55:14.102663 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640fb1ca-b1ed-4179-9ca4-b9d1482f2e69 admin admin] PATH_INFO: `/v3/projects/06a3563c812c48a09f5645453ff535b0/users/bada6478e8e04b278bef503ff26768d8/roles/0999d3da63a545f18ff3252afdc90518` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:55:14.108878 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 846/1700] 10.222.0.20 () {60 vars in 1144 bytes} [Mon Aug 1 16:55:13 2022] POST /identity/v3/users => generated 288 bytes in 308 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 16:55:14.113763 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65d8df41-2351-482f-8a1f-29291f072c65 admin admin] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:55:14.124058 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-640fb1ca-b1ed-4179-9ca4-b9d1482f2e69 admin admin] RBAC: Authorizing `identity:create_grant(project_id=06a3563c812c48a09f5645453ff535b0, user_id=bada6478e8e04b278bef503ff26768d8, role_id=0999d3da63a545f18ff3252afdc90518)` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:55:14.128964 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-640fb1ca-b1ed-4179-9ca4-b9d1482f2e69 admin admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:55:14.134088 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65d8df41-2351-482f-8a1f-29291f072c65 admin admin] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:55:14.150926 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6164026f-4574-4d06-9a6f-922cfb1abcc9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:55:14.151525 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6164026f-4574-4d06-9a6f-922cfb1abcc9 admin admin] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:55:14.152444 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6164026f-4574-4d06-9a6f-922cfb1abcc9 admin admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:55:14.152444 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6164026f-4574-4d06-9a6f-922cfb1abcc9 admin admin] PATH_INFO: `/v3/roles` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:55:14.152813 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6164026f-4574-4d06-9a6f-922cfb1abcc9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:55:14.154756 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6164026f-4574-4d06-9a6f-922cfb1abcc9 admin admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:55:14.175686 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 847/1701] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Aug 1 16:55:14 2022] GET /identity/v3/roles => generated 1420 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 01 16:55:14.183572 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6164026f-4574-4d06-9a6f-922cfb1abcc9 admin admin] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:55:14.208441 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6164026f-4574-4d06-9a6f-922cfb1abcc9 admin admin] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:55:14.233311 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 855/1702] 10.222.0.20 () {60 vars in 1483 bytes} [Mon Aug 1 16:55:14 2022] PUT /identity/v3/projects/06a3563c812c48a09f5645453ff535b0/users/bada6478e8e04b278bef503ff26768d8/roles/0999d3da63a545f18ff3252afdc90518 => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 16:55:14.233510 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-030e5195-f976-40a8-a2e7-fd390b299efc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:55:14.233510 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030e5195-f976-40a8-a2e7-fd390b299efc admin admin] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:55:14.233510 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030e5195-f976-40a8-a2e7-fd390b299efc admin admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:55:14.233510 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030e5195-f976-40a8-a2e7-fd390b299efc admin admin] PATH_INFO: `/v3/roles` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:55:14.233842 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-030e5195-f976-40a8-a2e7-fd390b299efc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:55:14.233842 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-030e5195-f976-40a8-a2e7-fd390b299efc admin admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:55:14.237631 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e495914-6e78-4bc9-a6bb-33508a59812b None None] REQUEST_METHOD: `POST` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:55:14.238225 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e495914-6e78-4bc9-a6bb-33508a59812b None None] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:55:14.238695 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e495914-6e78-4bc9-a6bb-33508a59812b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:55:14.257964 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 848/1703] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Aug 1 16:55:14 2022] GET /identity/v3/roles => generated 1420 bytes in 75 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 16:55:14.264196 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-030e5195-f976-40a8-a2e7-fd390b299efc admin admin] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:55:14.288802 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-030e5195-f976-40a8-a2e7-fd390b299efc admin admin] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:55:14.309207 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.auth.core [None req-0e495914-6e78-4bc9-a6bb-33508a59812b None None] MFA Rules not processed for user `bada6478e8e04b278bef503ff26768d8`. Rule list: `[]` (Enabled: `True`). {{(pid=87058) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:55:14.310707 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.fernet_utils [None req-0e495914-6e78-4bc9-a6bb-33508a59812b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:55:14.378809 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG dbcounter [-] [87058] Writing DB stats keystone:SELECT=91,keystone:DELETE=3,keystone:INSERT=6 {{(pid=87058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:55:14.482465 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 856/1704] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 1 16:55:14 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 248 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:55:14.503504 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e495914-6e78-4bc9-a6bb-33508a59812b None None] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:55:14.513920 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73128eaa-a43c-4c9d-a884-0f0d2255a680 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:55:14.516464 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73128eaa-a43c-4c9d-a884-0f0d2255a680 admin admin] REQUEST_METHOD: `PUT` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:55:14.517771 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73128eaa-a43c-4c9d-a884-0f0d2255a680 admin admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:55:14.519421 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73128eaa-a43c-4c9d-a884-0f0d2255a680 admin admin] PATH_INFO: `/v3/projects/061e8ec4ecb7446d843e2161a8566f82/users/cf1a2114c87e46f28e884428b4bd4fd3/roles/0999d3da63a545f18ff3252afdc90518` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:55:14.524433 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG dbcounter [-] [87057] Writing DB stats keystone:SELECT=100 {{(pid=87057) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:55:14.548836 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e495914-6e78-4bc9-a6bb-33508a59812b None None] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:55:14.574846 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73128eaa-a43c-4c9d-a884-0f0d2255a680 admin admin] RBAC: Authorizing `identity:create_grant(project_id=061e8ec4ecb7446d843e2161a8566f82, user_id=cf1a2114c87e46f28e884428b4bd4fd3, role_id=0999d3da63a545f18ff3252afdc90518)` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:55:14.581626 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-657b93e0-fcd2-4bdd-808d-bc8581663f6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:55:14.582418 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657b93e0-fcd2-4bdd-808d-bc8581663f6b admin admin] REQUEST_METHOD: `POST` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:55:14.582646 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657b93e0-fcd2-4bdd-808d-bc8581663f6b admin admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:55:14.582973 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73128eaa-a43c-4c9d-a884-0f0d2255a680 admin admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:55:14.587023 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657b93e0-fcd2-4bdd-808d-bc8581663f6b admin admin] PATH_INFO: `/v3/projects` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:55:14.587820 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-657b93e0-fcd2-4bdd-808d-bc8581663f6b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:55:14.598999 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-657b93e0-fcd2-4bdd-808d-bc8581663f6b admin admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:55:14.697434 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 849/1705] 10.222.0.20 () {60 vars in 1483 bytes} [Mon Aug 1 16:55:14 2022] PUT /identity/v3/projects/061e8ec4ecb7446d843e2161a8566f82/users/cf1a2114c87e46f28e884428b4bd4fd3/roles/0999d3da63a545f18ff3252afdc90518 => generated 0 bytes in 435 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 16:55:14.700084 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00f5180-98a1-4c41-9b20-e0b65454b67a None None] REQUEST_METHOD: `POST` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:55:14.700593 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00f5180-98a1-4c41-9b20-e0b65454b67a None None] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:55:14.701042 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00f5180-98a1-4c41-9b20-e0b65454b67a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:55:14.719854 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 857/1706] 10.222.0.20 () {60 vars in 1153 bytes} [Mon Aug 1 16:55:14 2022] POST /identity/v3/projects => generated 371 bytes in 220 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 16:55:14.724609 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0abab2f-fc4e-4f33-bd28-5b656655aef6 None None] REQUEST_METHOD: `POST` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:55:14.724776 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0abab2f-fc4e-4f33-bd28-5b656655aef6 None None] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:55:14.725056 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0abab2f-fc4e-4f33-bd28-5b656655aef6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:55:14.749412 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.auth.core [None req-e00f5180-98a1-4c41-9b20-e0b65454b67a None None] MFA Rules not processed for user `c2079b0fbf7b4a8c88d2522da9f2c73b`. Rule list: `[]` (Enabled: `True`). {{(pid=87057) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:55:14.750868 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.fernet_utils [None req-e00f5180-98a1-4c41-9b20-e0b65454b67a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87057) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:55:14.781004 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.auth.core [None req-e0abab2f-fc4e-4f33-bd28-5b656655aef6 None None] MFA Rules not processed for user `cf1a2114c87e46f28e884428b4bd4fd3`. Rule list: `[]` (Enabled: `True`). {{(pid=87058) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:55:14.782433 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.fernet_utils [None req-e0abab2f-fc4e-4f33-bd28-5b656655aef6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:55:14.946243 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 850/1707] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 1 16:55:14 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 247 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:55:14.946611 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 858/1708] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 1 16:55:14 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 223 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:55:14.947551 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e00f5180-98a1-4c41-9b20-e0b65454b67a None None] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:55:14.962671 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0abab2f-fc4e-4f33-bd28-5b656655aef6 None None] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:55:15.160807 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0abab2f-fc4e-4f33-bd28-5b656655aef6 None None] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:55:15.175267 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e00f5180-98a1-4c41-9b20-e0b65454b67a None None] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:55:15.178444 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b4b2c7a-77c5-4117-8205-8d536bfa3e97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:55:15.180751 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4b2c7a-77c5-4117-8205-8d536bfa3e97 admin admin] REQUEST_METHOD: `POST` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:55:15.180751 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4b2c7a-77c5-4117-8205-8d536bfa3e97 admin admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:55:15.182320 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4b2c7a-77c5-4117-8205-8d536bfa3e97 admin admin] PATH_INFO: `/v3/projects` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:55:15.183449 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b4b2c7a-77c5-4117-8205-8d536bfa3e97 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:55:15.187084 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b4b2c7a-77c5-4117-8205-8d536bfa3e97 admin admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:55:15.190132 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02143223-9b3f-4396-aa91-fd54327fba7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:55:15.190935 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02143223-9b3f-4396-aa91-fd54327fba7f admin admin] REQUEST_METHOD: `POST` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:55:15.191275 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02143223-9b3f-4396-aa91-fd54327fba7f admin admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:55:15.191875 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02143223-9b3f-4396-aa91-fd54327fba7f admin admin] PATH_INFO: `/v3/users` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:55:15.196174 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02143223-9b3f-4396-aa91-fd54327fba7f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:55:15.199541 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02143223-9b3f-4396-aa91-fd54327fba7f admin admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:55:15.317814 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG dbcounter [-] [87057] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=87057) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:55:15.327668 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 859/1709] 10.222.0.20 () {60 vars in 1153 bytes} [Mon Aug 1 16:55:14 2022] POST /identity/v3/projects => generated 375 bytes in 367 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 16:55:15.331219 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b4b2c7a-77c5-4117-8205-8d536bfa3e97 admin admin] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:55:15.357056 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 851/1710] 10.222.0.20 () {60 vars in 1144 bytes} [Mon Aug 1 16:55:14 2022] POST /identity/v3/users => generated 292 bytes in 410 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 16:55:15.363334 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02143223-9b3f-4396-aa91-fd54327fba7f admin admin] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:55:15.393437 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02143223-9b3f-4396-aa91-fd54327fba7f admin admin] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:55:15.496249 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b4b2c7a-77c5-4117-8205-8d536bfa3e97 admin admin] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:55:15.498954 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5650c67c-cacb-46df-b890-352af5d580f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:55:15.500176 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5650c67c-cacb-46df-b890-352af5d580f0 admin admin] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:55:15.500470 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5650c67c-cacb-46df-b890-352af5d580f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:55:15.500752 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5650c67c-cacb-46df-b890-352af5d580f0 admin admin] PATH_INFO: `/v3/roles` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:55:15.501476 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5650c67c-cacb-46df-b890-352af5d580f0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:55:15.503578 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5650c67c-cacb-46df-b890-352af5d580f0 admin admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:55:15.517907 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c2a000-4091-4c0d-b895-72606f61a24b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:55:15.521647 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c2a000-4091-4c0d-b895-72606f61a24b admin admin] REQUEST_METHOD: `POST` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:55:15.521935 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c2a000-4091-4c0d-b895-72606f61a24b admin admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:55:15.522701 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c2a000-4091-4c0d-b895-72606f61a24b admin admin] PATH_INFO: `/v3/users` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:55:15.526691 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8c2a000-4091-4c0d-b895-72606f61a24b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:55:15.530605 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8c2a000-4091-4c0d-b895-72606f61a24b admin admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:55:15.531819 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 852/1711] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Aug 1 16:55:15 2022] GET /identity/v3/roles => generated 1420 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 16:55:15.535314 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5650c67c-cacb-46df-b890-352af5d580f0 admin admin] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:55:15.552777 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5650c67c-cacb-46df-b890-352af5d580f0 admin admin] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:55:15.560435 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a58e77-3aba-47ed-9d66-9444c1c68207 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:55:15.561120 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a58e77-3aba-47ed-9d66-9444c1c68207 admin admin] REQUEST_METHOD: `PUT` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:55:15.561396 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a58e77-3aba-47ed-9d66-9444c1c68207 admin admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:55:15.561723 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a58e77-3aba-47ed-9d66-9444c1c68207 admin admin] PATH_INFO: `/v3/projects/5ec6de45cc04415f8ef9b61536c18844/users/55ee94efc7dc4cffbf4ae58bdca723a3/roles/440bb943dd5b46fea038b903b6eaaf81` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:55:15.582552 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91a58e77-3aba-47ed-9d66-9444c1c68207 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5ec6de45cc04415f8ef9b61536c18844, user_id=55ee94efc7dc4cffbf4ae58bdca723a3, role_id=440bb943dd5b46fea038b903b6eaaf81)` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:55:15.586467 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91a58e77-3aba-47ed-9d66-9444c1c68207 admin admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:55:15.651358 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 860/1712] 10.222.0.20 () {60 vars in 1144 bytes} [Mon Aug 1 16:55:15 2022] POST /identity/v3/users => generated 294 bytes in 320 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 16:55:15.657853 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c2a000-4091-4c0d-b895-72606f61a24b admin admin] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:55:15.685050 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 853/1713] 10.222.0.20 () {60 vars in 1483 bytes} [Mon Aug 1 16:55:15 2022] PUT /identity/v3/projects/5ec6de45cc04415f8ef9b61536c18844/users/55ee94efc7dc4cffbf4ae58bdca723a3/roles/440bb943dd5b46fea038b903b6eaaf81 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 16:55:15.695459 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c2a000-4091-4c0d-b895-72606f61a24b admin admin] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:55:15.699228 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a58e77-3aba-47ed-9d66-9444c1c68207 admin admin] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:55:15.820813 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a58e77-3aba-47ed-9d66-9444c1c68207 admin admin] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:55:15.827769 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00022140-b67b-4c7a-87de-3855b5de2521 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:55:15.828422 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00022140-b67b-4c7a-87de-3855b5de2521 admin admin] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:55:15.828825 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00022140-b67b-4c7a-87de-3855b5de2521 admin admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:55:15.829098 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00022140-b67b-4c7a-87de-3855b5de2521 admin admin] PATH_INFO: `/v3/roles` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:55:15.830631 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00022140-b67b-4c7a-87de-3855b5de2521 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:55:15.835886 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00022140-b67b-4c7a-87de-3855b5de2521 admin admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:55:15.836241 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f249510-4c67-4a29-bb79-948c18775542 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:55:15.837135 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f249510-4c67-4a29-bb79-948c18775542 admin admin] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:55:15.837558 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f249510-4c67-4a29-bb79-948c18775542 admin admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:55:15.838853 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f249510-4c67-4a29-bb79-948c18775542 admin admin] PATH_INFO: `/v3/roles` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:55:15.839876 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f249510-4c67-4a29-bb79-948c18775542 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:55:15.842665 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f249510-4c67-4a29-bb79-948c18775542 admin admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:55:15.855504 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 861/1714] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Aug 1 16:55:15 2022] GET /identity/v3/roles => generated 1420 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 01 16:55:15.856475 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdfadefb-1b7e-48fc-bd73-ecb3d6db99ba None None] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:55:15.856569 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 854/1715] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Aug 1 16:55:15 2022] GET /identity/v3/roles => generated 1420 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 16:55:15.856911 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdfadefb-1b7e-48fc-bd73-ecb3d6db99ba None None] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:55:15.857194 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdfadefb-1b7e-48fc-bd73-ecb3d6db99ba None None] PATH_INFO: `/` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:55:15.860672 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 862/1716] 10.222.0.20 () {54 vars in 912 bytes} [Mon Aug 1 16:55:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 16:55:15.861727 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdfadefb-1b7e-48fc-bd73-ecb3d6db99ba None None] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:55:15.863265 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f249510-4c67-4a29-bb79-948c18775542 admin admin] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:55:15.882435 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f249510-4c67-4a29-bb79-948c18775542 admin admin] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:55:15.890657 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9193187e-c866-438c-985d-9ed547e46a9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:55:15.891868 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9193187e-c866-438c-985d-9ed547e46a9c admin admin] REQUEST_METHOD: `PUT` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:55:15.892330 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9193187e-c866-438c-985d-9ed547e46a9c admin admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:55:15.892865 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9193187e-c866-438c-985d-9ed547e46a9c admin admin] PATH_INFO: `/v3/projects/5ec6de45cc04415f8ef9b61536c18844/users/55ee94efc7dc4cffbf4ae58bdca723a3/roles/440bb943dd5b46fea038b903b6eaaf81` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:55:15.895162 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9193187e-c866-438c-985d-9ed547e46a9c admin admin] RBAC: Authorizing `identity:create_grant(project_id=5ec6de45cc04415f8ef9b61536c18844, user_id=55ee94efc7dc4cffbf4ae58bdca723a3, role_id=440bb943dd5b46fea038b903b6eaaf81)` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:55:15.897500 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdfadefb-1b7e-48fc-bd73-ecb3d6db99ba None None] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:55:15.908273 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9193187e-c866-438c-985d-9ed547e46a9c admin admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:55:15.918251 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19ebbe00-5fd8-437e-b3eb-68be8d521919 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:55:15.919248 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ebbe00-5fd8-437e-b3eb-68be8d521919 admin admin] REQUEST_METHOD: `PUT` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:55:15.920264 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ebbe00-5fd8-437e-b3eb-68be8d521919 admin admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:55:15.920622 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ebbe00-5fd8-437e-b3eb-68be8d521919 admin admin] PATH_INFO: `/v3/projects/7a466783f2934e2ba939506eadfc4326/users/109046d3dcad4e189a6476fc0e98e5ed/roles/440bb943dd5b46fea038b903b6eaaf81` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:55:15.976360 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 855/1717] 10.222.0.20 () {60 vars in 1483 bytes} [Mon Aug 1 16:55:15 2022] PUT /identity/v3/projects/5ec6de45cc04415f8ef9b61536c18844/users/55ee94efc7dc4cffbf4ae58bdca723a3/roles/440bb943dd5b46fea038b903b6eaaf81 => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 16:55:15.988728 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10384fc-ecc7-4665-a5f2-15abba82b6ef None None] REQUEST_METHOD: `POST` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:55:15.989631 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10384fc-ecc7-4665-a5f2-15abba82b6ef None None] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:55:15.990942 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10384fc-ecc7-4665-a5f2-15abba82b6ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:55:16.008559 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19ebbe00-5fd8-437e-b3eb-68be8d521919 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7a466783f2934e2ba939506eadfc4326, user_id=109046d3dcad4e189a6476fc0e98e5ed, role_id=440bb943dd5b46fea038b903b6eaaf81)` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:55:16.014593 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19ebbe00-5fd8-437e-b3eb-68be8d521919 admin admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:55:16.056085 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.auth.core [None req-b10384fc-ecc7-4665-a5f2-15abba82b6ef None None] MFA Rules not processed for user `3d9bd8d649154c979be3744bfd016f95`. Rule list: `[]` (Enabled: `True`). {{(pid=87057) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:55:16.056085 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.fernet_utils [None req-b10384fc-ecc7-4665-a5f2-15abba82b6ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87057) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:55:16.115329 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG dbcounter [-] [87058] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=87058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:55:16.158985 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 863/1718] 10.222.0.20 () {60 vars in 1483 bytes} [Mon Aug 1 16:55:15 2022] PUT /identity/v3/projects/7a466783f2934e2ba939506eadfc4326/users/109046d3dcad4e189a6476fc0e98e5ed/roles/440bb943dd5b46fea038b903b6eaaf81 => generated 0 bytes in 295 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 16:55:16.160512 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98ae116-527d-4ce7-9ba0-63d457b2d95b None None] REQUEST_METHOD: `POST` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:55:16.160805 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98ae116-527d-4ce7-9ba0-63d457b2d95b None None] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:55:16.161083 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98ae116-527d-4ce7-9ba0-63d457b2d95b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:55:16.179816 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 856/1719] 10.222.0.20 () {58 vars in 1009 bytes} [Mon Aug 1 16:55:15 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:55:16.180715 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b10384fc-ecc7-4665-a5f2-15abba82b6ef None None] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:55:16.222248 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.auth.core [None req-a98ae116-527d-4ce7-9ba0-63d457b2d95b None None] MFA Rules not processed for user `55ee94efc7dc4cffbf4ae58bdca723a3`. Rule list: `[]` (Enabled: `True`). {{(pid=87058) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:55:16.223979 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.fernet_utils [None req-a98ae116-527d-4ce7-9ba0-63d457b2d95b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:55:16.242944 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG dbcounter [-] [87057] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=87057) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:55:16.303248 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b10384fc-ecc7-4665-a5f2-15abba82b6ef None None] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:55:16.384585 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 864/1720] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 1 16:55:16 2022] POST /identity/v3/auth/tokens => generated 3305 bytes in 225 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:55:16.388241 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a98ae116-527d-4ce7-9ba0-63d457b2d95b None None] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:55:16.393511 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.fernet_utils [None req-a98ae116-527d-4ce7-9ba0-63d457b2d95b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:55:16.476457 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c98310d-2e6c-4d45-b3bb-fca94acbb7a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:55:16.477277 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c98310d-2e6c-4d45-b3bb-fca94acbb7a1 admin admin] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:55:16.477662 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c98310d-2e6c-4d45-b3bb-fca94acbb7a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:55:16.478056 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c98310d-2e6c-4d45-b3bb-fca94acbb7a1 admin admin] PATH_INFO: `/v3/roles` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:55:16.479051 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c98310d-2e6c-4d45-b3bb-fca94acbb7a1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:55:16.481189 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c98310d-2e6c-4d45-b3bb-fca94acbb7a1 admin admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:55:16.493910 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 857/1721] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Aug 1 16:55:16 2022] GET /identity/v3/roles => generated 1420 bytes in 313 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 16:55:16.494180 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c98310d-2e6c-4d45-b3bb-fca94acbb7a1 admin admin] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:55:16.499285 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.fernet_utils [None req-5c98310d-2e6c-4d45-b3bb-fca94acbb7a1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87057) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:55:16.536640 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a98ae116-527d-4ce7-9ba0-63d457b2d95b None None] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:55:16.543466 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d86216a-68ee-45f4-87e3-d608d55e6877 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:55:16.544145 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d86216a-68ee-45f4-87e3-d608d55e6877 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:55:16.544444 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d86216a-68ee-45f4-87e3-d608d55e6877 admin admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:55:16.544714 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d86216a-68ee-45f4-87e3-d608d55e6877 admin admin] PATH_INFO: `/v3/users/3abf76e9664a4723bfbac555c914d50d` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:55:16.545879 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d86216a-68ee-45f4-87e3-d608d55e6877 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3abf76e9664a4723bfbac555c914d50d)` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:55:16.548045 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d86216a-68ee-45f4-87e3-d608d55e6877 admin admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:55:16.573510 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c98310d-2e6c-4d45-b3bb-fca94acbb7a1 admin admin] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:55:16.574257 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2cbfd37-6bbf-429c-8444-712427bc19e0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d9bd8d649154c979be3744bfd016f95', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b992d6a65df4bf9b27c13192d57c5f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:55:16.574912 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2cbfd37-6bbf-429c-8444-712427bc19e0 service neutron] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:55:16.575236 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2cbfd37-6bbf-429c-8444-712427bc19e0 service neutron] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:55:16.575519 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2cbfd37-6bbf-429c-8444-712427bc19e0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:55:16.583742 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2cbfd37-6bbf-429c-8444-712427bc19e0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:55:16.585201 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2cbfd37-6bbf-429c-8444-712427bc19e0 service neutron] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:55:16.595912 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 858/1722] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 1 16:55:16 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 16:55:16.596883 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2cbfd37-6bbf-429c-8444-712427bc19e0 service neutron] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:55:16.598384 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.fernet_utils [None req-e2cbfd37-6bbf-429c-8444-712427bc19e0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87057) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:55:16.667889 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2cbfd37-6bbf-429c-8444-712427bc19e0 service neutron] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:55:16.672321 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a48b5ad-5c6b-4efc-a3e3-c1ff610d1881 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25970b52ca15448fa1abb1f974d64038', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b992d6a65df4bf9b27c13192d57c5f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:55:16.673164 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a48b5ad-5c6b-4efc-a3e3-c1ff610d1881 service cinder] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:55:16.673514 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a48b5ad-5c6b-4efc-a3e3-c1ff610d1881 service cinder] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:55:16.673860 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a48b5ad-5c6b-4efc-a3e3-c1ff610d1881 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:55:16.687815 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a48b5ad-5c6b-4efc-a3e3-c1ff610d1881 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:55:16.689494 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a48b5ad-5c6b-4efc-a3e3-c1ff610d1881 service cinder] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:55:16.704469 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 859/1723] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 1 16:55:16 2022] GET /identity/v3/auth/tokens => generated 3305 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 16:55:16.705492 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a48b5ad-5c6b-4efc-a3e3-c1ff610d1881 service cinder] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:55:16.719001 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a48b5ad-5c6b-4efc-a3e3-c1ff610d1881 service cinder] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:55:16.724153 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-125cce49-9e85-4a96-9ec9-1b4546d26499 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:55:16.724992 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125cce49-9e85-4a96-9ec9-1b4546d26499 admin admin] REQUEST_METHOD: `PUT` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:55:16.725373 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125cce49-9e85-4a96-9ec9-1b4546d26499 admin admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:55:16.725750 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125cce49-9e85-4a96-9ec9-1b4546d26499 admin admin] PATH_INFO: `/v3/projects/7a466783f2934e2ba939506eadfc4326/users/109046d3dcad4e189a6476fc0e98e5ed/roles/440bb943dd5b46fea038b903b6eaaf81` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:55:16.727947 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-125cce49-9e85-4a96-9ec9-1b4546d26499 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7a466783f2934e2ba939506eadfc4326, user_id=109046d3dcad4e189a6476fc0e98e5ed, role_id=440bb943dd5b46fea038b903b6eaaf81)` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:55:16.731015 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-125cce49-9e85-4a96-9ec9-1b4546d26499 admin admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:55:16.748605 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 860/1724] 10.222.0.20 () {60 vars in 1483 bytes} [Mon Aug 1 16:55:16 2022] PUT /identity/v3/projects/7a466783f2934e2ba939506eadfc4326/users/109046d3dcad4e189a6476fc0e98e5ed/roles/440bb943dd5b46fea038b903b6eaaf81 => generated 0 bytes in 44 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 16:55:16.761182 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f726cf-4cbc-489b-844c-aa943eae0a91 None None] REQUEST_METHOD: `POST` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:55:16.767854 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f726cf-4cbc-489b-844c-aa943eae0a91 None None] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:55:16.768779 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f726cf-4cbc-489b-844c-aa943eae0a91 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:55:16.804397 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.auth.core [None req-f8f726cf-4cbc-489b-844c-aa943eae0a91 None None] MFA Rules not processed for user `109046d3dcad4e189a6476fc0e98e5ed`. Rule list: `[]` (Enabled: `True`). {{(pid=87057) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:55:16.805797 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.fernet_utils [None req-f8f726cf-4cbc-489b-844c-aa943eae0a91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87057) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:55:16.826495 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.notifications [None req-5d86216a-68ee-45f4-87e3-d608d55e6877 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3abf76e9664a4723bfbac555c914d50d'} {{(pid=87058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:55:16.831829 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.notifications [None req-5d86216a-68ee-45f4-87e3-d608d55e6877 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3abf76e9664a4723bfbac555c914d50d'} {{(pid=87058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:55:16.834801 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.notifications [None req-5d86216a-68ee-45f4-87e3-d608d55e6877 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3abf76e9664a4723bfbac555c914d50d'} {{(pid=87058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:55:16.868836 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.notifications [None req-5d86216a-68ee-45f4-87e3-d608d55e6877 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3abf76e9664a4723bfbac555c914d50d'} {{(pid=87058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:55:16.884969 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG dbcounter [-] [87058] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=87058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:55:16.889548 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.notifications [None req-5d86216a-68ee-45f4-87e3-d608d55e6877 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3abf76e9664a4723bfbac555c914d50d'} {{(pid=87058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:55:16.946957 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG dbcounter [-] [87057] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87057) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:55:17.028078 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 865/1725] 10.222.0.20 () {58 vars in 1224 bytes} [Mon Aug 1 16:55:16 2022] DELETE /identity/v3/users/3abf76e9664a4723bfbac555c914d50d => generated 0 bytes in 643 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 16:55:17.034897 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c756fed8-432e-4f01-ae77-5aec803a67d2 None None] REQUEST_METHOD: `POST` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:55:17.035358 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 861/1726] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 1 16:55:16 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 276 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 16:55:17.035511 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c756fed8-432e-4f01-ae77-5aec803a67d2 None None] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:55:17.035792 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c756fed8-432e-4f01-ae77-5aec803a67d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:55:17.086333 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.auth.core [None req-c756fed8-432e-4f01-ae77-5aec803a67d2 None None] MFA Rules not processed for user `1007f8e379b74e169b98fb7c3c0ba5b4`. Rule list: `[]` (Enabled: `True`). {{(pid=87058) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:55:17.093064 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.fernet_utils [None req-c756fed8-432e-4f01-ae77-5aec803a67d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:55:17.228540 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 866/1727] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 1 16:55:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:55:17.270679 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8f726cf-4cbc-489b-844c-aa943eae0a91 None None] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:55:17.273003 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.fernet_utils [None req-f8f726cf-4cbc-489b-844c-aa943eae0a91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87057) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:55:17.332199 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c756fed8-432e-4f01-ae77-5aec803a67d2 None None] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:55:17.333665 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.fernet_utils [None req-c756fed8-432e-4f01-ae77-5aec803a67d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:55:17.455351 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c756fed8-432e-4f01-ae77-5aec803a67d2 None None] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:55:17.458896 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8f726cf-4cbc-489b-844c-aa943eae0a91 None None] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:55:17.460829 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cfa113c-27b8-4459-bb43-301a05bb859c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25970b52ca15448fa1abb1f974d64038', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b992d6a65df4bf9b27c13192d57c5f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:55:17.461995 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cfa113c-27b8-4459-bb43-301a05bb859c service cinder] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:55:17.462333 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cfa113c-27b8-4459-bb43-301a05bb859c service cinder] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:55:17.462778 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cfa113c-27b8-4459-bb43-301a05bb859c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:55:17.469341 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.fernet_utils [None req-4cfa113c-27b8-4459-bb43-301a05bb859c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:55:17.622310 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7207c7fd-66e4-4842-89d2-0cc4f7bab8eb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d9bd8d649154c979be3744bfd016f95', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b992d6a65df4bf9b27c13192d57c5f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:55:17.623211 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7207c7fd-66e4-4842-89d2-0cc4f7bab8eb service neutron] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:55:17.623501 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7207c7fd-66e4-4842-89d2-0cc4f7bab8eb service neutron] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:55:17.631058 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7207c7fd-66e4-4842-89d2-0cc4f7bab8eb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:55:17.644332 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cfa113c-27b8-4459-bb43-301a05bb859c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:55:17.646359 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cfa113c-27b8-4459-bb43-301a05bb859c service cinder] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:55:17.655590 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7207c7fd-66e4-4842-89d2-0cc4f7bab8eb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:55:17.657094 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7207c7fd-66e4-4842-89d2-0cc4f7bab8eb service neutron] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:55:17.666892 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 867/1728] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 1 16:55:17 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 335 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 01 16:55:17.668460 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034839db-9758-4dc4-9537-161aff57c53a None None] REQUEST_METHOD: `POST` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:55:17.668927 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034839db-9758-4dc4-9537-161aff57c53a None None] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:55:17.670734 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034839db-9758-4dc4-9537-161aff57c53a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:55:17.677657 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 862/1729] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 1 16:55:17 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 409 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 16:55:17.706985 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG dbcounter [-] [87058] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:55:17.729952 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG dbcounter [-] [87057] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87057) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:55:17.824418 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.auth.core [None req-034839db-9758-4dc4-9537-161aff57c53a None None] MFA Rules not processed for user `b6a93d1c6c82489099adc8b413280fc0`. Rule list: `[]` (Enabled: `True`). {{(pid=87058) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:55:17.827476 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.fernet_utils [None req-034839db-9758-4dc4-9537-161aff57c53a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:55:18.023933 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 868/1730] 10.222.0.20 () {58 vars in 1009 bytes} [Mon Aug 1 16:55:17 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 356 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:55:18.299804 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7207c7fd-66e4-4842-89d2-0cc4f7bab8eb service neutron] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:55:18.301395 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.fernet_utils [None req-7207c7fd-66e4-4842-89d2-0cc4f7bab8eb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87057) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:55:18.327871 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7207c7fd-66e4-4842-89d2-0cc4f7bab8eb service neutron] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:55:18.334721 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff9b1c4a-2209-4fa1-b707-e74555615bf9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:55:18.335480 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9b1c4a-2209-4fa1-b707-e74555615bf9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:55:18.335752 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9b1c4a-2209-4fa1-b707-e74555615bf9 admin admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:55:18.337753 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9b1c4a-2209-4fa1-b707-e74555615bf9 admin admin] PATH_INFO: `/v3/users/faebdbd71dfd4dbbbb4daa61de6b9806` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:55:18.337753 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff9b1c4a-2209-4fa1-b707-e74555615bf9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=faebdbd71dfd4dbbbb4daa61de6b9806)` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:55:18.342527 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff9b1c4a-2209-4fa1-b707-e74555615bf9 admin admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:55:18.502723 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-034839db-9758-4dc4-9537-161aff57c53a None None] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:55:18.504253 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.fernet_utils [None req-034839db-9758-4dc4-9537-161aff57c53a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:55:18.591156 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.notifications [None req-ff9b1c4a-2209-4fa1-b707-e74555615bf9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'faebdbd71dfd4dbbbb4daa61de6b9806'} {{(pid=87057) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:55:18.591907 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.notifications [None req-ff9b1c4a-2209-4fa1-b707-e74555615bf9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'faebdbd71dfd4dbbbb4daa61de6b9806'} {{(pid=87057) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:55:18.667005 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG dbcounter [-] [87058] Writing DB stats keystone:SELECT=100 {{(pid=87058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:55:18.700303 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.notifications [None req-ff9b1c4a-2209-4fa1-b707-e74555615bf9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'faebdbd71dfd4dbbbb4daa61de6b9806'} {{(pid=87057) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:55:18.701703 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.notifications [None req-ff9b1c4a-2209-4fa1-b707-e74555615bf9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'faebdbd71dfd4dbbbb4daa61de6b9806'} {{(pid=87057) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:55:18.713494 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.notifications [None req-ff9b1c4a-2209-4fa1-b707-e74555615bf9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'faebdbd71dfd4dbbbb4daa61de6b9806'} {{(pid=87057) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:55:18.749081 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-034839db-9758-4dc4-9537-161aff57c53a None None] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:55:18.762156 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-879a879f-5de5-46f2-b244-886d6d07ffff service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ab4add1faa04faa811ef7a9341cc35c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b992d6a65df4bf9b27c13192d57c5f0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:55:18.764080 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879a879f-5de5-46f2-b244-886d6d07ffff service swift] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:55:18.764656 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879a879f-5de5-46f2-b244-886d6d07ffff service swift] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:55:18.765075 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879a879f-5de5-46f2-b244-886d6d07ffff service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:55:18.771491 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.fernet_utils [None req-879a879f-5de5-46f2-b244-886d6d07ffff service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:55:18.778308 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 863/1731] 10.222.0.20 () {58 vars in 1224 bytes} [Mon Aug 1 16:55:18 2022] DELETE /identity/v3/users/faebdbd71dfd4dbbbb4daa61de6b9806 => generated 0 bytes in 476 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 16:55:18.781711 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62abfce4-6060-433e-94bb-c7173bfd3414 None None] REQUEST_METHOD: `POST` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:55:18.790536 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62abfce4-6060-433e-94bb-c7173bfd3414 None None] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:55:18.790847 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62abfce4-6060-433e-94bb-c7173bfd3414 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:55:18.831381 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-879a879f-5de5-46f2-b244-886d6d07ffff service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:55:18.842981 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-879a879f-5de5-46f2-b244-886d6d07ffff service swift] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:55:18.868930 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 869/1732] 10.222.0.20 () {60 vars in 1384 bytes} [Mon Aug 1 16:55:18 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 368 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Aug 01 16:55:18.880107 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.auth.core [None req-62abfce4-6060-433e-94bb-c7173bfd3414 None None] MFA Rules not processed for user `1007f8e379b74e169b98fb7c3c0ba5b4`. Rule list: `[]` (Enabled: `True`). {{(pid=87057) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:55:18.883153 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.fernet_utils [None req-62abfce4-6060-433e-94bb-c7173bfd3414 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87057) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:55:19.161682 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 864/1733] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 1 16:55:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 381 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:55:19.257438 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-879a879f-5de5-46f2-b244-886d6d07ffff service swift] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:55:19.259707 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.fernet_utils [None req-879a879f-5de5-46f2-b244-886d6d07ffff service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:55:19.278778 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-879a879f-5de5-46f2-b244-886d6d07ffff service swift] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:55:19.387366 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e7900e7-5d9d-495b-95eb-9abe6825e794 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d9bd8d649154c979be3744bfd016f95', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b992d6a65df4bf9b27c13192d57c5f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:55:19.388223 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7900e7-5d9d-495b-95eb-9abe6825e794 service neutron] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:55:19.388646 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7900e7-5d9d-495b-95eb-9abe6825e794 service neutron] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:55:19.389166 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7900e7-5d9d-495b-95eb-9abe6825e794 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:55:19.417342 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e7900e7-5d9d-495b-95eb-9abe6825e794 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:55:19.419094 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e7900e7-5d9d-495b-95eb-9abe6825e794 service neutron] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:55:19.476541 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 870/1734] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 1 16:55:19 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 216 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 16:55:19.542934 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62abfce4-6060-433e-94bb-c7173bfd3414 None None] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:55:19.543432 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.fernet_utils [None req-62abfce4-6060-433e-94bb-c7173bfd3414 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87057) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:55:19.557431 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74762921-b8ee-42d7-adc0-12f698c22df4 None None] REQUEST_METHOD: `POST` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:55:19.558682 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74762921-b8ee-42d7-adc0-12f698c22df4 None None] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:55:19.559128 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74762921-b8ee-42d7-adc0-12f698c22df4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:55:19.570640 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62abfce4-6060-433e-94bb-c7173bfd3414 None None] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:55:19.594308 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-413ae179-8a05-415d-bc29-44c2709f3dba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:55:19.595527 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413ae179-8a05-415d-bc29-44c2709f3dba admin admin] REQUEST_METHOD: `DELETE` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:55:19.596335 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413ae179-8a05-415d-bc29-44c2709f3dba admin admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:55:19.598456 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413ae179-8a05-415d-bc29-44c2709f3dba admin admin] PATH_INFO: `/v3/users/bada6478e8e04b278bef503ff26768d8` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:55:19.600918 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-413ae179-8a05-415d-bc29-44c2709f3dba admin admin] RBAC: Authorizing `identity:delete_user(user_id=bada6478e8e04b278bef503ff26768d8)` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:55:19.609384 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-413ae179-8a05-415d-bc29-44c2709f3dba admin admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:55:19.630661 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG dbcounter [-] [87057] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87057) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:55:19.632813 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.auth.core [None req-74762921-b8ee-42d7-adc0-12f698c22df4 None None] MFA Rules not processed for user `3d9bd8d649154c979be3744bfd016f95`. Rule list: `[]` (Enabled: `True`). {{(pid=87058) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:55:19.659320 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.fernet_utils [None req-74762921-b8ee-42d7-adc0-12f698c22df4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:55:19.682670 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 871/1735] 10.222.0.128 () {58 vars in 1016 bytes} [Mon Aug 1 16:55:19 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 16:55:19.714061 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74762921-b8ee-42d7-adc0-12f698c22df4 None None] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:55:19.739056 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74762921-b8ee-42d7-adc0-12f698c22df4 None None] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:55:19.760780 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d8f71e-0ef6-44b7-96c1-ed51b5604ae1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d9bd8d649154c979be3744bfd016f95', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b992d6a65df4bf9b27c13192d57c5f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:55:19.761926 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d8f71e-0ef6-44b7-96c1-ed51b5604ae1 service neutron] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:55:19.763663 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d8f71e-0ef6-44b7-96c1-ed51b5604ae1 service neutron] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:55:19.768263 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d8f71e-0ef6-44b7-96c1-ed51b5604ae1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:55:19.781665 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4d8f71e-0ef6-44b7-96c1-ed51b5604ae1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:55:19.783576 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4d8f71e-0ef6-44b7-96c1-ed51b5604ae1 service neutron] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:55:19.807026 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 872/1736] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 1 16:55:19 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 16:55:19.967638 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.notifications [None req-413ae179-8a05-415d-bc29-44c2709f3dba admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bada6478e8e04b278bef503ff26768d8'} {{(pid=87057) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:55:19.969766 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.notifications [None req-413ae179-8a05-415d-bc29-44c2709f3dba admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bada6478e8e04b278bef503ff26768d8'} {{(pid=87057) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:55:20.042464 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d8f71e-0ef6-44b7-96c1-ed51b5604ae1 service neutron] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:55:20.044220 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.fernet_utils [None req-c4d8f71e-0ef6-44b7-96c1-ed51b5604ae1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:55:20.061266 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.notifications [None req-413ae179-8a05-415d-bc29-44c2709f3dba admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bada6478e8e04b278bef503ff26768d8'} {{(pid=87057) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:55:20.062263 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.notifications [None req-413ae179-8a05-415d-bc29-44c2709f3dba admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bada6478e8e04b278bef503ff26768d8'} {{(pid=87057) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:55:20.077440 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.notifications [None req-413ae179-8a05-415d-bc29-44c2709f3dba admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bada6478e8e04b278bef503ff26768d8'} {{(pid=87057) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:55:20.081936 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d8f71e-0ef6-44b7-96c1-ed51b5604ae1 service neutron] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:55:20.103558 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49c4be9b-44e4-424d-8ad5-799f2b0cd0d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:55:20.107338 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c4be9b-44e4-424d-8ad5-799f2b0cd0d4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:55:20.108199 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c4be9b-44e4-424d-8ad5-799f2b0cd0d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:55:20.108689 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c4be9b-44e4-424d-8ad5-799f2b0cd0d4 admin admin] PATH_INFO: `/v3/projects/3730f131740e40af8d6d83d7844df33d` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:55:20.111340 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49c4be9b-44e4-424d-8ad5-799f2b0cd0d4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3730f131740e40af8d6d83d7844df33d)` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:55:20.115498 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49c4be9b-44e4-424d-8ad5-799f2b0cd0d4 admin admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:55:20.143802 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 865/1737] 10.222.0.20 () {58 vars in 1224 bytes} [Mon Aug 1 16:55:19 2022] DELETE /identity/v3/users/bada6478e8e04b278bef503ff26768d8 => generated 0 bytes in 603 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 16:55:20.162153 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008d15e6-5203-4473-8cb6-b8083f26d4e1 None None] REQUEST_METHOD: `POST` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:55:20.162153 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008d15e6-5203-4473-8cb6-b8083f26d4e1 None None] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:55:20.162153 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008d15e6-5203-4473-8cb6-b8083f26d4e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:55:20.222289 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.auth.core [None req-008d15e6-5203-4473-8cb6-b8083f26d4e1 None None] MFA Rules not processed for user `1007f8e379b74e169b98fb7c3c0ba5b4`. Rule list: `[]` (Enabled: `True`). {{(pid=87057) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:55:20.224693 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.fernet_utils [None req-008d15e6-5203-4473-8cb6-b8083f26d4e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87057) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:55:20.336334 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.notifications [None req-49c4be9b-44e4-424d-8ad5-799f2b0cd0d4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3730f131740e40af8d6d83d7844df33d', 'request_id': 'req-49c4be9b-44e4-424d-8ad5-799f2b0cd0d4'} {{(pid=87058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:55:20.348176 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.notifications [None req-49c4be9b-44e4-424d-8ad5-799f2b0cd0d4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3730f131740e40af8d6d83d7844df33d', 'request_id': 'req-49c4be9b-44e4-424d-8ad5-799f2b0cd0d4'} {{(pid=87058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:55:20.355299 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 866/1738] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 1 16:55:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:55:20.360090 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.notifications [None req-49c4be9b-44e4-424d-8ad5-799f2b0cd0d4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3730f131740e40af8d6d83d7844df33d', 'request_id': 'req-49c4be9b-44e4-424d-8ad5-799f2b0cd0d4'} {{(pid=87058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:55:20.415903 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-008d15e6-5203-4473-8cb6-b8083f26d4e1 None None] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:55:20.417629 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.fernet_utils [None req-008d15e6-5203-4473-8cb6-b8083f26d4e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87057) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:55:20.516810 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-008d15e6-5203-4473-8cb6-b8083f26d4e1 None None] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:55:20.756465 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.notifications [None req-49c4be9b-44e4-424d-8ad5-799f2b0cd0d4 admin admin] The token cache is being invalidate because project 3730f131740e40af8d6d83d7844df33d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87058) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 16:55:20.756465 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.notifications [None req-49c4be9b-44e4-424d-8ad5-799f2b0cd0d4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:55:20.756465 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.notifications [None req-49c4be9b-44e4-424d-8ad5-799f2b0cd0d4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:55:20.756465 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 873/1739] 10.222.0.20 () {58 vars in 1233 bytes} [Mon Aug 1 16:55:20 2022] DELETE /identity/v3/projects/3730f131740e40af8d6d83d7844df33d => generated 0 bytes in 531 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 16:55:20.757868 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba0ce9d-df09-4ede-9075-a23a46401013 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d9bd8d649154c979be3744bfd016f95', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b992d6a65df4bf9b27c13192d57c5f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:55:20.757868 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba0ce9d-df09-4ede-9075-a23a46401013 service neutron] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:55:20.757868 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba0ce9d-df09-4ede-9075-a23a46401013 service neutron] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:55:20.757868 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba0ce9d-df09-4ede-9075-a23a46401013 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:55:20.758896 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.fernet_utils [None req-bba0ce9d-df09-4ede-9075-a23a46401013 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87057) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:55:20.758896 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bba0ce9d-df09-4ede-9075-a23a46401013 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:55:20.758896 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bba0ce9d-df09-4ede-9075-a23a46401013 service neutron] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:55:20.759120 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167d35d7-4fde-4390-ae07-c1586491902e None None] REQUEST_METHOD: `POST` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:55:20.759120 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167d35d7-4fde-4390-ae07-c1586491902e None None] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:55:20.759120 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167d35d7-4fde-4390-ae07-c1586491902e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:55:20.759437 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 867/1740] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 1 16:55:20 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 270 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 01 16:55:20.759529 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.auth.core [None req-167d35d7-4fde-4390-ae07-c1586491902e None None] MFA Rules not processed for user `1007f8e379b74e169b98fb7c3c0ba5b4`. Rule list: `[]` (Enabled: `True`). {{(pid=87058) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:55:20.759529 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.fernet_utils [None req-167d35d7-4fde-4390-ae07-c1586491902e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:55:20.759529 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 874/1741] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 1 16:55:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 69 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:55:20.764616 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44da4ef1-fb4d-42b4-b227-7f7b68fa7c5c None None] REQUEST_METHOD: `POST` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:55:20.765258 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44da4ef1-fb4d-42b4-b227-7f7b68fa7c5c None None] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:55:20.765683 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44da4ef1-fb4d-42b4-b227-7f7b68fa7c5c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:55:20.793906 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.auth.core [None req-44da4ef1-fb4d-42b4-b227-7f7b68fa7c5c None None] MFA Rules not processed for user `1007f8e379b74e169b98fb7c3c0ba5b4`. Rule list: `[]` (Enabled: `True`). {{(pid=87057) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:55:20.795104 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.fernet_utils [None req-44da4ef1-fb4d-42b4-b227-7f7b68fa7c5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87057) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:55:20.825192 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 868/1742] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 1 16:55:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 63 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:55:20.831284 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-167d35d7-4fde-4390-ae07-c1586491902e None None] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:55:20.849847 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-167d35d7-4fde-4390-ae07-c1586491902e None None] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:55:20.879731 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0224af7e-ee83-486b-898d-195dcec5e9ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:55:20.879731 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0224af7e-ee83-486b-898d-195dcec5e9ef admin admin] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:55:20.879731 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0224af7e-ee83-486b-898d-195dcec5e9ef admin admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:55:20.879731 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0224af7e-ee83-486b-898d-195dcec5e9ef admin admin] PATH_INFO: `/v3/domains` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:55:20.880444 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0224af7e-ee83-486b-898d-195dcec5e9ef admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87058) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:55:20.880444 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0224af7e-ee83-486b-898d-195dcec5e9ef admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:55:20.880444 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0224af7e-ee83-486b-898d-195dcec5e9ef admin admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:55:20.889364 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44da4ef1-fb4d-42b4-b227-7f7b68fa7c5c None None] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:55:20.897018 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 875/1743] 10.222.0.20 () {58 vars in 1153 bytes} [Mon Aug 1 16:55:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 16:55:20.904273 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44da4ef1-fb4d-42b4-b227-7f7b68fa7c5c None None] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:55:20.909883 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e3c582a-e83d-42b0-816a-d1d3a8531143 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:55:20.915070 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3c582a-e83d-42b0-816a-d1d3a8531143 admin admin] REQUEST_METHOD: `POST` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:55:20.915070 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3c582a-e83d-42b0-816a-d1d3a8531143 admin admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:55:20.915070 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3c582a-e83d-42b0-816a-d1d3a8531143 admin admin] PATH_INFO: `/v3/projects` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:55:20.915070 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e3c582a-e83d-42b0-816a-d1d3a8531143 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:55:20.915070 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e3c582a-e83d-42b0-816a-d1d3a8531143 admin admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:55:21.107412 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0224af7e-ee83-486b-898d-195dcec5e9ef admin admin] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:55:21.118545 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG dbcounter [-] [87057] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87057) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:55:21.122238 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 869/1744] 10.222.0.20 () {60 vars in 1153 bytes} [Mon Aug 1 16:55:20 2022] POST /identity/v3/projects => generated 403 bytes in 230 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 16:55:21.134002 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0224af7e-ee83-486b-898d-195dcec5e9ef admin admin] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:55:21.182515 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG dbcounter [-] [87058] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=87058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:55:21.265358 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77ee59b1-3bd1-48b0-8341-98b088a7e4f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:55:21.266090 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ee59b1-3bd1-48b0-8341-98b088a7e4f0 admin admin] REQUEST_METHOD: `POST` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:55:21.266381 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ee59b1-3bd1-48b0-8341-98b088a7e4f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:55:21.266697 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ee59b1-3bd1-48b0-8341-98b088a7e4f0 admin admin] PATH_INFO: `/v3/users` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:55:21.267383 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77ee59b1-3bd1-48b0-8341-98b088a7e4f0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:55:21.269948 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77ee59b1-3bd1-48b0-8341-98b088a7e4f0 admin admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:55:21.350635 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e3c582a-e83d-42b0-816a-d1d3a8531143 admin admin] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:55:21.358313 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 876/1745] 10.222.0.20 () {60 vars in 1144 bytes} [Mon Aug 1 16:55:21 2022] POST /identity/v3/users => generated 302 bytes in 253 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 16:55:21.368560 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e3c582a-e83d-42b0-816a-d1d3a8531143 admin admin] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:55:21.384637 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a51ac01-7469-4158-945b-5a81ea48b8de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:55:21.384637 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a51ac01-7469-4158-945b-5a81ea48b8de admin admin] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:55:21.384637 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a51ac01-7469-4158-945b-5a81ea48b8de admin admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:55:21.384637 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a51ac01-7469-4158-945b-5a81ea48b8de admin admin] PATH_INFO: `/v3/roles` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:55:21.385530 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a51ac01-7469-4158-945b-5a81ea48b8de admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:55:21.385530 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a51ac01-7469-4158-945b-5a81ea48b8de admin admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:55:21.423495 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77ee59b1-3bd1-48b0-8341-98b088a7e4f0 admin admin] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:55:21.430330 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 870/1746] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Aug 1 16:55:21 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 16:55:21.437762 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77ee59b1-3bd1-48b0-8341-98b088a7e4f0 admin admin] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:55:21.444522 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a3efd0-edc6-4ad0-8445-2400c820f929 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:55:21.445346 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a3efd0-edc6-4ad0-8445-2400c820f929 admin admin] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:55:21.447149 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a51ac01-7469-4158-945b-5a81ea48b8de admin admin] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:55:21.449803 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.fernet_utils [None req-9a51ac01-7469-4158-945b-5a81ea48b8de admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87057) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:55:21.459553 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a3efd0-edc6-4ad0-8445-2400c820f929 admin admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:55:21.460087 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a3efd0-edc6-4ad0-8445-2400c820f929 admin admin] PATH_INFO: `/v3/roles` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:55:21.461479 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19a3efd0-edc6-4ad0-8445-2400c820f929 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:55:21.467220 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19a3efd0-edc6-4ad0-8445-2400c820f929 admin admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:55:21.478694 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a51ac01-7469-4158-945b-5a81ea48b8de admin admin] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:55:21.486522 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d37a89c-afe0-4cc8-bcfa-7e009f48fde6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:55:21.487501 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d37a89c-afe0-4cc8-bcfa-7e009f48fde6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:55:21.488076 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d37a89c-afe0-4cc8-bcfa-7e009f48fde6 admin admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:55:21.490278 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d37a89c-afe0-4cc8-bcfa-7e009f48fde6 admin admin] PATH_INFO: `/v3/projects/2c036dda19f64bfca49f657f667ac8f3` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:55:21.490278 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d37a89c-afe0-4cc8-bcfa-7e009f48fde6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2c036dda19f64bfca49f657f667ac8f3)` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:55:21.491783 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d37a89c-afe0-4cc8-bcfa-7e009f48fde6 admin admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:55:21.533674 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 877/1747] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Aug 1 16:55:21 2022] GET /identity/v3/roles => generated 1420 bytes in 111 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 16:55:21.539492 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a3efd0-edc6-4ad0-8445-2400c820f929 admin admin] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:55:21.559368 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a3efd0-edc6-4ad0-8445-2400c820f929 admin admin] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:55:21.564987 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d45ae6f-efaf-47b1-8399-618307bc56b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:55:21.566485 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d45ae6f-efaf-47b1-8399-618307bc56b2 admin admin] REQUEST_METHOD: `PUT` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:55:21.567101 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d45ae6f-efaf-47b1-8399-618307bc56b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:55:21.568279 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d45ae6f-efaf-47b1-8399-618307bc56b2 admin admin] PATH_INFO: `/v3/projects/be237170c5384dcb8cb74b603fddca4b/users/30551e23b80642a3b78a082fb23b5cf4/roles/0999d3da63a545f18ff3252afdc90518` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:55:21.621983 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d45ae6f-efaf-47b1-8399-618307bc56b2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=be237170c5384dcb8cb74b603fddca4b, user_id=30551e23b80642a3b78a082fb23b5cf4, role_id=0999d3da63a545f18ff3252afdc90518)` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:55:21.629830 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d45ae6f-efaf-47b1-8399-618307bc56b2 admin admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:55:21.777886 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 878/1748] 10.222.0.20 () {60 vars in 1483 bytes} [Mon Aug 1 16:55:21 2022] PUT /identity/v3/projects/be237170c5384dcb8cb74b603fddca4b/users/30551e23b80642a3b78a082fb23b5cf4/roles/0999d3da63a545f18ff3252afdc90518 => generated 0 bytes in 223 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 16:55:21.787368 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b229134-9b46-4381-afc5-158707409c39 None None] REQUEST_METHOD: `POST` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:55:21.788910 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b229134-9b46-4381-afc5-158707409c39 None None] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:55:21.789598 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b229134-9b46-4381-afc5-158707409c39 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:55:21.794298 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.notifications [None req-2d37a89c-afe0-4cc8-bcfa-7e009f48fde6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2c036dda19f64bfca49f657f667ac8f3', 'request_id': 'req-2d37a89c-afe0-4cc8-bcfa-7e009f48fde6'} {{(pid=87057) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:55:21.823929 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.notifications [None req-2d37a89c-afe0-4cc8-bcfa-7e009f48fde6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2c036dda19f64bfca49f657f667ac8f3', 'request_id': 'req-2d37a89c-afe0-4cc8-bcfa-7e009f48fde6'} {{(pid=87057) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:55:21.843018 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.notifications [None req-2d37a89c-afe0-4cc8-bcfa-7e009f48fde6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2c036dda19f64bfca49f657f667ac8f3', 'request_id': 'req-2d37a89c-afe0-4cc8-bcfa-7e009f48fde6'} {{(pid=87057) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:55:21.862982 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.auth.core [None req-6b229134-9b46-4381-afc5-158707409c39 None None] MFA Rules not processed for user `30551e23b80642a3b78a082fb23b5cf4`. Rule list: `[]` (Enabled: `True`). {{(pid=87058) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:55:21.864245 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.fernet_utils [None req-6b229134-9b46-4381-afc5-158707409c39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:55:21.953778 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.notifications [None req-2d37a89c-afe0-4cc8-bcfa-7e009f48fde6 admin admin] The token cache is being invalidate because project 2c036dda19f64bfca49f657f667ac8f3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87057) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 16:55:21.953778 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.notifications [None req-2d37a89c-afe0-4cc8-bcfa-7e009f48fde6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87057) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:55:21.958090 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.notifications [None req-2d37a89c-afe0-4cc8-bcfa-7e009f48fde6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87057) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:55:21.958090 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 871/1749] 10.222.0.20 () {58 vars in 1233 bytes} [Mon Aug 1 16:55:21 2022] DELETE /identity/v3/projects/2c036dda19f64bfca49f657f667ac8f3 => generated 0 bytes in 510 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 16:55:21.960486 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d37a89c-afe0-4cc8-bcfa-7e009f48fde6 admin admin] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:55:21.966901 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.fernet_utils [None req-2d37a89c-afe0-4cc8-bcfa-7e009f48fde6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87057) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:55:22.085470 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 879/1750] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 1 16:55:21 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 299 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 16:55:22.090850 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b229134-9b46-4381-afc5-158707409c39 None None] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:55:22.094947 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.fernet_utils [None req-6b229134-9b46-4381-afc5-158707409c39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:55:22.130801 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d37a89c-afe0-4cc8-bcfa-7e009f48fde6 admin admin] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:55:22.266174 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72be416f-9407-418d-a936-84b4f35cb305 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:55:22.267137 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72be416f-9407-418d-a936-84b4f35cb305 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:55:22.267367 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72be416f-9407-418d-a936-84b4f35cb305 admin admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:55:22.270395 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72be416f-9407-418d-a936-84b4f35cb305 admin admin] PATH_INFO: `/v3/users/00aed21d40604d6088a5e0cdee4ef18f` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:55:22.270395 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72be416f-9407-418d-a936-84b4f35cb305 admin admin] RBAC: Authorizing `identity:delete_user(user_id=00aed21d40604d6088a5e0cdee4ef18f)` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:55:22.272353 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72be416f-9407-418d-a936-84b4f35cb305 admin admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:55:22.289123 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b229134-9b46-4381-afc5-158707409c39 None None] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:55:22.295781 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27651293-f2b3-406c-809a-e3563d73ec26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:55:22.296861 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27651293-f2b3-406c-809a-e3563d73ec26 admin admin] REQUEST_METHOD: `POST` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:55:22.297335 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27651293-f2b3-406c-809a-e3563d73ec26 admin admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:55:22.297622 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27651293-f2b3-406c-809a-e3563d73ec26 admin admin] PATH_INFO: `/v3/projects` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:55:22.298494 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27651293-f2b3-406c-809a-e3563d73ec26 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:55:22.301544 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27651293-f2b3-406c-809a-e3563d73ec26 admin admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:55:22.442435 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG dbcounter [-] [87057] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=87057) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:55:22.457709 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 880/1751] 10.222.0.20 () {60 vars in 1153 bytes} [Mon Aug 1 16:55:22 2022] POST /identity/v3/projects => generated 403 bytes in 367 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 16:55:22.462094 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e70dcfb-b111-4e3d-af97-3855c3c6a783 None None] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:55:22.463754 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e70dcfb-b111-4e3d-af97-3855c3c6a783 None None] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:55:22.464548 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e70dcfb-b111-4e3d-af97-3855c3c6a783 None None] PATH_INFO: `/` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:55:22.469362 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 881/1752] 10.222.0.20 () {54 vars in 921 bytes} [Mon Aug 1 16:55:22 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 16:55:22.472235 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e70dcfb-b111-4e3d-af97-3855c3c6a783 None None] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:55:22.481183 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.fernet_utils [None req-1e70dcfb-b111-4e3d-af97-3855c3c6a783 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:55:22.524234 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.notifications [None req-72be416f-9407-418d-a936-84b4f35cb305 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '00aed21d40604d6088a5e0cdee4ef18f'} {{(pid=87057) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:55:22.525032 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.notifications [None req-72be416f-9407-418d-a936-84b4f35cb305 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '00aed21d40604d6088a5e0cdee4ef18f'} {{(pid=87057) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:55:22.529924 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG dbcounter [-] [87058] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=87058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:55:22.601953 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e70dcfb-b111-4e3d-af97-3855c3c6a783 None None] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:55:22.614207 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.notifications [None req-72be416f-9407-418d-a936-84b4f35cb305 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '00aed21d40604d6088a5e0cdee4ef18f'} {{(pid=87057) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:55:22.614850 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.notifications [None req-72be416f-9407-418d-a936-84b4f35cb305 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '00aed21d40604d6088a5e0cdee4ef18f'} {{(pid=87057) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:55:22.644066 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.notifications [None req-72be416f-9407-418d-a936-84b4f35cb305 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '00aed21d40604d6088a5e0cdee4ef18f'} {{(pid=87057) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:55:22.645394 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97d8ce5d-1654-421c-ad53-fb8e05eaa1a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:55:22.646112 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d8ce5d-1654-421c-ad53-fb8e05eaa1a8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:55:22.646366 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d8ce5d-1654-421c-ad53-fb8e05eaa1a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:55:22.646647 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d8ce5d-1654-421c-ad53-fb8e05eaa1a8 admin admin] PATH_INFO: `/v3/projects/06a3563c812c48a09f5645453ff535b0` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:55:22.648672 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97d8ce5d-1654-421c-ad53-fb8e05eaa1a8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=06a3563c812c48a09f5645453ff535b0)` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:55:22.654067 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97d8ce5d-1654-421c-ad53-fb8e05eaa1a8 admin admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:55:22.670444 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 872/1753] 10.222.0.20 () {58 vars in 1224 bytes} [Mon Aug 1 16:55:21 2022] DELETE /identity/v3/users/00aed21d40604d6088a5e0cdee4ef18f => generated 0 bytes in 710 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 16:55:22.671554 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72be416f-9407-418d-a936-84b4f35cb305 admin admin] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:55:22.673196 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.fernet_utils [None req-72be416f-9407-418d-a936-84b4f35cb305 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87057) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:55:22.768367 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72be416f-9407-418d-a936-84b4f35cb305 admin admin] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:55:22.785226 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.notifications [None req-97d8ce5d-1654-421c-ad53-fb8e05eaa1a8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '06a3563c812c48a09f5645453ff535b0', 'request_id': 'req-97d8ce5d-1654-421c-ad53-fb8e05eaa1a8'} {{(pid=87058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:55:22.796988 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.notifications [None req-97d8ce5d-1654-421c-ad53-fb8e05eaa1a8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '06a3563c812c48a09f5645453ff535b0', 'request_id': 'req-97d8ce5d-1654-421c-ad53-fb8e05eaa1a8'} {{(pid=87058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:55:22.810316 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.notifications [None req-97d8ce5d-1654-421c-ad53-fb8e05eaa1a8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '06a3563c812c48a09f5645453ff535b0', 'request_id': 'req-97d8ce5d-1654-421c-ad53-fb8e05eaa1a8'} {{(pid=87058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:55:22.869064 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3834c433-f954-4aa8-b3ab-50670ab23a95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:55:22.873066 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3834c433-f954-4aa8-b3ab-50670ab23a95 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:55:22.873066 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3834c433-f954-4aa8-b3ab-50670ab23a95 admin admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:55:22.873277 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3834c433-f954-4aa8-b3ab-50670ab23a95 admin admin] PATH_INFO: `/v3/users/4df0b959497d4d51a308c1db5626be6d` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:55:22.876266 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3834c433-f954-4aa8-b3ab-50670ab23a95 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4df0b959497d4d51a308c1db5626be6d)` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:55:22.880053 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3834c433-f954-4aa8-b3ab-50670ab23a95 admin admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:55:22.919426 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.notifications [None req-97d8ce5d-1654-421c-ad53-fb8e05eaa1a8 admin admin] The token cache is being invalidate because project 06a3563c812c48a09f5645453ff535b0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87058) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 16:55:22.919802 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.notifications [None req-97d8ce5d-1654-421c-ad53-fb8e05eaa1a8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:55:22.920585 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.notifications [None req-97d8ce5d-1654-421c-ad53-fb8e05eaa1a8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:55:22.925289 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 882/1754] 10.222.0.20 () {58 vars in 1233 bytes} [Mon Aug 1 16:55:22 2022] DELETE /identity/v3/projects/06a3563c812c48a09f5645453ff535b0 => generated 0 bytes in 452 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 16:55:22.925289 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97d8ce5d-1654-421c-ad53-fb8e05eaa1a8 admin admin] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:55:22.927322 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.fernet_utils [None req-97d8ce5d-1654-421c-ad53-fb8e05eaa1a8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:55:22.962543 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97d8ce5d-1654-421c-ad53-fb8e05eaa1a8 admin admin] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:55:22.970331 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ad845b-e352-4e4e-8c8f-f88d632d6067 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:55:22.971075 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ad845b-e352-4e4e-8c8f-f88d632d6067 admin admin] REQUEST_METHOD: `POST` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:55:22.971426 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ad845b-e352-4e4e-8c8f-f88d632d6067 admin admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:55:22.971776 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ad845b-e352-4e4e-8c8f-f88d632d6067 admin admin] PATH_INFO: `/v3/users` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:55:22.972589 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81ad845b-e352-4e4e-8c8f-f88d632d6067 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:55:22.979182 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81ad845b-e352-4e4e-8c8f-f88d632d6067 admin admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:55:23.087403 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.notifications [None req-3834c433-f954-4aa8-b3ab-50670ab23a95 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4df0b959497d4d51a308c1db5626be6d'} {{(pid=87057) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:55:23.092789 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 883/1755] 10.222.0.20 () {60 vars in 1144 bytes} [Mon Aug 1 16:55:22 2022] POST /identity/v3/users => generated 308 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 16:55:23.096799 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b524fa2-09d3-4ca9-a2bd-d39c8f9d02d6 None None] REQUEST_METHOD: `POST` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:55:23.096931 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.notifications [None req-3834c433-f954-4aa8-b3ab-50670ab23a95 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4df0b959497d4d51a308c1db5626be6d'} {{(pid=87057) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:55:23.097027 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b524fa2-09d3-4ca9-a2bd-d39c8f9d02d6 None None] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:55:23.097027 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b524fa2-09d3-4ca9-a2bd-d39c8f9d02d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:55:23.131672 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.auth.core [None req-4b524fa2-09d3-4ca9-a2bd-d39c8f9d02d6 None None] MFA Rules not processed for user `6533106d03eb4ca185a8839336d2aff9`. Rule list: `[]` (Enabled: `True`). {{(pid=87058) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:55:23.132919 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.fernet_utils [None req-4b524fa2-09d3-4ca9-a2bd-d39c8f9d02d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:55:23.190983 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.notifications [None req-3834c433-f954-4aa8-b3ab-50670ab23a95 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4df0b959497d4d51a308c1db5626be6d'} {{(pid=87057) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:55:23.192564 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.notifications [None req-3834c433-f954-4aa8-b3ab-50670ab23a95 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4df0b959497d4d51a308c1db5626be6d'} {{(pid=87057) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:55:23.202382 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.notifications [None req-3834c433-f954-4aa8-b3ab-50670ab23a95 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4df0b959497d4d51a308c1db5626be6d'} {{(pid=87057) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:55:23.261029 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG dbcounter [-] [87058] Writing DB stats keystone:SELECT=88,keystone:INSERT=5,keystone:DELETE=7 {{(pid=87058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:55:23.268344 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 873/1756] 10.222.0.20 () {58 vars in 1224 bytes} [Mon Aug 1 16:55:22 2022] DELETE /identity/v3/users/4df0b959497d4d51a308c1db5626be6d => generated 0 bytes in 597 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 16:55:23.270124 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3834c433-f954-4aa8-b3ab-50670ab23a95 admin admin] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:55:23.275737 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.fernet_utils [None req-3834c433-f954-4aa8-b3ab-50670ab23a95 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87057) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:55:23.288649 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 884/1757] 10.222.0.20 () {58 vars in 1018 bytes} [Mon Aug 1 16:55:23 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:55:23.294362 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b524fa2-09d3-4ca9-a2bd-d39c8f9d02d6 None None] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:55:23.294362 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.fernet_utils [None req-4b524fa2-09d3-4ca9-a2bd-d39c8f9d02d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:55:23.313869 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b524fa2-09d3-4ca9-a2bd-d39c8f9d02d6 None None] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:55:23.421409 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3834c433-f954-4aa8-b3ab-50670ab23a95 admin admin] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:55:23.438336 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b143585-99eb-4c26-9986-c398c21f1155 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:55:23.439157 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b143585-99eb-4c26-9986-c398c21f1155 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:55:23.439463 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b143585-99eb-4c26-9986-c398c21f1155 admin admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:55:23.441103 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b143585-99eb-4c26-9986-c398c21f1155 admin admin] PATH_INFO: `/v3/users/55ee94efc7dc4cffbf4ae58bdca723a3` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:55:23.441274 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6556bf0-3e39-4bc8-be6a-ad3072346a5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:55:23.441886 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6556bf0-3e39-4bc8-be6a-ad3072346a5f admin admin] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:55:23.442344 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6556bf0-3e39-4bc8-be6a-ad3072346a5f admin admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:55:23.445348 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6556bf0-3e39-4bc8-be6a-ad3072346a5f admin admin] PATH_INFO: `/v3/roles` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:55:23.445348 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6556bf0-3e39-4bc8-be6a-ad3072346a5f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:55:23.445541 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b143585-99eb-4c26-9986-c398c21f1155 admin admin] RBAC: Authorizing `identity:delete_user(user_id=55ee94efc7dc4cffbf4ae58bdca723a3)` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:55:23.445639 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6556bf0-3e39-4bc8-be6a-ad3072346a5f admin admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:55:23.447156 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b143585-99eb-4c26-9986-c398c21f1155 admin admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:55:23.460736 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 885/1758] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Aug 1 16:55:23 2022] GET /identity/v3/roles => generated 1420 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 16:55:23.462271 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4664284-7c07-4df3-b4d9-03f1831b6a92 None None] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:55:23.463984 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4664284-7c07-4df3-b4d9-03f1831b6a92 None None] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:55:23.463984 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4664284-7c07-4df3-b4d9-03f1831b6a92 None None] PATH_INFO: `/` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:55:23.464884 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 886/1759] 10.222.0.20 () {54 vars in 921 bytes} [Mon Aug 1 16:55:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 16:55:23.466250 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1494c5-d452-420e-9b5f-86d062b68a90 None None] REQUEST_METHOD: `POST` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:55:23.466643 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1494c5-d452-420e-9b5f-86d062b68a90 None None] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:55:23.467018 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1494c5-d452-420e-9b5f-86d062b68a90 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:55:23.488796 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.auth.core [None req-ed1494c5-d452-420e-9b5f-86d062b68a90 None None] MFA Rules not processed for user `1007f8e379b74e169b98fb7c3c0ba5b4`. Rule list: `[]` (Enabled: `True`). {{(pid=87058) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:55:23.490206 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.fernet_utils [None req-ed1494c5-d452-420e-9b5f-86d062b68a90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:55:23.503051 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG dbcounter [-] [87057] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87057) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:55:23.508877 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 887/1760] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 1 16:55:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:55:23.508877 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed1494c5-d452-420e-9b5f-86d062b68a90 None None] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:55:23.510769 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.fernet_utils [None req-ed1494c5-d452-420e-9b5f-86d062b68a90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:55:23.613233 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed1494c5-d452-420e-9b5f-86d062b68a90 None None] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:55:23.625073 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68622ff7-bd03-4efc-921a-4af37c19b505 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6533106d03eb4ca185a8839336d2aff9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b992d6a65df4bf9b27c13192d57c5f0', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:55:23.625966 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68622ff7-bd03-4efc-921a-4af37c19b505 service nova] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:55:23.628130 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68622ff7-bd03-4efc-921a-4af37c19b505 service nova] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:55:23.628130 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68622ff7-bd03-4efc-921a-4af37c19b505 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:55:23.637895 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68622ff7-bd03-4efc-921a-4af37c19b505 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:55:23.639800 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68622ff7-bd03-4efc-921a-4af37c19b505 service nova] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:55:23.665027 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 888/1761] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 1 16:55:23 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 157 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 16:55:23.665741 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68622ff7-bd03-4efc-921a-4af37c19b505 service nova] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:55:23.697195 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68622ff7-bd03-4efc-921a-4af37c19b505 service nova] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:55:23.702253 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.notifications [None req-2b143585-99eb-4c26-9986-c398c21f1155 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '55ee94efc7dc4cffbf4ae58bdca723a3'} {{(pid=87057) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:55:23.703070 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.notifications [None req-2b143585-99eb-4c26-9986-c398c21f1155 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '55ee94efc7dc4cffbf4ae58bdca723a3'} {{(pid=87057) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:55:23.703219 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-addb4927-c5d8-4574-89ad-af4476958c9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:55:23.703825 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-addb4927-c5d8-4574-89ad-af4476958c9f admin admin] REQUEST_METHOD: `PUT` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:55:23.704103 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-addb4927-c5d8-4574-89ad-af4476958c9f admin admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:55:23.704375 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-addb4927-c5d8-4574-89ad-af4476958c9f admin admin] PATH_INFO: `/v3/projects/2a8c40feb1fb49edb84665134d4374e0/users/9225c88654b84473afa5296594705c31/roles/440bb943dd5b46fea038b903b6eaaf81` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:55:23.726076 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-addb4927-c5d8-4574-89ad-af4476958c9f admin admin] RBAC: Authorizing `identity:create_grant(project_id=2a8c40feb1fb49edb84665134d4374e0, user_id=9225c88654b84473afa5296594705c31, role_id=440bb943dd5b46fea038b903b6eaaf81)` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:55:23.729153 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-addb4927-c5d8-4574-89ad-af4476958c9f admin admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:55:23.774463 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.notifications [None req-2b143585-99eb-4c26-9986-c398c21f1155 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '55ee94efc7dc4cffbf4ae58bdca723a3'} {{(pid=87057) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:55:23.775362 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.notifications [None req-2b143585-99eb-4c26-9986-c398c21f1155 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '55ee94efc7dc4cffbf4ae58bdca723a3'} {{(pid=87057) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:55:23.785074 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.notifications [None req-2b143585-99eb-4c26-9986-c398c21f1155 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '55ee94efc7dc4cffbf4ae58bdca723a3'} {{(pid=87057) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:55:23.808479 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 889/1762] 10.222.0.20 () {60 vars in 1483 bytes} [Mon Aug 1 16:55:23 2022] PUT /identity/v3/projects/2a8c40feb1fb49edb84665134d4374e0/users/9225c88654b84473afa5296594705c31/roles/440bb943dd5b46fea038b903b6eaaf81 => generated 0 bytes in 140 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 16:55:23.808479 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80bdfacf-5323-429d-b4c9-64e4da57e9f0 None None] REQUEST_METHOD: `POST` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:55:23.808479 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80bdfacf-5323-429d-b4c9-64e4da57e9f0 None None] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:55:23.808479 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80bdfacf-5323-429d-b4c9-64e4da57e9f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:55:23.829036 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 874/1763] 10.222.0.20 () {58 vars in 1224 bytes} [Mon Aug 1 16:55:23 2022] DELETE /identity/v3/users/55ee94efc7dc4cffbf4ae58bdca723a3 => generated 0 bytes in 560 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 16:55:23.829979 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b143585-99eb-4c26-9986-c398c21f1155 admin admin] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:55:23.832476 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.fernet_utils [None req-2b143585-99eb-4c26-9986-c398c21f1155 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87057) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:55:23.846070 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.auth.core [None req-80bdfacf-5323-429d-b4c9-64e4da57e9f0 None None] MFA Rules not processed for user `6533106d03eb4ca185a8839336d2aff9`. Rule list: `[]` (Enabled: `True`). {{(pid=87058) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:55:23.847713 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.fernet_utils [None req-80bdfacf-5323-429d-b4c9-64e4da57e9f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:55:23.939375 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b143585-99eb-4c26-9986-c398c21f1155 admin admin] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:55:23.944697 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b99e4a04-dae6-4de2-8d5e-35b973c3c4ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:55:23.945355 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99e4a04-dae6-4de2-8d5e-35b973c3c4ab admin admin] REQUEST_METHOD: `DELETE` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:55:23.945636 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99e4a04-dae6-4de2-8d5e-35b973c3c4ab admin admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:55:23.945909 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99e4a04-dae6-4de2-8d5e-35b973c3c4ab admin admin] PATH_INFO: `/v3/projects/2d5994265bf54c1ab513dff4848f63ef` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:55:23.947143 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b99e4a04-dae6-4de2-8d5e-35b973c3c4ab admin admin] RBAC: Authorizing `identity:delete_project(project_id=2d5994265bf54c1ab513dff4848f63ef)` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:55:23.949789 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b99e4a04-dae6-4de2-8d5e-35b973c3c4ab admin admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:55:23.961130 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 890/1764] 10.222.0.20 () {58 vars in 1018 bytes} [Mon Aug 1 16:55:23 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:55:23.961130 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80bdfacf-5323-429d-b4c9-64e4da57e9f0 None None] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:55:23.961725 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.fernet_utils [None req-80bdfacf-5323-429d-b4c9-64e4da57e9f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:55:24.022904 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80bdfacf-5323-429d-b4c9-64e4da57e9f0 None None] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:55:24.027554 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8235f9d8-79df-4a6e-b38b-9547f241d0a0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d9bd8d649154c979be3744bfd016f95', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b992d6a65df4bf9b27c13192d57c5f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:55:24.028177 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8235f9d8-79df-4a6e-b38b-9547f241d0a0 service neutron] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:55:24.028457 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8235f9d8-79df-4a6e-b38b-9547f241d0a0 service neutron] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:55:24.028750 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8235f9d8-79df-4a6e-b38b-9547f241d0a0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:55:24.030746 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.fernet_utils [None req-8235f9d8-79df-4a6e-b38b-9547f241d0a0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:55:24.042367 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8235f9d8-79df-4a6e-b38b-9547f241d0a0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:55:24.043813 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8235f9d8-79df-4a6e-b38b-9547f241d0a0 service neutron] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:55:24.053346 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 891/1765] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 1 16:55:23 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 16:55:24.054235 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8235f9d8-79df-4a6e-b38b-9547f241d0a0 service neutron] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:55:24.055538 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.fernet_utils [None req-8235f9d8-79df-4a6e-b38b-9547f241d0a0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:55:24.066877 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8235f9d8-79df-4a6e-b38b-9547f241d0a0 service neutron] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:55:24.078247 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a8bee51-4005-4ace-b43e-0ec507ad0e3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:55:24.078247 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8bee51-4005-4ace-b43e-0ec507ad0e3c admin admin] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:55:24.078247 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8bee51-4005-4ace-b43e-0ec507ad0e3c admin admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:55:24.078247 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8bee51-4005-4ace-b43e-0ec507ad0e3c admin admin] PATH_INFO: `/v3/roles` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:55:24.078727 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a8bee51-4005-4ace-b43e-0ec507ad0e3c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:55:24.078727 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a8bee51-4005-4ace-b43e-0ec507ad0e3c admin admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:55:24.085437 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 892/1766] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Aug 1 16:55:24 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 16:55:24.086510 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a8bee51-4005-4ace-b43e-0ec507ad0e3c admin admin] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:55:24.090805 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.fernet_utils [None req-9a8bee51-4005-4ace-b43e-0ec507ad0e3c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:55:24.109299 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a8bee51-4005-4ace-b43e-0ec507ad0e3c admin admin] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:55:24.109818 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.notifications [None req-b99e4a04-dae6-4de2-8d5e-35b973c3c4ab admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2d5994265bf54c1ab513dff4848f63ef', 'request_id': 'req-b99e4a04-dae6-4de2-8d5e-35b973c3c4ab'} {{(pid=87057) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:55:24.116912 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.notifications [None req-b99e4a04-dae6-4de2-8d5e-35b973c3c4ab admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2d5994265bf54c1ab513dff4848f63ef', 'request_id': 'req-b99e4a04-dae6-4de2-8d5e-35b973c3c4ab'} {{(pid=87057) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:55:24.124544 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.notifications [None req-b99e4a04-dae6-4de2-8d5e-35b973c3c4ab admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2d5994265bf54c1ab513dff4848f63ef', 'request_id': 'req-b99e4a04-dae6-4de2-8d5e-35b973c3c4ab'} {{(pid=87057) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:55:24.202489 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.notifications [None req-b99e4a04-dae6-4de2-8d5e-35b973c3c4ab admin admin] The token cache is being invalidate because project 2d5994265bf54c1ab513dff4848f63ef was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87057) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 16:55:24.202823 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.notifications [None req-b99e4a04-dae6-4de2-8d5e-35b973c3c4ab admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87057) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:55:24.203316 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45d8d190-be7e-45b8-a8cb-4e6b9340e209 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6533106d03eb4ca185a8839336d2aff9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b992d6a65df4bf9b27c13192d57c5f0', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:55:24.203532 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.notifications [None req-b99e4a04-dae6-4de2-8d5e-35b973c3c4ab admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87057) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:55:24.203968 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d8d190-be7e-45b8-a8cb-4e6b9340e209 service nova] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:55:24.204427 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d8d190-be7e-45b8-a8cb-4e6b9340e209 service nova] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:55:24.204710 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d8d190-be7e-45b8-a8cb-4e6b9340e209 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:55:24.206258 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 875/1767] 10.222.0.20 () {58 vars in 1233 bytes} [Mon Aug 1 16:55:23 2022] DELETE /identity/v3/projects/2d5994265bf54c1ab513dff4848f63ef => generated 0 bytes in 376 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 16:55:24.206666 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.fernet_utils [None req-45d8d190-be7e-45b8-a8cb-4e6b9340e209 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:55:24.207748 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b99e4a04-dae6-4de2-8d5e-35b973c3c4ab admin admin] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:55:24.209360 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.fernet_utils [None req-b99e4a04-dae6-4de2-8d5e-35b973c3c4ab admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87057) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:55:24.222283 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45d8d190-be7e-45b8-a8cb-4e6b9340e209 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:55:24.223957 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45d8d190-be7e-45b8-a8cb-4e6b9340e209 service nova] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:55:24.235506 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 893/1768] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 1 16:55:24 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 150 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 16:55:24.237069 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7cfe8e1-83fe-4e00-9a9a-9899a61b070a None None] REQUEST_METHOD: `POST` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:55:24.237581 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7cfe8e1-83fe-4e00-9a9a-9899a61b070a None None] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:55:24.238112 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7cfe8e1-83fe-4e00-9a9a-9899a61b070a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:55:24.261113 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.auth.core [None req-d7cfe8e1-83fe-4e00-9a9a-9899a61b070a None None] MFA Rules not processed for user `1007f8e379b74e169b98fb7c3c0ba5b4`. Rule list: `[]` (Enabled: `True`). {{(pid=87058) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:55:24.262395 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.fernet_utils [None req-d7cfe8e1-83fe-4e00-9a9a-9899a61b070a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:55:24.288243 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG dbcounter [-] [87058] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:55:24.309764 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b99e4a04-dae6-4de2-8d5e-35b973c3c4ab admin admin] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:55:24.314743 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d401de-3244-4ca0-80d9-ee0b39cb26c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:55:24.315552 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d401de-3244-4ca0-80d9-ee0b39cb26c2 admin admin] REQUEST_METHOD: `PUT` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:55:24.315837 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d401de-3244-4ca0-80d9-ee0b39cb26c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:55:24.316117 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d401de-3244-4ca0-80d9-ee0b39cb26c2 admin admin] PATH_INFO: `/v3/projects/2a8c40feb1fb49edb84665134d4374e0/users/9225c88654b84473afa5296594705c31/roles/440bb943dd5b46fea038b903b6eaaf81` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:55:24.318284 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74d401de-3244-4ca0-80d9-ee0b39cb26c2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2a8c40feb1fb49edb84665134d4374e0, user_id=9225c88654b84473afa5296594705c31, role_id=440bb943dd5b46fea038b903b6eaaf81)` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:55:24.321100 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74d401de-3244-4ca0-80d9-ee0b39cb26c2 admin admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:55:24.346564 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 876/1769] 10.222.0.20 () {60 vars in 1483 bytes} [Mon Aug 1 16:55:24 2022] PUT /identity/v3/projects/2a8c40feb1fb49edb84665134d4374e0/users/9225c88654b84473afa5296594705c31/roles/440bb943dd5b46fea038b903b6eaaf81 => generated 0 bytes in 130 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 16:55:24.349467 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70999cdc-b05e-4094-ad5d-db5480e50b60 None None] REQUEST_METHOD: `POST` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:55:24.349826 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70999cdc-b05e-4094-ad5d-db5480e50b60 None None] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:55:24.350224 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70999cdc-b05e-4094-ad5d-db5480e50b60 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:55:24.352420 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 894/1770] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 1 16:55:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:55:24.366543 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41b5e86-72c1-4def-b16d-1a879d80c4d7 None None] REQUEST_METHOD: `POST` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:55:24.367037 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41b5e86-72c1-4def-b16d-1a879d80c4d7 None None] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:55:24.367386 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41b5e86-72c1-4def-b16d-1a879d80c4d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:55:24.395023 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.auth.core [None req-c41b5e86-72c1-4def-b16d-1a879d80c4d7 None None] MFA Rules not processed for user `1007f8e379b74e169b98fb7c3c0ba5b4`. Rule list: `[]` (Enabled: `True`). {{(pid=87058) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:55:24.395023 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.fernet_utils [None req-c41b5e86-72c1-4def-b16d-1a879d80c4d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:55:24.402884 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.auth.core [None req-70999cdc-b05e-4094-ad5d-db5480e50b60 None None] MFA Rules not processed for user `9225c88654b84473afa5296594705c31`. Rule list: `[]` (Enabled: `True`). {{(pid=87057) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:55:24.405111 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.fernet_utils [None req-70999cdc-b05e-4094-ad5d-db5480e50b60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87057) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:55:24.437661 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG dbcounter [-] [87057] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=87057) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:55:24.508853 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 877/1771] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 1 16:55:24 2022] POST /identity/v3/auth/tokens => generated 3337 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:55:24.513660 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70999cdc-b05e-4094-ad5d-db5480e50b60 None None] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:55:24.513660 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.fernet_utils [None req-70999cdc-b05e-4094-ad5d-db5480e50b60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87057) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:55:24.515195 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 895/1772] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 1 16:55:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 16:55:24.519200 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c41b5e86-72c1-4def-b16d-1a879d80c4d7 None None] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:55:24.528638 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70999cdc-b05e-4094-ad5d-db5480e50b60 None None] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:55:24.532988 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c41b5e86-72c1-4def-b16d-1a879d80c4d7 None None] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:55:24.533661 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26a86665-9b32-4294-9c0c-5d75e1cc604a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:55:24.534425 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a86665-9b32-4294-9c0c-5d75e1cc604a admin admin] REQUEST_METHOD: `DELETE` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:55:24.534780 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a86665-9b32-4294-9c0c-5d75e1cc604a admin admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:55:24.535130 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a86665-9b32-4294-9c0c-5d75e1cc604a admin admin] PATH_INFO: `/v3/projects/5ec6de45cc04415f8ef9b61536c18844` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:55:24.549707 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26a86665-9b32-4294-9c0c-5d75e1cc604a admin admin] RBAC: Authorizing `identity:delete_project(project_id=5ec6de45cc04415f8ef9b61536c18844)` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:55:24.549809 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e92b3ef0-18ba-4c7f-a8d5-df609107666f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:55:24.549809 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92b3ef0-18ba-4c7f-a8d5-df609107666f admin admin] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:55:24.550011 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26a86665-9b32-4294-9c0c-5d75e1cc604a admin admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:55:24.550136 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92b3ef0-18ba-4c7f-a8d5-df609107666f admin admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:55:24.550136 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92b3ef0-18ba-4c7f-a8d5-df609107666f admin admin] PATH_INFO: `/v3/domains` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:55:24.550136 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e92b3ef0-18ba-4c7f-a8d5-df609107666f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87058) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:55:24.550136 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e92b3ef0-18ba-4c7f-a8d5-df609107666f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:55:24.550136 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e92b3ef0-18ba-4c7f-a8d5-df609107666f admin admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:55:24.554443 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 896/1773] 10.222.0.20 () {58 vars in 1153 bytes} [Mon Aug 1 16:55:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 16:55:24.555303 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e92b3ef0-18ba-4c7f-a8d5-df609107666f admin admin] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:55:24.557540 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.fernet_utils [None req-e92b3ef0-18ba-4c7f-a8d5-df609107666f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:55:24.618979 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e92b3ef0-18ba-4c7f-a8d5-df609107666f admin admin] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:55:24.623393 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba5d17c6-1649-4707-807c-609387c587a8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25970b52ca15448fa1abb1f974d64038', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b992d6a65df4bf9b27c13192d57c5f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:55:24.624041 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5d17c6-1649-4707-807c-609387c587a8 service cinder] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:55:24.624322 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5d17c6-1649-4707-807c-609387c587a8 service cinder] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:55:24.624600 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5d17c6-1649-4707-807c-609387c587a8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:55:24.633555 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba5d17c6-1649-4707-807c-609387c587a8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:55:24.635759 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba5d17c6-1649-4707-807c-609387c587a8 service cinder] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:55:24.645475 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 897/1774] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 1 16:55:24 2022] GET /identity/v3/auth/tokens => generated 3337 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 16:55:24.650094 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba5d17c6-1649-4707-807c-609387c587a8 service cinder] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:55:24.658952 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.notifications [None req-26a86665-9b32-4294-9c0c-5d75e1cc604a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5ec6de45cc04415f8ef9b61536c18844', 'request_id': 'req-26a86665-9b32-4294-9c0c-5d75e1cc604a'} {{(pid=87057) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:55:24.659980 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba5d17c6-1649-4707-807c-609387c587a8 service cinder] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:55:24.665877 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.notifications [None req-26a86665-9b32-4294-9c0c-5d75e1cc604a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5ec6de45cc04415f8ef9b61536c18844', 'request_id': 'req-26a86665-9b32-4294-9c0c-5d75e1cc604a'} {{(pid=87057) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:55:24.671904 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.notifications [None req-26a86665-9b32-4294-9c0c-5d75e1cc604a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5ec6de45cc04415f8ef9b61536c18844', 'request_id': 'req-26a86665-9b32-4294-9c0c-5d75e1cc604a'} {{(pid=87057) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:55:24.753369 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.notifications [None req-26a86665-9b32-4294-9c0c-5d75e1cc604a admin admin] The token cache is being invalidate because project 5ec6de45cc04415f8ef9b61536c18844 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87057) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 16:55:24.753660 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.notifications [None req-26a86665-9b32-4294-9c0c-5d75e1cc604a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87057) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:55:24.754368 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.notifications [None req-26a86665-9b32-4294-9c0c-5d75e1cc604a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87057) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:55:24.756248 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-552ef065-5220-408e-a2a8-3f48d04d53e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:55:24.756884 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 878/1775] 10.222.0.20 () {58 vars in 1233 bytes} [Mon Aug 1 16:55:24 2022] DELETE /identity/v3/projects/5ec6de45cc04415f8ef9b61536c18844 => generated 0 bytes in 247 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 16:55:24.757089 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552ef065-5220-408e-a2a8-3f48d04d53e1 admin admin] REQUEST_METHOD: `POST` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:55:24.757380 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552ef065-5220-408e-a2a8-3f48d04d53e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:55:24.757707 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552ef065-5220-408e-a2a8-3f48d04d53e1 admin admin] PATH_INFO: `/v3/projects` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:55:24.757964 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26a86665-9b32-4294-9c0c-5d75e1cc604a admin admin] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:55:24.758471 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-552ef065-5220-408e-a2a8-3f48d04d53e1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:55:24.759219 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.fernet_utils [None req-26a86665-9b32-4294-9c0c-5d75e1cc604a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87057) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:55:24.762084 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-552ef065-5220-408e-a2a8-3f48d04d53e1 admin admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:55:24.785899 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26a86665-9b32-4294-9c0c-5d75e1cc604a admin admin] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:55:24.792523 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa01886-e951-400a-a6c6-94e5790ee5b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:55:24.793362 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa01886-e951-400a-a6c6-94e5790ee5b8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:55:24.793787 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa01886-e951-400a-a6c6-94e5790ee5b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:55:24.794100 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa01886-e951-400a-a6c6-94e5790ee5b8 admin admin] PATH_INFO: `/v3/projects/204b0c7d4bf843659ec38b79fa7c1815` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:55:24.795626 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aa01886-e951-400a-a6c6-94e5790ee5b8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=204b0c7d4bf843659ec38b79fa7c1815)` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:55:24.797620 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aa01886-e951-400a-a6c6-94e5790ee5b8 admin admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:55:24.911890 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 898/1776] 10.222.0.20 () {60 vars in 1153 bytes} [Mon Aug 1 16:55:24 2022] POST /identity/v3/projects => generated 371 bytes in 266 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 16:55:24.913282 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aefafc0-d487-483c-ad0a-27f8da1e24a3 None None] REQUEST_METHOD: `POST` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:55:24.913460 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aefafc0-d487-483c-ad0a-27f8da1e24a3 None None] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:55:24.916749 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aefafc0-d487-483c-ad0a-27f8da1e24a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:55:24.966004 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.auth.core [None req-3aefafc0-d487-483c-ad0a-27f8da1e24a3 None None] MFA Rules not processed for user `1007f8e379b74e169b98fb7c3c0ba5b4`. Rule list: `[]` (Enabled: `True`). {{(pid=87058) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:55:24.971806 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.fernet_utils [None req-3aefafc0-d487-483c-ad0a-27f8da1e24a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:55:24.971896 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.notifications [None req-9aa01886-e951-400a-a6c6-94e5790ee5b8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '204b0c7d4bf843659ec38b79fa7c1815', 'request_id': 'req-9aa01886-e951-400a-a6c6-94e5790ee5b8'} {{(pid=87057) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:55:24.985170 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.notifications [None req-9aa01886-e951-400a-a6c6-94e5790ee5b8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '204b0c7d4bf843659ec38b79fa7c1815', 'request_id': 'req-9aa01886-e951-400a-a6c6-94e5790ee5b8'} {{(pid=87057) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:55:25.021095 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.notifications [None req-9aa01886-e951-400a-a6c6-94e5790ee5b8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '204b0c7d4bf843659ec38b79fa7c1815', 'request_id': 'req-9aa01886-e951-400a-a6c6-94e5790ee5b8'} {{(pid=87057) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:55:25.071778 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG dbcounter [-] [87058] Writing DB stats keystone:SELECT=100 {{(pid=87058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:55:25.078474 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.notifications [None req-9aa01886-e951-400a-a6c6-94e5790ee5b8 admin admin] The token cache is being invalidate because project 204b0c7d4bf843659ec38b79fa7c1815 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87057) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 16:55:25.078969 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.notifications [None req-9aa01886-e951-400a-a6c6-94e5790ee5b8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87057) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:55:25.079769 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.notifications [None req-9aa01886-e951-400a-a6c6-94e5790ee5b8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87057) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:55:25.081926 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 879/1777] 10.222.0.20 () {58 vars in 1233 bytes} [Mon Aug 1 16:55:24 2022] DELETE /identity/v3/projects/204b0c7d4bf843659ec38b79fa7c1815 => generated 0 bytes in 324 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 16:55:25.082597 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa01886-e951-400a-a6c6-94e5790ee5b8 admin admin] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:55:25.084125 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.fernet_utils [None req-9aa01886-e951-400a-a6c6-94e5790ee5b8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87057) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:55:25.087673 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 899/1778] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 1 16:55:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:55:25.091467 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aefafc0-d487-483c-ad0a-27f8da1e24a3 None None] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:55:25.101517 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa01886-e951-400a-a6c6-94e5790ee5b8 admin admin] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:55:25.107085 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aefafc0-d487-483c-ad0a-27f8da1e24a3 None None] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:55:25.108596 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c8c53d1-e8bc-4cfe-a64b-cdcb1773d488 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:55:25.109643 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8c53d1-e8bc-4cfe-a64b-cdcb1773d488 admin admin] REQUEST_METHOD: `POST` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:55:25.109927 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8c53d1-e8bc-4cfe-a64b-cdcb1773d488 admin admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:55:25.110649 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8c53d1-e8bc-4cfe-a64b-cdcb1773d488 admin admin] PATH_INFO: `/v3/users` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:55:25.111340 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c8c53d1-e8bc-4cfe-a64b-cdcb1773d488 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:55:25.113394 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c8c53d1-e8bc-4cfe-a64b-cdcb1773d488 admin admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:55:25.121958 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f1bff0f-2eef-4f09-858e-178795c9d69d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:55:25.123026 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1bff0f-2eef-4f09-858e-178795c9d69d admin admin] REQUEST_METHOD: `DELETE` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:55:25.123369 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1bff0f-2eef-4f09-858e-178795c9d69d admin admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:55:25.123709 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1bff0f-2eef-4f09-858e-178795c9d69d admin admin] PATH_INFO: `/v3/users/9e7dd0b9bb5743f4918fca51d577bcaa` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:55:25.127968 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f1bff0f-2eef-4f09-858e-178795c9d69d admin admin] RBAC: Authorizing `identity:delete_user(user_id=9e7dd0b9bb5743f4918fca51d577bcaa)` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:55:25.130539 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f1bff0f-2eef-4f09-858e-178795c9d69d admin admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:55:25.226326 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 880/1779] 10.222.0.20 () {60 vars in 1144 bytes} [Mon Aug 1 16:55:25 2022] POST /identity/v3/users => generated 286 bytes in 144 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 16:55:25.227586 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d6a0c9-6502-4244-95e7-ce802a8410a8 None None] REQUEST_METHOD: `POST` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:55:25.227980 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d6a0c9-6502-4244-95e7-ce802a8410a8 None None] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:55:25.228143 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d6a0c9-6502-4244-95e7-ce802a8410a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:55:25.251184 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.auth.core [None req-47d6a0c9-6502-4244-95e7-ce802a8410a8 None None] MFA Rules not processed for user `1007f8e379b74e169b98fb7c3c0ba5b4`. Rule list: `[]` (Enabled: `True`). {{(pid=87057) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:55:25.252525 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.fernet_utils [None req-47d6a0c9-6502-4244-95e7-ce802a8410a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87057) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:55:25.261083 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 881/1780] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 1 16:55:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:55:25.261937 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d6a0c9-6502-4244-95e7-ce802a8410a8 None None] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:55:25.272218 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d6a0c9-6502-4244-95e7-ce802a8410a8 None None] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:55:25.277683 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e85c0fe-1cf9-4a14-9d10-1054af7f571a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:55:25.278374 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e85c0fe-1cf9-4a14-9d10-1054af7f571a admin admin] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:55:25.278653 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e85c0fe-1cf9-4a14-9d10-1054af7f571a admin admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:55:25.278923 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e85c0fe-1cf9-4a14-9d10-1054af7f571a admin admin] PATH_INFO: `/v3/roles` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:55:25.279619 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e85c0fe-1cf9-4a14-9d10-1054af7f571a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:55:25.281310 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e85c0fe-1cf9-4a14-9d10-1054af7f571a admin admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:55:25.291793 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 882/1781] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Aug 1 16:55:25 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 16:55:25.292644 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e85c0fe-1cf9-4a14-9d10-1054af7f571a admin admin] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:55:25.302371 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e85c0fe-1cf9-4a14-9d10-1054af7f571a admin admin] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:55:25.307281 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f20d1a-60e5-49b4-973e-d619005a0fc1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:55:25.308114 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f20d1a-60e5-49b4-973e-d619005a0fc1 admin admin] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:55:25.308386 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f20d1a-60e5-49b4-973e-d619005a0fc1 admin admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:55:25.308771 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f20d1a-60e5-49b4-973e-d619005a0fc1 admin admin] PATH_INFO: `/v3/domains` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:55:25.309478 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4f20d1a-60e5-49b4-973e-d619005a0fc1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87057) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:55:25.309793 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4f20d1a-60e5-49b4-973e-d619005a0fc1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:55:25.311444 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4f20d1a-60e5-49b4-973e-d619005a0fc1 admin admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:55:25.325855 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 883/1782] 10.222.0.20 () {58 vars in 1153 bytes} [Mon Aug 1 16:55:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:55:25.326910 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f20d1a-60e5-49b4-973e-d619005a0fc1 admin admin] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:55:25.339586 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f20d1a-60e5-49b4-973e-d619005a0fc1 admin admin] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:55:25.344373 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19fa49e1-5c55-4ab2-9921-b1ed25d321ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:55:25.345171 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19fa49e1-5c55-4ab2-9921-b1ed25d321ef admin admin] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:55:25.345459 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19fa49e1-5c55-4ab2-9921-b1ed25d321ef admin admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:55:25.345725 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19fa49e1-5c55-4ab2-9921-b1ed25d321ef admin admin] PATH_INFO: `/v3/roles` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:55:25.346462 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19fa49e1-5c55-4ab2-9921-b1ed25d321ef admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:55:25.348060 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19fa49e1-5c55-4ab2-9921-b1ed25d321ef admin admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:55:25.359619 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 884/1783] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Aug 1 16:55:25 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 16:55:25.360321 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19fa49e1-5c55-4ab2-9921-b1ed25d321ef admin admin] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:55:25.370362 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19fa49e1-5c55-4ab2-9921-b1ed25d321ef admin admin] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:55:25.372432 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.notifications [None req-9f1bff0f-2eef-4f09-858e-178795c9d69d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9e7dd0b9bb5743f4918fca51d577bcaa'} {{(pid=87058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:55:25.373338 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.notifications [None req-9f1bff0f-2eef-4f09-858e-178795c9d69d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9e7dd0b9bb5743f4918fca51d577bcaa'} {{(pid=87058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:55:25.374359 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.notifications [None req-9f1bff0f-2eef-4f09-858e-178795c9d69d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9e7dd0b9bb5743f4918fca51d577bcaa'} {{(pid=87058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:55:25.375481 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-676d05a9-ebc0-4794-9293-36a7574285fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:55:25.376169 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676d05a9-ebc0-4794-9293-36a7574285fa admin admin] REQUEST_METHOD: `POST` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:55:25.376511 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676d05a9-ebc0-4794-9293-36a7574285fa admin admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:55:25.376842 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676d05a9-ebc0-4794-9293-36a7574285fa admin admin] PATH_INFO: `/v3/projects` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:55:25.377620 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-676d05a9-ebc0-4794-9293-36a7574285fa admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:55:25.379703 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-676d05a9-ebc0-4794-9293-36a7574285fa admin admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:55:25.394503 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.notifications [None req-9f1bff0f-2eef-4f09-858e-178795c9d69d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9e7dd0b9bb5743f4918fca51d577bcaa'} {{(pid=87058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:55:25.404575 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.notifications [None req-9f1bff0f-2eef-4f09-858e-178795c9d69d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9e7dd0b9bb5743f4918fca51d577bcaa'} {{(pid=87058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:55:25.413938 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG dbcounter [-] [87058] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:55:25.493575 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 885/1784] 10.222.0.20 () {60 vars in 1153 bytes} [Mon Aug 1 16:55:25 2022] POST /identity/v3/projects => generated 365 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 16:55:25.493575 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-676d05a9-ebc0-4794-9293-36a7574285fa admin admin] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:55:25.493575 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.fernet_utils [None req-676d05a9-ebc0-4794-9293-36a7574285fa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87057) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:55:25.502264 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 900/1785] 10.222.0.20 () {58 vars in 1224 bytes} [Mon Aug 1 16:55:25 2022] DELETE /identity/v3/users/9e7dd0b9bb5743f4918fca51d577bcaa => generated 0 bytes in 412 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 16:55:25.506258 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f1bff0f-2eef-4f09-858e-178795c9d69d admin admin] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:55:25.512604 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.fernet_utils [None req-9f1bff0f-2eef-4f09-858e-178795c9d69d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:55:25.568421 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG dbcounter [-] [87057] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=87057) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:55:25.574474 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-676d05a9-ebc0-4794-9293-36a7574285fa admin admin] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:55:25.640812 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f1bff0f-2eef-4f09-858e-178795c9d69d admin admin] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:55:25.646594 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e14260d7-ca99-4899-91ba-368c23b1a5b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:55:25.647464 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14260d7-ca99-4899-91ba-368c23b1a5b6 admin admin] REQUEST_METHOD: `POST` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:55:25.647901 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14260d7-ca99-4899-91ba-368c23b1a5b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:55:25.648401 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14260d7-ca99-4899-91ba-368c23b1a5b6 admin admin] PATH_INFO: `/v3/users` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:55:25.649302 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e14260d7-ca99-4899-91ba-368c23b1a5b6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:55:25.652617 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e14260d7-ca99-4899-91ba-368c23b1a5b6 admin admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:55:25.688904 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3063f59d-9bb0-46ca-8074-6923b5b08916 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:55:25.690081 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3063f59d-9bb0-46ca-8074-6923b5b08916 admin admin] REQUEST_METHOD: `PUT` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:55:25.691961 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3063f59d-9bb0-46ca-8074-6923b5b08916 admin admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:55:25.693318 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3063f59d-9bb0-46ca-8074-6923b5b08916 admin admin] PATH_INFO: `/v3/projects/7d94184fe46443a6a339d35908cc2b13/users/bcdd827171964509bc43c7c24bd34da6/roles/0999d3da63a545f18ff3252afdc90518` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:55:25.722979 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3063f59d-9bb0-46ca-8074-6923b5b08916 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7d94184fe46443a6a339d35908cc2b13, user_id=bcdd827171964509bc43c7c24bd34da6, role_id=0999d3da63a545f18ff3252afdc90518)` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:55:25.731804 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3063f59d-9bb0-46ca-8074-6923b5b08916 admin admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:55:25.760694 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 901/1786] 10.222.0.20 () {60 vars in 1144 bytes} [Mon Aug 1 16:55:25 2022] POST /identity/v3/users => generated 283 bytes in 257 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 16:55:25.764263 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e14260d7-ca99-4899-91ba-368c23b1a5b6 admin admin] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:55:25.774231 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e14260d7-ca99-4899-91ba-368c23b1a5b6 admin admin] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:55:25.780766 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a96f12-d018-42c5-a469-979ff7ddbf9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:55:25.781444 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a96f12-d018-42c5-a469-979ff7ddbf9a admin admin] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:55:25.781724 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a96f12-d018-42c5-a469-979ff7ddbf9a admin admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:55:25.781997 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a96f12-d018-42c5-a469-979ff7ddbf9a admin admin] PATH_INFO: `/v3/roles` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:55:25.782727 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45a96f12-d018-42c5-a469-979ff7ddbf9a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:55:25.789487 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45a96f12-d018-42c5-a469-979ff7ddbf9a admin admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:55:25.795042 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 902/1787] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Aug 1 16:55:25 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 01 16:55:25.799003 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a96f12-d018-42c5-a469-979ff7ddbf9a admin admin] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:55:25.809339 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a96f12-d018-42c5-a469-979ff7ddbf9a admin admin] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:55:25.824279 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab9f03d-f886-45c6-8c7d-a8fc0b3d7e9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:55:25.825077 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab9f03d-f886-45c6-8c7d-a8fc0b3d7e9d admin admin] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:55:25.825852 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab9f03d-f886-45c6-8c7d-a8fc0b3d7e9d admin admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:55:25.825852 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab9f03d-f886-45c6-8c7d-a8fc0b3d7e9d admin admin] PATH_INFO: `/v3/roles` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:55:25.826732 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ab9f03d-f886-45c6-8c7d-a8fc0b3d7e9d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:55:25.828592 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ab9f03d-f886-45c6-8c7d-a8fc0b3d7e9d admin admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:55:25.830113 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 886/1788] 10.222.0.20 () {60 vars in 1483 bytes} [Mon Aug 1 16:55:25 2022] PUT /identity/v3/projects/7d94184fe46443a6a339d35908cc2b13/users/bcdd827171964509bc43c7c24bd34da6/roles/0999d3da63a545f18ff3252afdc90518 => generated 0 bytes in 336 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 16:55:25.831738 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b586822-f260-489e-9444-dfca0936b3ae None None] REQUEST_METHOD: `POST` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:55:25.832201 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b586822-f260-489e-9444-dfca0936b3ae None None] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:55:25.832484 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b586822-f260-489e-9444-dfca0936b3ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:55:25.843022 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 903/1789] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Aug 1 16:55:25 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 01 16:55:25.847339 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab9f03d-f886-45c6-8c7d-a8fc0b3d7e9d admin admin] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:55:25.854777 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.auth.core [None req-5b586822-f260-489e-9444-dfca0936b3ae None None] MFA Rules not processed for user `1007f8e379b74e169b98fb7c3c0ba5b4`. Rule list: `[]` (Enabled: `True`). {{(pid=87057) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:55:25.856011 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.fernet_utils [None req-5b586822-f260-489e-9444-dfca0936b3ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87057) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:55:25.862712 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab9f03d-f886-45c6-8c7d-a8fc0b3d7e9d admin admin] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:55:25.930451 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 887/1790] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 1 16:55:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:55:25.933964 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa7ead9b-7c3f-4bb4-bf79-17f502d83ada admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:55:25.934688 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7ead9b-7c3f-4bb4-bf79-17f502d83ada admin admin] REQUEST_METHOD: `PUT` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:55:25.934954 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7ead9b-7c3f-4bb4-bf79-17f502d83ada admin admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:55:25.935269 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7ead9b-7c3f-4bb4-bf79-17f502d83ada admin admin] PATH_INFO: `/v3/projects/14d7566fedb3472eb7bf7c2d6aef59b8/users/05b21ffc473e41839197077355739b2d/roles/0999d3da63a545f18ff3252afdc90518` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:55:25.944653 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b586822-f260-489e-9444-dfca0936b3ae None None] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:55:25.946703 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.fernet_utils [None req-5b586822-f260-489e-9444-dfca0936b3ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87057) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:55:25.951305 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa7ead9b-7c3f-4bb4-bf79-17f502d83ada admin admin] RBAC: Authorizing `identity:create_grant(project_id=14d7566fedb3472eb7bf7c2d6aef59b8, user_id=05b21ffc473e41839197077355739b2d, role_id=0999d3da63a545f18ff3252afdc90518)` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:55:25.954757 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa7ead9b-7c3f-4bb4-bf79-17f502d83ada admin admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:55:26.015994 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b586822-f260-489e-9444-dfca0936b3ae None None] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:55:26.020252 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9151cdd6-c2d6-4341-9bb6-b8de2a64fe5c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d9bd8d649154c979be3744bfd016f95', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b992d6a65df4bf9b27c13192d57c5f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:55:26.021002 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9151cdd6-c2d6-4341-9bb6-b8de2a64fe5c service neutron] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:55:26.021320 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9151cdd6-c2d6-4341-9bb6-b8de2a64fe5c service neutron] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:55:26.021629 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9151cdd6-c2d6-4341-9bb6-b8de2a64fe5c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:55:26.032959 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 904/1791] 10.222.0.20 () {60 vars in 1483 bytes} [Mon Aug 1 16:55:25 2022] PUT /identity/v3/projects/14d7566fedb3472eb7bf7c2d6aef59b8/users/05b21ffc473e41839197077355739b2d/roles/0999d3da63a545f18ff3252afdc90518 => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 16:55:26.036135 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9151cdd6-c2d6-4341-9bb6-b8de2a64fe5c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:55:26.037980 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9151cdd6-c2d6-4341-9bb6-b8de2a64fe5c service neutron] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:55:26.043017 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346f3bae-fb1e-4cb9-8600-59b767f79d1b None None] REQUEST_METHOD: `POST` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:55:26.043309 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346f3bae-fb1e-4cb9-8600-59b767f79d1b None None] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:55:26.043589 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346f3bae-fb1e-4cb9-8600-59b767f79d1b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:55:26.049912 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 888/1792] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 1 16:55:25 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 01 16:55:26.075202 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.auth.core [None req-346f3bae-fb1e-4cb9-8600-59b767f79d1b None None] MFA Rules not processed for user `05b21ffc473e41839197077355739b2d`. Rule list: `[]` (Enabled: `True`). {{(pid=87058) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:55:26.079806 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.fernet_utils [None req-346f3bae-fb1e-4cb9-8600-59b767f79d1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:55:26.097279 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG dbcounter [-] [87058] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=87058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:55:26.109143 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9151cdd6-c2d6-4341-9bb6-b8de2a64fe5c service neutron] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:55:26.111432 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.fernet_utils [None req-9151cdd6-c2d6-4341-9bb6-b8de2a64fe5c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87057) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:55:26.147310 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9151cdd6-c2d6-4341-9bb6-b8de2a64fe5c service neutron] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:55:26.153858 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 905/1793] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 1 16:55:26 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 16:55:26.158705 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-346f3bae-fb1e-4cb9-8600-59b767f79d1b None None] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:55:26.265469 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93bdc548-843d-40bc-81ea-72139b39300e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:55:26.266252 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93bdc548-843d-40bc-81ea-72139b39300e admin admin] REQUEST_METHOD: `DELETE` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:55:26.266529 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93bdc548-843d-40bc-81ea-72139b39300e admin admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:55:26.266761 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93bdc548-843d-40bc-81ea-72139b39300e admin admin] PATH_INFO: `/v3/projects/7267007bed914c8091878bd1f4773616` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:55:26.267998 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93bdc548-843d-40bc-81ea-72139b39300e admin admin] RBAC: Authorizing `identity:delete_project(project_id=7267007bed914c8091878bd1f4773616)` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:55:26.270473 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93bdc548-843d-40bc-81ea-72139b39300e admin admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:55:26.301711 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-346f3bae-fb1e-4cb9-8600-59b767f79d1b None None] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:55:26.316824 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-373473e8-353d-4e1f-a8ff-0b06aab1d081 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:55:26.321796 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373473e8-353d-4e1f-a8ff-0b06aab1d081 admin admin] REQUEST_METHOD: `POST` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:55:26.321796 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373473e8-353d-4e1f-a8ff-0b06aab1d081 admin admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:55:26.321796 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373473e8-353d-4e1f-a8ff-0b06aab1d081 admin admin] PATH_INFO: `/v3/projects` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:55:26.321796 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-373473e8-353d-4e1f-a8ff-0b06aab1d081 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:55:26.321796 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-373473e8-353d-4e1f-a8ff-0b06aab1d081 admin admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:55:26.437604 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 906/1794] 10.222.0.20 () {60 vars in 1153 bytes} [Mon Aug 1 16:55:26 2022] POST /identity/v3/projects => generated 365 bytes in 276 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 16:55:26.437604 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-373473e8-353d-4e1f-a8ff-0b06aab1d081 admin admin] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:55:26.439328 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.fernet_utils [None req-373473e8-353d-4e1f-a8ff-0b06aab1d081 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:55:26.468645 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.notifications [None req-93bdc548-843d-40bc-81ea-72139b39300e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7267007bed914c8091878bd1f4773616', 'request_id': 'req-93bdc548-843d-40bc-81ea-72139b39300e'} {{(pid=87057) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:55:26.485591 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.notifications [None req-93bdc548-843d-40bc-81ea-72139b39300e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7267007bed914c8091878bd1f4773616', 'request_id': 'req-93bdc548-843d-40bc-81ea-72139b39300e'} {{(pid=87057) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:55:26.500154 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.notifications [None req-93bdc548-843d-40bc-81ea-72139b39300e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7267007bed914c8091878bd1f4773616', 'request_id': 'req-93bdc548-843d-40bc-81ea-72139b39300e'} {{(pid=87057) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:55:26.558920 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG dbcounter [-] [87057] Writing DB stats keystone:SELECT=89,keystone:INSERT=2,keystone:DELETE=9 {{(pid=87057) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:55:26.584599 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-373473e8-353d-4e1f-a8ff-0b06aab1d081 admin admin] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:55:26.589469 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.notifications [None req-93bdc548-843d-40bc-81ea-72139b39300e admin admin] The token cache is being invalidate because project 7267007bed914c8091878bd1f4773616 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87057) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 16:55:26.590019 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.notifications [None req-93bdc548-843d-40bc-81ea-72139b39300e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87057) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:55:26.591230 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.notifications [None req-93bdc548-843d-40bc-81ea-72139b39300e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87057) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:55:26.593220 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 889/1795] 10.222.0.20 () {58 vars in 1233 bytes} [Mon Aug 1 16:55:26 2022] DELETE /identity/v3/projects/7267007bed914c8091878bd1f4773616 => generated 0 bytes in 486 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 16:55:26.593954 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93bdc548-843d-40bc-81ea-72139b39300e admin admin] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:55:26.595718 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.fernet_utils [None req-93bdc548-843d-40bc-81ea-72139b39300e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87057) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:55:26.608808 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f6b5d9-6983-4a23-984d-86d77d949d2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:55:26.609557 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f6b5d9-6983-4a23-984d-86d77d949d2e admin admin] REQUEST_METHOD: `DELETE` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:55:26.609841 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f6b5d9-6983-4a23-984d-86d77d949d2e admin admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:55:26.610143 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f6b5d9-6983-4a23-984d-86d77d949d2e admin admin] PATH_INFO: `/v3/users/30551e23b80642a3b78a082fb23b5cf4` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:55:26.611584 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5f6b5d9-6983-4a23-984d-86d77d949d2e admin admin] RBAC: Authorizing `identity:delete_user(user_id=30551e23b80642a3b78a082fb23b5cf4)` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:55:26.613629 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5f6b5d9-6983-4a23-984d-86d77d949d2e admin admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:55:26.684056 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93bdc548-843d-40bc-81ea-72139b39300e admin admin] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:55:26.689517 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12b3ca9f-5f18-4fd6-91b2-2f34f4d22782 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:55:26.690192 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b3ca9f-5f18-4fd6-91b2-2f34f4d22782 admin admin] REQUEST_METHOD: `POST` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:55:26.690514 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b3ca9f-5f18-4fd6-91b2-2f34f4d22782 admin admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:55:26.690776 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b3ca9f-5f18-4fd6-91b2-2f34f4d22782 admin admin] PATH_INFO: `/v3/users` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:55:26.691543 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12b3ca9f-5f18-4fd6-91b2-2f34f4d22782 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:55:26.693659 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12b3ca9f-5f18-4fd6-91b2-2f34f4d22782 admin admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:55:26.767281 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 890/1796] 10.222.0.20 () {60 vars in 1144 bytes} [Mon Aug 1 16:55:26 2022] POST /identity/v3/users => generated 289 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 16:55:26.768765 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3a51b1-a0a2-4e0e-91f6-433138ddd459 None None] REQUEST_METHOD: `POST` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:55:26.769150 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3a51b1-a0a2-4e0e-91f6-433138ddd459 None None] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:55:26.769546 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3a51b1-a0a2-4e0e-91f6-433138ddd459 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:55:26.797691 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.notifications [None req-e5f6b5d9-6983-4a23-984d-86d77d949d2e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '30551e23b80642a3b78a082fb23b5cf4'} {{(pid=87058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:55:26.798483 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.notifications [None req-e5f6b5d9-6983-4a23-984d-86d77d949d2e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '30551e23b80642a3b78a082fb23b5cf4'} {{(pid=87058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:55:26.799418 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.notifications [None req-e5f6b5d9-6983-4a23-984d-86d77d949d2e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '30551e23b80642a3b78a082fb23b5cf4'} {{(pid=87058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:55:26.809712 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.auth.core [None req-2f3a51b1-a0a2-4e0e-91f6-433138ddd459 None None] MFA Rules not processed for user `1007f8e379b74e169b98fb7c3c0ba5b4`. Rule list: `[]` (Enabled: `True`). {{(pid=87057) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:55:26.813725 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.fernet_utils [None req-2f3a51b1-a0a2-4e0e-91f6-433138ddd459 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87057) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:55:26.823335 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 891/1797] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 1 16:55:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 55 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:55:26.824288 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f3a51b1-a0a2-4e0e-91f6-433138ddd459 None None] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:55:26.828667 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.fernet_utils [None req-2f3a51b1-a0a2-4e0e-91f6-433138ddd459 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87057) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:55:26.833645 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.notifications [None req-e5f6b5d9-6983-4a23-984d-86d77d949d2e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '30551e23b80642a3b78a082fb23b5cf4'} {{(pid=87058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:55:26.842362 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG dbcounter [-] [87058] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:55:26.843446 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.notifications [None req-e5f6b5d9-6983-4a23-984d-86d77d949d2e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '30551e23b80642a3b78a082fb23b5cf4'} {{(pid=87058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:55:26.849129 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f3a51b1-a0a2-4e0e-91f6-433138ddd459 None None] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:55:26.858354 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-951f6e00-db3e-4658-97ee-0dc86dc2c22a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:55:26.859231 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951f6e00-db3e-4658-97ee-0dc86dc2c22a admin admin] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:55:26.859708 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951f6e00-db3e-4658-97ee-0dc86dc2c22a admin admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:55:26.860374 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951f6e00-db3e-4658-97ee-0dc86dc2c22a admin admin] PATH_INFO: `/v3/roles` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:55:26.862849 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-951f6e00-db3e-4658-97ee-0dc86dc2c22a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:55:26.862849 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-951f6e00-db3e-4658-97ee-0dc86dc2c22a admin admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:55:26.878970 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 892/1798] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Aug 1 16:55:26 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 16:55:26.878970 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5366a56e-5d34-4386-a3ac-f587b6fff76a None None] REQUEST_METHOD: `POST` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:55:26.880901 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5366a56e-5d34-4386-a3ac-f587b6fff76a None None] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:55:26.881393 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5366a56e-5d34-4386-a3ac-f587b6fff76a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:55:26.920803 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.auth.core [None req-5366a56e-5d34-4386-a3ac-f587b6fff76a None None] MFA Rules not processed for user `1007f8e379b74e169b98fb7c3c0ba5b4`. Rule list: `[]` (Enabled: `True`). {{(pid=87057) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:55:26.925742 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 907/1799] 10.222.0.20 () {58 vars in 1224 bytes} [Mon Aug 1 16:55:26 2022] DELETE /identity/v3/users/30551e23b80642a3b78a082fb23b5cf4 => generated 0 bytes in 491 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 16:55:26.926846 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f6b5d9-6983-4a23-984d-86d77d949d2e admin admin] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:55:26.930152 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.fernet_utils [None req-5366a56e-5d34-4386-a3ac-f587b6fff76a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87057) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:55:27.071792 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG dbcounter [-] [87057] Writing DB stats keystone:SELECT=90,keystone:DELETE=5,keystone:INSERT=5 {{(pid=87057) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:55:27.076006 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 893/1800] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 1 16:55:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:55:27.076006 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1546dda-c964-45da-bf7a-39810bdce2e1 None None] REQUEST_METHOD: `POST` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:55:27.079781 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1546dda-c964-45da-bf7a-39810bdce2e1 None None] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:55:27.079781 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1546dda-c964-45da-bf7a-39810bdce2e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:55:27.083682 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f6b5d9-6983-4a23-984d-86d77d949d2e admin admin] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:55:27.095014 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13424bbc-fcbd-4485-9fcf-99cdeea9b690 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:55:27.095730 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13424bbc-fcbd-4485-9fcf-99cdeea9b690 admin admin] REQUEST_METHOD: `PUT` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:55:27.096015 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13424bbc-fcbd-4485-9fcf-99cdeea9b690 admin admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:55:27.096311 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13424bbc-fcbd-4485-9fcf-99cdeea9b690 admin admin] PATH_INFO: `/v3/projects/b2493237ab69425787f161b81b26407a/users/9f0da3bf8fd64382907da9412ddc3b37/roles/440bb943dd5b46fea038b903b6eaaf81` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:55:27.117909 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13424bbc-fcbd-4485-9fcf-99cdeea9b690 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b2493237ab69425787f161b81b26407a, user_id=9f0da3bf8fd64382907da9412ddc3b37, role_id=440bb943dd5b46fea038b903b6eaaf81)` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:55:27.122993 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.auth.core [None req-f1546dda-c964-45da-bf7a-39810bdce2e1 None None] MFA Rules not processed for user `1007f8e379b74e169b98fb7c3c0ba5b4`. Rule list: `[]` (Enabled: `True`). {{(pid=87057) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:55:27.123220 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13424bbc-fcbd-4485-9fcf-99cdeea9b690 admin admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:55:27.124987 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.fernet_utils [None req-f1546dda-c964-45da-bf7a-39810bdce2e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87057) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:55:27.152460 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG dbcounter [-] [87058] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=87058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:55:27.160083 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 894/1801] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 1 16:55:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:55:27.161203 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1546dda-c964-45da-bf7a-39810bdce2e1 None None] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:55:27.179026 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1546dda-c964-45da-bf7a-39810bdce2e1 None None] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:55:27.193815 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebbc3aca-e3a4-413c-81b7-47949ae92ade admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:55:27.195564 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebbc3aca-e3a4-413c-81b7-47949ae92ade admin admin] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:55:27.196038 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebbc3aca-e3a4-413c-81b7-47949ae92ade admin admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:55:27.196940 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebbc3aca-e3a4-413c-81b7-47949ae92ade admin admin] PATH_INFO: `/v3/domains` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:55:27.202357 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebbc3aca-e3a4-413c-81b7-47949ae92ade admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87057) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:55:27.202695 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebbc3aca-e3a4-413c-81b7-47949ae92ade admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:55:27.205644 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebbc3aca-e3a4-413c-81b7-47949ae92ade admin admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:55:27.214936 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 908/1802] 10.222.0.20 () {60 vars in 1483 bytes} [Mon Aug 1 16:55:26 2022] PUT /identity/v3/projects/b2493237ab69425787f161b81b26407a/users/9f0da3bf8fd64382907da9412ddc3b37/roles/440bb943dd5b46fea038b903b6eaaf81 => generated 0 bytes in 288 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 16:55:27.216313 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13424bbc-fcbd-4485-9fcf-99cdeea9b690 admin admin] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:55:27.218395 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.fernet_utils [None req-13424bbc-fcbd-4485-9fcf-99cdeea9b690 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:55:27.241696 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 895/1803] 10.222.0.20 () {58 vars in 1153 bytes} [Mon Aug 1 16:55:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:55:27.242702 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebbc3aca-e3a4-413c-81b7-47949ae92ade admin admin] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:55:27.259330 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebbc3aca-e3a4-413c-81b7-47949ae92ade admin admin] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:55:27.322065 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13424bbc-fcbd-4485-9fcf-99cdeea9b690 admin admin] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:55:27.327179 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80fbcef1-ca17-43a1-a1ff-be007e8eed4f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d9bd8d649154c979be3744bfd016f95', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b992d6a65df4bf9b27c13192d57c5f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:55:27.328270 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80fbcef1-ca17-43a1-a1ff-be007e8eed4f service neutron] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:55:27.328726 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80fbcef1-ca17-43a1-a1ff-be007e8eed4f service neutron] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:55:27.329144 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80fbcef1-ca17-43a1-a1ff-be007e8eed4f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:55:27.346021 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07af8d74-f5f1-476e-9a98-be2480166fae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:55:27.347140 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07af8d74-f5f1-476e-9a98-be2480166fae admin admin] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:55:27.347577 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07af8d74-f5f1-476e-9a98-be2480166fae admin admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:55:27.348760 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07af8d74-f5f1-476e-9a98-be2480166fae admin admin] PATH_INFO: `/v3/roles` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:55:27.349837 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07af8d74-f5f1-476e-9a98-be2480166fae admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:55:27.353295 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07af8d74-f5f1-476e-9a98-be2480166fae admin admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:55:27.373157 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 896/1804] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Aug 1 16:55:27 2022] GET /identity/v3/roles => generated 1420 bytes in 128 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 16:55:27.373157 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07af8d74-f5f1-476e-9a98-be2480166fae admin admin] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:55:27.387100 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07af8d74-f5f1-476e-9a98-be2480166fae admin admin] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:55:27.394583 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4690518-9096-481b-9463-9ba8dca6bed9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:55:27.395312 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4690518-9096-481b-9463-9ba8dca6bed9 admin admin] REQUEST_METHOD: `POST` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:55:27.395645 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4690518-9096-481b-9463-9ba8dca6bed9 admin admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:55:27.396152 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4690518-9096-481b-9463-9ba8dca6bed9 admin admin] PATH_INFO: `/v3/projects` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:55:27.397287 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4690518-9096-481b-9463-9ba8dca6bed9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:55:27.401345 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4690518-9096-481b-9463-9ba8dca6bed9 admin admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:55:27.415092 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80fbcef1-ca17-43a1-a1ff-be007e8eed4f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:55:27.418218 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80fbcef1-ca17-43a1-a1ff-be007e8eed4f service neutron] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:55:27.431399 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 909/1805] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 1 16:55:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 215 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 16:55:27.432639 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80fbcef1-ca17-43a1-a1ff-be007e8eed4f service neutron] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:55:27.443120 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80fbcef1-ca17-43a1-a1ff-be007e8eed4f service neutron] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:55:27.448579 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb0d05c-1140-4940-8cdc-3025b49ef41c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:55:27.449255 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb0d05c-1140-4940-8cdc-3025b49ef41c admin admin] REQUEST_METHOD: `PUT` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:55:27.449532 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb0d05c-1140-4940-8cdc-3025b49ef41c admin admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:55:27.449810 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb0d05c-1140-4940-8cdc-3025b49ef41c admin admin] PATH_INFO: `/v3/projects/b2493237ab69425787f161b81b26407a/users/9f0da3bf8fd64382907da9412ddc3b37/roles/440bb943dd5b46fea038b903b6eaaf81` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:55:27.451902 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcb0d05c-1140-4940-8cdc-3025b49ef41c admin admin] RBAC: Authorizing `identity:create_grant(project_id=b2493237ab69425787f161b81b26407a, user_id=9f0da3bf8fd64382907da9412ddc3b37, role_id=440bb943dd5b46fea038b903b6eaaf81)` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:55:27.454867 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcb0d05c-1140-4940-8cdc-3025b49ef41c admin admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:55:27.474138 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 910/1806] 10.222.0.20 () {60 vars in 1483 bytes} [Mon Aug 1 16:55:27 2022] PUT /identity/v3/projects/b2493237ab69425787f161b81b26407a/users/9f0da3bf8fd64382907da9412ddc3b37/roles/440bb943dd5b46fea038b903b6eaaf81 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 16:55:27.480490 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010ecd2f-45a0-4617-8a19-4464d59e0140 None None] REQUEST_METHOD: `POST` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:55:27.481485 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010ecd2f-45a0-4617-8a19-4464d59e0140 None None] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:55:27.481485 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010ecd2f-45a0-4617-8a19-4464d59e0140 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:55:27.522731 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 897/1807] 10.222.0.20 () {60 vars in 1153 bytes} [Mon Aug 1 16:55:27 2022] POST /identity/v3/projects => generated 367 bytes in 148 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 16:55:27.526817 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4690518-9096-481b-9463-9ba8dca6bed9 admin admin] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:55:27.539313 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.auth.core [None req-010ecd2f-45a0-4617-8a19-4464d59e0140 None None] MFA Rules not processed for user `9f0da3bf8fd64382907da9412ddc3b37`. Rule list: `[]` (Enabled: `True`). {{(pid=87058) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:55:27.543126 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.fernet_utils [None req-010ecd2f-45a0-4617-8a19-4464d59e0140 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:55:27.669702 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4690518-9096-481b-9463-9ba8dca6bed9 admin admin] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:55:27.686008 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 911/1808] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 1 16:55:27 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:55:27.688067 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9418e46-d016-431c-b0dc-b9fd481ddad1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:55:27.688363 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9418e46-d016-431c-b0dc-b9fd481ddad1 admin admin] REQUEST_METHOD: `POST` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:55:27.688644 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9418e46-d016-431c-b0dc-b9fd481ddad1 admin admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:55:27.688924 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9418e46-d016-431c-b0dc-b9fd481ddad1 admin admin] PATH_INFO: `/v3/users` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:55:27.690152 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9418e46-d016-431c-b0dc-b9fd481ddad1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:55:27.692428 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9418e46-d016-431c-b0dc-b9fd481ddad1 admin admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:55:27.697947 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-010ecd2f-45a0-4617-8a19-4464d59e0140 None None] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:55:27.703445 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.fernet_utils [None req-010ecd2f-45a0-4617-8a19-4464d59e0140 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:55:27.719798 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG dbcounter [-] [87057] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87057) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:55:27.795528 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-010ecd2f-45a0-4617-8a19-4464d59e0140 None None] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:55:27.800120 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d41377a-8d8f-4512-b70c-0b8a973042a1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25970b52ca15448fa1abb1f974d64038', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b992d6a65df4bf9b27c13192d57c5f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:55:27.801253 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d41377a-8d8f-4512-b70c-0b8a973042a1 service cinder] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:55:27.801585 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d41377a-8d8f-4512-b70c-0b8a973042a1 service cinder] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:55:27.801963 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d41377a-8d8f-4512-b70c-0b8a973042a1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:55:27.828416 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 898/1809] 10.222.0.20 () {60 vars in 1144 bytes} [Mon Aug 1 16:55:27 2022] POST /identity/v3/users => generated 284 bytes in 303 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 16:55:27.828580 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d41377a-8d8f-4512-b70c-0b8a973042a1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:55:27.831114 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9418e46-d016-431c-b0dc-b9fd481ddad1 admin admin] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:55:27.832346 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d41377a-8d8f-4512-b70c-0b8a973042a1 service cinder] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:55:27.862304 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9418e46-d016-431c-b0dc-b9fd481ddad1 admin admin] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:55:27.863392 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 912/1810] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 1 16:55:27 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 166 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 16:55:27.868141 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c19a3a-73d9-4346-a608-817ec2c20027 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:55:27.868897 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c19a3a-73d9-4346-a608-817ec2c20027 admin admin] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:55:27.870235 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c19a3a-73d9-4346-a608-817ec2c20027 admin admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:55:27.870625 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c19a3a-73d9-4346-a608-817ec2c20027 admin admin] PATH_INFO: `/v3/roles` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:55:27.871772 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0c19a3a-73d9-4346-a608-817ec2c20027 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:55:27.873802 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0c19a3a-73d9-4346-a608-817ec2c20027 admin admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:55:27.883139 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG dbcounter [-] [87058] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=87058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:55:27.894304 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 899/1811] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Aug 1 16:55:27 2022] GET /identity/v3/roles => generated 1420 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 16:55:27.895487 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d41377a-8d8f-4512-b70c-0b8a973042a1 service cinder] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:55:27.922298 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d41377a-8d8f-4512-b70c-0b8a973042a1 service cinder] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:55:27.928472 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07643240-f8d9-47c3-9be6-71d32bacd303 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:55:27.929217 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07643240-f8d9-47c3-9be6-71d32bacd303 admin admin] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:55:27.929887 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07643240-f8d9-47c3-9be6-71d32bacd303 admin admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:55:27.930296 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07643240-f8d9-47c3-9be6-71d32bacd303 admin admin] PATH_INFO: `/v3/roles` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:55:27.932339 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07643240-f8d9-47c3-9be6-71d32bacd303 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:55:27.934120 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07643240-f8d9-47c3-9be6-71d32bacd303 admin admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:55:27.950437 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 913/1812] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Aug 1 16:55:27 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 16:55:27.952041 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c19a3a-73d9-4346-a608-817ec2c20027 admin admin] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:55:27.956824 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c65646c-1fff-41b8-971a-a3e3b93f4137 None None] REQUEST_METHOD: `POST` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:55:27.958142 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c65646c-1fff-41b8-971a-a3e3b93f4137 None None] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:55:27.958705 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c65646c-1fff-41b8-971a-a3e3b93f4137 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:55:27.969490 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c19a3a-73d9-4346-a608-817ec2c20027 admin admin] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:55:27.975077 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce31622f-bd60-491d-a9cf-1933aac9940e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:55:27.977895 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce31622f-bd60-491d-a9cf-1933aac9940e admin admin] REQUEST_METHOD: `PUT` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:55:27.977895 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce31622f-bd60-491d-a9cf-1933aac9940e admin admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:55:27.977895 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce31622f-bd60-491d-a9cf-1933aac9940e admin admin] PATH_INFO: `/v3/projects/7e6dba3c3e624cf090439b3545d22944/users/4fa2710924b444188db11766e97762db/roles/0999d3da63a545f18ff3252afdc90518` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:55:27.996479 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce31622f-bd60-491d-a9cf-1933aac9940e admin admin] RBAC: Authorizing `identity:create_grant(project_id=7e6dba3c3e624cf090439b3545d22944, user_id=4fa2710924b444188db11766e97762db, role_id=0999d3da63a545f18ff3252afdc90518)` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:55:27.999996 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce31622f-bd60-491d-a9cf-1933aac9940e admin admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:55:28.006320 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.auth.core [None req-5c65646c-1fff-41b8-971a-a3e3b93f4137 None None] MFA Rules not processed for user `bcdd827171964509bc43c7c24bd34da6`. Rule list: `[]` (Enabled: `True`). {{(pid=87058) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:55:28.007575 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.fernet_utils [None req-5c65646c-1fff-41b8-971a-a3e3b93f4137 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:55:28.094176 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 914/1813] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 1 16:55:27 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:55:28.100479 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c65646c-1fff-41b8-971a-a3e3b93f4137 None None] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:55:28.104855 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.fernet_utils [None req-5c65646c-1fff-41b8-971a-a3e3b93f4137 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:55:28.127923 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c65646c-1fff-41b8-971a-a3e3b93f4137 None None] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:55:28.128166 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba750c43-7ea4-4ab8-84dd-4a9eb486287c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:55:28.128889 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba750c43-7ea4-4ab8-84dd-4a9eb486287c admin admin] REQUEST_METHOD: `DELETE` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:55:28.129259 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba750c43-7ea4-4ab8-84dd-4a9eb486287c admin admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:55:28.129607 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba750c43-7ea4-4ab8-84dd-4a9eb486287c admin admin] PATH_INFO: `/v3/projects/be237170c5384dcb8cb74b603fddca4b` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:55:28.130898 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba750c43-7ea4-4ab8-84dd-4a9eb486287c admin admin] RBAC: Authorizing `identity:delete_project(project_id=be237170c5384dcb8cb74b603fddca4b)` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:55:28.132898 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba750c43-7ea4-4ab8-84dd-4a9eb486287c admin admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:55:28.152160 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 900/1814] 10.222.0.20 () {60 vars in 1483 bytes} [Mon Aug 1 16:55:27 2022] PUT /identity/v3/projects/7e6dba3c3e624cf090439b3545d22944/users/4fa2710924b444188db11766e97762db/roles/0999d3da63a545f18ff3252afdc90518 => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 16:55:28.152762 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce31622f-bd60-491d-a9cf-1933aac9940e admin admin] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:55:28.154465 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.fernet_utils [None req-ce31622f-bd60-491d-a9cf-1933aac9940e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87057) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:55:28.255303 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce31622f-bd60-491d-a9cf-1933aac9940e admin admin] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:55:28.263027 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.notifications [None req-ba750c43-7ea4-4ab8-84dd-4a9eb486287c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'be237170c5384dcb8cb74b603fddca4b', 'request_id': 'req-ba750c43-7ea4-4ab8-84dd-4a9eb486287c'} {{(pid=87058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:55:28.273343 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.notifications [None req-ba750c43-7ea4-4ab8-84dd-4a9eb486287c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'be237170c5384dcb8cb74b603fddca4b', 'request_id': 'req-ba750c43-7ea4-4ab8-84dd-4a9eb486287c'} {{(pid=87058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:55:28.283959 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.notifications [None req-ba750c43-7ea4-4ab8-84dd-4a9eb486287c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'be237170c5384dcb8cb74b603fddca4b', 'request_id': 'req-ba750c43-7ea4-4ab8-84dd-4a9eb486287c'} {{(pid=87058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:55:28.331427 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c62a6a-faa1-4b94-ac7f-ba8ed8d710f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:55:28.332158 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c62a6a-faa1-4b94-ac7f-ba8ed8d710f4 admin admin] REQUEST_METHOD: `POST` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:55:28.332505 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c62a6a-faa1-4b94-ac7f-ba8ed8d710f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:55:28.332851 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c62a6a-faa1-4b94-ac7f-ba8ed8d710f4 admin admin] PATH_INFO: `/v3/projects` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:55:28.333614 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8c62a6a-faa1-4b94-ac7f-ba8ed8d710f4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:55:28.335782 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8c62a6a-faa1-4b94-ac7f-ba8ed8d710f4 admin admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:55:28.375855 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.notifications [None req-ba750c43-7ea4-4ab8-84dd-4a9eb486287c admin admin] The token cache is being invalidate because project be237170c5384dcb8cb74b603fddca4b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87058) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 16:55:28.375855 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.notifications [None req-ba750c43-7ea4-4ab8-84dd-4a9eb486287c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:55:28.376632 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.notifications [None req-ba750c43-7ea4-4ab8-84dd-4a9eb486287c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:55:28.378562 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 915/1815] 10.222.0.20 () {58 vars in 1233 bytes} [Mon Aug 1 16:55:28 2022] DELETE /identity/v3/projects/be237170c5384dcb8cb74b603fddca4b => generated 0 bytes in 282 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 16:55:28.382704 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7390bc96-9e08-428a-943e-c80daa80767c None None] REQUEST_METHOD: `POST` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:55:28.382991 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7390bc96-9e08-428a-943e-c80daa80767c None None] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:55:28.383428 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7390bc96-9e08-428a-943e-c80daa80767c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:55:28.429905 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.auth.core [None req-7390bc96-9e08-428a-943e-c80daa80767c None None] MFA Rules not processed for user `4fa2710924b444188db11766e97762db`. Rule list: `[]` (Enabled: `True`). {{(pid=87058) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:55:28.431228 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.fernet_utils [None req-7390bc96-9e08-428a-943e-c80daa80767c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:55:28.475337 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 901/1816] 10.222.0.20 () {60 vars in 1153 bytes} [Mon Aug 1 16:55:28 2022] POST /identity/v3/projects => generated 371 bytes in 323 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 16:55:28.476185 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c62a6a-faa1-4b94-ac7f-ba8ed8d710f4 admin admin] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:55:28.481072 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.fernet_utils [None req-d8c62a6a-faa1-4b94-ac7f-ba8ed8d710f4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87057) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:55:28.497404 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 916/1817] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 1 16:55:28 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:55:28.498455 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7390bc96-9e08-428a-943e-c80daa80767c None None] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:55:28.499782 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.fernet_utils [None req-7390bc96-9e08-428a-943e-c80daa80767c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:55:28.513510 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7390bc96-9e08-428a-943e-c80daa80767c None None] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:55:28.571807 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c62a6a-faa1-4b94-ac7f-ba8ed8d710f4 admin admin] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:55:28.581038 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49c93aaa-5a86-4999-8945-2b3e75a88f2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:55:28.581908 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c93aaa-5a86-4999-8945-2b3e75a88f2d admin admin] REQUEST_METHOD: `DELETE` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:55:28.582300 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c93aaa-5a86-4999-8945-2b3e75a88f2d admin admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:55:28.582670 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c93aaa-5a86-4999-8945-2b3e75a88f2d admin admin] PATH_INFO: `/v3/users/9225c88654b84473afa5296594705c31` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:55:28.584164 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49c93aaa-5a86-4999-8945-2b3e75a88f2d admin admin] RBAC: Authorizing `identity:delete_user(user_id=9225c88654b84473afa5296594705c31)` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:55:28.586527 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49c93aaa-5a86-4999-8945-2b3e75a88f2d admin admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:55:28.593199 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5024074-c979-4170-a0a0-73d70808fca4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:55:28.598833 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5024074-c979-4170-a0a0-73d70808fca4 admin admin] REQUEST_METHOD: `POST` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:55:28.598833 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5024074-c979-4170-a0a0-73d70808fca4 admin admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:55:28.598833 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5024074-c979-4170-a0a0-73d70808fca4 admin admin] PATH_INFO: `/v3/users` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:55:28.598833 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5024074-c979-4170-a0a0-73d70808fca4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:55:28.598833 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5024074-c979-4170-a0a0-73d70808fca4 admin admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:55:28.677872 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 917/1818] 10.222.0.20 () {60 vars in 1144 bytes} [Mon Aug 1 16:55:28 2022] POST /identity/v3/users => generated 292 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 16:55:28.679115 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5024074-c979-4170-a0a0-73d70808fca4 admin admin] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:55:28.683509 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.fernet_utils [None req-d5024074-c979-4170-a0a0-73d70808fca4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:55:28.697044 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5024074-c979-4170-a0a0-73d70808fca4 admin admin] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:55:28.706674 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea7cf818-8285-4657-becf-539925b4cfd5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:55:28.707400 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7cf818-8285-4657-becf-539925b4cfd5 admin admin] REQUEST_METHOD: `POST` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:55:28.708078 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7cf818-8285-4657-becf-539925b4cfd5 admin admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:55:28.708425 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7cf818-8285-4657-becf-539925b4cfd5 admin admin] PATH_INFO: `/v3/projects` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:55:28.709157 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea7cf818-8285-4657-becf-539925b4cfd5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:55:28.710871 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG dbcounter [-] [87057] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=87057) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:55:28.711811 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea7cf818-8285-4657-becf-539925b4cfd5 admin admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:55:28.757761 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.notifications [None req-49c93aaa-5a86-4999-8945-2b3e75a88f2d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9225c88654b84473afa5296594705c31'} {{(pid=87057) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:55:28.758461 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.notifications [None req-49c93aaa-5a86-4999-8945-2b3e75a88f2d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9225c88654b84473afa5296594705c31'} {{(pid=87057) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:55:28.842495 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.notifications [None req-49c93aaa-5a86-4999-8945-2b3e75a88f2d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9225c88654b84473afa5296594705c31'} {{(pid=87057) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:55:28.843166 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.notifications [None req-49c93aaa-5a86-4999-8945-2b3e75a88f2d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9225c88654b84473afa5296594705c31'} {{(pid=87057) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:55:28.851196 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 918/1819] 10.222.0.20 () {60 vars in 1153 bytes} [Mon Aug 1 16:55:28 2022] POST /identity/v3/projects => generated 369 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 16:55:28.852081 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea7cf818-8285-4657-becf-539925b4cfd5 admin admin] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:55:28.853708 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.notifications [None req-49c93aaa-5a86-4999-8945-2b3e75a88f2d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9225c88654b84473afa5296594705c31'} {{(pid=87057) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:55:28.858195 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.fernet_utils [None req-ea7cf818-8285-4657-becf-539925b4cfd5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:55:28.874732 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 902/1820] 10.222.0.20 () {58 vars in 1224 bytes} [Mon Aug 1 16:55:28 2022] DELETE /identity/v3/users/9225c88654b84473afa5296594705c31 => generated 0 bytes in 399 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 16:55:28.875756 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49c93aaa-5a86-4999-8945-2b3e75a88f2d admin admin] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:55:28.877160 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.fernet_utils [None req-49c93aaa-5a86-4999-8945-2b3e75a88f2d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87057) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:55:28.962495 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea7cf818-8285-4657-becf-539925b4cfd5 admin admin] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:55:28.987605 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49c93aaa-5a86-4999-8945-2b3e75a88f2d admin admin] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:55:28.993327 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b8bdd24-d788-4346-945f-50fc6142ba35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:55:28.994855 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8bdd24-d788-4346-945f-50fc6142ba35 admin admin] REQUEST_METHOD: `POST` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:55:28.995142 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8bdd24-d788-4346-945f-50fc6142ba35 admin admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:55:28.995577 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8bdd24-d788-4346-945f-50fc6142ba35 admin admin] PATH_INFO: `/v3/users` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:55:28.996270 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b8bdd24-d788-4346-945f-50fc6142ba35 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:55:28.998341 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b8bdd24-d788-4346-945f-50fc6142ba35 admin admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:55:29.058429 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fdced23-93b1-409f-b3e1-c883ed76e3c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:55:29.060657 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fdced23-93b1-409f-b3e1-c883ed76e3c5 admin admin] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:55:29.060983 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fdced23-93b1-409f-b3e1-c883ed76e3c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:55:29.061263 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fdced23-93b1-409f-b3e1-c883ed76e3c5 admin admin] PATH_INFO: `/v3/roles` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:55:29.062014 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fdced23-93b1-409f-b3e1-c883ed76e3c5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:55:29.066710 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fdced23-93b1-409f-b3e1-c883ed76e3c5 admin admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:55:29.077195 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG dbcounter [-] [87058] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=87058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:55:29.079643 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 903/1821] 10.222.0.20 () {60 vars in 1144 bytes} [Mon Aug 1 16:55:28 2022] POST /identity/v3/users => generated 291 bytes in 204 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 16:55:29.081908 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b8bdd24-d788-4346-945f-50fc6142ba35 admin admin] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:55:29.084702 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.fernet_utils [None req-6b8bdd24-d788-4346-945f-50fc6142ba35 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87057) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:55:29.085320 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 919/1822] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Aug 1 16:55:28 2022] GET /identity/v3/roles => generated 1420 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 16:55:29.088610 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fdced23-93b1-409f-b3e1-c883ed76e3c5 admin admin] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:55:29.101548 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b8bdd24-d788-4346-945f-50fc6142ba35 admin admin] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:55:29.101744 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fdced23-93b1-409f-b3e1-c883ed76e3c5 admin admin] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:55:29.112727 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b0cee5-3aff-4897-91f5-23203c68e6af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:55:29.113173 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17b1b101-c5f3-489d-82e6-c37fbe730171 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:55:29.114684 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b0cee5-3aff-4897-91f5-23203c68e6af admin admin] REQUEST_METHOD: `DELETE` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:55:29.115293 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b1b101-c5f3-489d-82e6-c37fbe730171 admin admin] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:55:29.115979 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b1b101-c5f3-489d-82e6-c37fbe730171 admin admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:55:29.116726 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b0cee5-3aff-4897-91f5-23203c68e6af admin admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:55:29.117265 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b1b101-c5f3-489d-82e6-c37fbe730171 admin admin] PATH_INFO: `/v3/roles` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:55:29.117637 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b0cee5-3aff-4897-91f5-23203c68e6af admin admin] PATH_INFO: `/v3/users/05b21ffc473e41839197077355739b2d` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:55:29.122797 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8b0cee5-3aff-4897-91f5-23203c68e6af admin admin] RBAC: Authorizing `identity:delete_user(user_id=05b21ffc473e41839197077355739b2d)` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:55:29.123026 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17b1b101-c5f3-489d-82e6-c37fbe730171 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:55:29.123120 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8b0cee5-3aff-4897-91f5-23203c68e6af admin admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:55:29.125930 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17b1b101-c5f3-489d-82e6-c37fbe730171 admin admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:55:29.157348 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 920/1823] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Aug 1 16:55:29 2022] GET /identity/v3/roles => generated 1420 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 16:55:29.159089 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17b1b101-c5f3-489d-82e6-c37fbe730171 admin admin] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:55:29.169832 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17b1b101-c5f3-489d-82e6-c37fbe730171 admin admin] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:55:29.176352 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-176d25e0-0063-4678-97c6-2e8edc72c75f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:55:29.177103 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176d25e0-0063-4678-97c6-2e8edc72c75f admin admin] REQUEST_METHOD: `PUT` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:55:29.177851 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176d25e0-0063-4678-97c6-2e8edc72c75f admin admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:55:29.178333 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176d25e0-0063-4678-97c6-2e8edc72c75f admin admin] PATH_INFO: `/v3/projects/dc905974d67c4366b4a6a91c43706466/users/20b77010606549cb8f5d23dad50f16d9/roles/440bb943dd5b46fea038b903b6eaaf81` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:55:29.202439 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-176d25e0-0063-4678-97c6-2e8edc72c75f admin admin] RBAC: Authorizing `identity:create_grant(project_id=dc905974d67c4366b4a6a91c43706466, user_id=20b77010606549cb8f5d23dad50f16d9, role_id=440bb943dd5b46fea038b903b6eaaf81)` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:55:29.231965 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-176d25e0-0063-4678-97c6-2e8edc72c75f admin admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:55:29.309848 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 921/1824] 10.222.0.20 () {60 vars in 1483 bytes} [Mon Aug 1 16:55:29 2022] PUT /identity/v3/projects/dc905974d67c4366b4a6a91c43706466/users/20b77010606549cb8f5d23dad50f16d9/roles/440bb943dd5b46fea038b903b6eaaf81 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 16:55:29.309848 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-176d25e0-0063-4678-97c6-2e8edc72c75f admin admin] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:55:29.311651 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.fernet_utils [None req-176d25e0-0063-4678-97c6-2e8edc72c75f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:55:29.373867 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.notifications [None req-e8b0cee5-3aff-4897-91f5-23203c68e6af admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '05b21ffc473e41839197077355739b2d'} {{(pid=87057) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:55:29.375831 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.notifications [None req-e8b0cee5-3aff-4897-91f5-23203c68e6af admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '05b21ffc473e41839197077355739b2d'} {{(pid=87057) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:55:29.405683 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-176d25e0-0063-4678-97c6-2e8edc72c75f admin admin] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:55:29.409482 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e289e0c-68ba-40db-a4e0-9f047f8bda18 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25970b52ca15448fa1abb1f974d64038', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b992d6a65df4bf9b27c13192d57c5f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:55:29.410100 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e289e0c-68ba-40db-a4e0-9f047f8bda18 service cinder] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:55:29.410391 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e289e0c-68ba-40db-a4e0-9f047f8bda18 service cinder] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:55:29.410664 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e289e0c-68ba-40db-a4e0-9f047f8bda18 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:55:29.412939 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.fernet_utils [None req-6e289e0c-68ba-40db-a4e0-9f047f8bda18 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:55:29.449264 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.notifications [None req-e8b0cee5-3aff-4897-91f5-23203c68e6af admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '05b21ffc473e41839197077355739b2d'} {{(pid=87057) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:55:29.449985 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.notifications [None req-e8b0cee5-3aff-4897-91f5-23203c68e6af admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '05b21ffc473e41839197077355739b2d'} {{(pid=87057) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:55:29.472381 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.notifications [None req-e8b0cee5-3aff-4897-91f5-23203c68e6af admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '05b21ffc473e41839197077355739b2d'} {{(pid=87057) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:55:29.509398 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 904/1825] 10.222.0.20 () {58 vars in 1224 bytes} [Mon Aug 1 16:55:29 2022] DELETE /identity/v3/users/05b21ffc473e41839197077355739b2d => generated 0 bytes in 429 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 16:55:29.510800 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b0cee5-3aff-4897-91f5-23203c68e6af admin admin] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:55:29.514786 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.fernet_utils [None req-e8b0cee5-3aff-4897-91f5-23203c68e6af admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87057) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:55:29.528097 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e289e0c-68ba-40db-a4e0-9f047f8bda18 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:55:29.531446 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e289e0c-68ba-40db-a4e0-9f047f8bda18 service cinder] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:55:29.544740 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 922/1826] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 1 16:55:29 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 236 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 16:55:29.546165 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e289e0c-68ba-40db-a4e0-9f047f8bda18 service cinder] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:55:29.661147 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b0cee5-3aff-4897-91f5-23203c68e6af admin admin] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:55:29.666679 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG dbcounter [-] [87057] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=87057) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:55:29.669829 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e289e0c-68ba-40db-a4e0-9f047f8bda18 service cinder] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:55:29.675906 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a03b4182-3990-4a5b-bfa1-c24cbd21061c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:55:29.676635 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03b4182-3990-4a5b-bfa1-c24cbd21061c admin admin] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:55:29.676749 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7bc7119-0bbf-4a9d-9601-773ea0d9553b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:55:29.676918 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03b4182-3990-4a5b-bfa1-c24cbd21061c admin admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:55:29.677242 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03b4182-3990-4a5b-bfa1-c24cbd21061c admin admin] PATH_INFO: `/v3/roles` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:55:29.677794 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7bc7119-0bbf-4a9d-9601-773ea0d9553b admin admin] REQUEST_METHOD: `PUT` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:55:29.678005 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a03b4182-3990-4a5b-bfa1-c24cbd21061c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:55:29.678299 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7bc7119-0bbf-4a9d-9601-773ea0d9553b admin admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:55:29.678679 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7bc7119-0bbf-4a9d-9601-773ea0d9553b admin admin] PATH_INFO: `/v3/projects/3fa4576551874b1692f62cc8d1b16704/users/22872d2368ae4407ba6230dede22fb25/roles/440bb943dd5b46fea038b903b6eaaf81` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:55:29.679812 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a03b4182-3990-4a5b-bfa1-c24cbd21061c admin admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:55:29.693531 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 923/1827] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Aug 1 16:55:29 2022] GET /identity/v3/roles => generated 1420 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 16:55:29.694473 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a03b4182-3990-4a5b-bfa1-c24cbd21061c admin admin] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:55:29.696680 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.fernet_utils [None req-a03b4182-3990-4a5b-bfa1-c24cbd21061c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:55:29.701932 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7bc7119-0bbf-4a9d-9601-773ea0d9553b admin admin] RBAC: Authorizing `identity:create_grant(project_id=3fa4576551874b1692f62cc8d1b16704, user_id=22872d2368ae4407ba6230dede22fb25, role_id=440bb943dd5b46fea038b903b6eaaf81)` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:55:29.712453 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a03b4182-3990-4a5b-bfa1-c24cbd21061c admin admin] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:55:29.717541 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7bc7119-0bbf-4a9d-9601-773ea0d9553b admin admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:55:29.719101 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db9712d5-c93a-4b7f-8586-8432b07e98a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:55:29.719758 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9712d5-c93a-4b7f-8586-8432b07e98a6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:55:29.720039 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9712d5-c93a-4b7f-8586-8432b07e98a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:55:29.720485 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9712d5-c93a-4b7f-8586-8432b07e98a6 admin admin] PATH_INFO: `/v3/projects/2a8c40feb1fb49edb84665134d4374e0` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:55:29.721984 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db9712d5-c93a-4b7f-8586-8432b07e98a6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2a8c40feb1fb49edb84665134d4374e0)` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:55:29.724730 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db9712d5-c93a-4b7f-8586-8432b07e98a6 admin admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:55:29.798448 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 905/1828] 10.222.0.20 () {60 vars in 1483 bytes} [Mon Aug 1 16:55:29 2022] PUT /identity/v3/projects/3fa4576551874b1692f62cc8d1b16704/users/22872d2368ae4407ba6230dede22fb25/roles/440bb943dd5b46fea038b903b6eaaf81 => generated 0 bytes in 288 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 16:55:29.799688 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a236df46-62e4-4dea-b584-5cec566de089 None None] REQUEST_METHOD: `POST` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:55:29.799983 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a236df46-62e4-4dea-b584-5cec566de089 None None] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:55:29.800258 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a236df46-62e4-4dea-b584-5cec566de089 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:55:29.822088 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.auth.core [None req-a236df46-62e4-4dea-b584-5cec566de089 None None] MFA Rules not processed for user `1007f8e379b74e169b98fb7c3c0ba5b4`. Rule list: `[]` (Enabled: `True`). {{(pid=87057) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:55:29.824101 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.fernet_utils [None req-a236df46-62e4-4dea-b584-5cec566de089 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87057) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:55:29.872323 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.notifications [None req-db9712d5-c93a-4b7f-8586-8432b07e98a6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2a8c40feb1fb49edb84665134d4374e0', 'request_id': 'req-db9712d5-c93a-4b7f-8586-8432b07e98a6'} {{(pid=87058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:55:29.878292 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG dbcounter [-] [87058] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=87058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:55:29.884313 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.notifications [None req-db9712d5-c93a-4b7f-8586-8432b07e98a6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2a8c40feb1fb49edb84665134d4374e0', 'request_id': 'req-db9712d5-c93a-4b7f-8586-8432b07e98a6'} {{(pid=87058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:55:29.891993 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.notifications [None req-db9712d5-c93a-4b7f-8586-8432b07e98a6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2a8c40feb1fb49edb84665134d4374e0', 'request_id': 'req-db9712d5-c93a-4b7f-8586-8432b07e98a6'} {{(pid=87058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:55:29.904919 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 906/1829] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 1 16:55:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:55:29.910888 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a236df46-62e4-4dea-b584-5cec566de089 None None] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:55:29.910888 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.fernet_utils [None req-a236df46-62e4-4dea-b584-5cec566de089 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87057) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:55:29.923332 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a236df46-62e4-4dea-b584-5cec566de089 None None] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:55:29.991669 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a0cb33a-7e3f-4aeb-9ef7-ddca3178d764 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:55:29.992370 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0cb33a-7e3f-4aeb-9ef7-ddca3178d764 admin admin] REQUEST_METHOD: `PUT` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:55:29.992654 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0cb33a-7e3f-4aeb-9ef7-ddca3178d764 admin admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:55:29.992948 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0cb33a-7e3f-4aeb-9ef7-ddca3178d764 admin admin] PATH_INFO: `/v3/projects/dc905974d67c4366b4a6a91c43706466/users/20b77010606549cb8f5d23dad50f16d9/roles/440bb943dd5b46fea038b903b6eaaf81` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:55:29.996940 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a0cb33a-7e3f-4aeb-9ef7-ddca3178d764 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dc905974d67c4366b4a6a91c43706466, user_id=20b77010606549cb8f5d23dad50f16d9, role_id=440bb943dd5b46fea038b903b6eaaf81)` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:55:30.006425 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a0cb33a-7e3f-4aeb-9ef7-ddca3178d764 admin admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:55:30.007348 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.notifications [None req-db9712d5-c93a-4b7f-8586-8432b07e98a6 admin admin] The token cache is being invalidate because project 2a8c40feb1fb49edb84665134d4374e0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87058) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 16:55:30.007637 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.notifications [None req-db9712d5-c93a-4b7f-8586-8432b07e98a6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:55:30.010455 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.notifications [None req-db9712d5-c93a-4b7f-8586-8432b07e98a6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:55:30.012408 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 924/1830] 10.222.0.20 () {58 vars in 1233 bytes} [Mon Aug 1 16:55:29 2022] DELETE /identity/v3/projects/2a8c40feb1fb49edb84665134d4374e0 => generated 0 bytes in 318 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 16:55:30.012971 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db9712d5-c93a-4b7f-8586-8432b07e98a6 admin admin] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:55:30.014405 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.fernet_utils [None req-db9712d5-c93a-4b7f-8586-8432b07e98a6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:55:30.035349 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db9712d5-c93a-4b7f-8586-8432b07e98a6 admin admin] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:55:30.036263 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 907/1831] 10.222.0.20 () {60 vars in 1483 bytes} [Mon Aug 1 16:55:29 2022] PUT /identity/v3/projects/dc905974d67c4366b4a6a91c43706466/users/20b77010606549cb8f5d23dad50f16d9/roles/440bb943dd5b46fea038b903b6eaaf81 => generated 0 bytes in 130 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 16:55:30.037180 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a0cb33a-7e3f-4aeb-9ef7-ddca3178d764 admin admin] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:55:30.039917 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.fernet_utils [None req-4a0cb33a-7e3f-4aeb-9ef7-ddca3178d764 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87057) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:55:30.151720 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f45b24df-59e2-45b7-a6be-8de097c6917c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:55:30.152417 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45b24df-59e2-45b7-a6be-8de097c6917c admin admin] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:55:30.152633 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45b24df-59e2-45b7-a6be-8de097c6917c admin admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:55:30.152940 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45b24df-59e2-45b7-a6be-8de097c6917c admin admin] PATH_INFO: `/v3/roles` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:55:30.153654 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f45b24df-59e2-45b7-a6be-8de097c6917c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:55:30.153759 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a0cb33a-7e3f-4aeb-9ef7-ddca3178d764 admin admin] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:55:30.155317 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f45b24df-59e2-45b7-a6be-8de097c6917c admin admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:55:30.159377 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa5dd6fe-52d3-42d9-bc81-213c862ed34a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d9bd8d649154c979be3744bfd016f95', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b992d6a65df4bf9b27c13192d57c5f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:55:30.159377 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5dd6fe-52d3-42d9-bc81-213c862ed34a service neutron] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:55:30.159377 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5dd6fe-52d3-42d9-bc81-213c862ed34a service neutron] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:55:30.159377 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5dd6fe-52d3-42d9-bc81-213c862ed34a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:55:30.161473 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.fernet_utils [None req-fa5dd6fe-52d3-42d9-bc81-213c862ed34a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87057) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:55:30.189187 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 925/1832] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Aug 1 16:55:30 2022] GET /identity/v3/roles => generated 1420 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 16:55:30.198954 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ed3ddd-f7a5-4afe-9424-14f5021ce554 None None] REQUEST_METHOD: `POST` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:55:30.198954 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ed3ddd-f7a5-4afe-9424-14f5021ce554 None None] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:55:30.198954 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ed3ddd-f7a5-4afe-9424-14f5021ce554 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:55:30.208725 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa5dd6fe-52d3-42d9-bc81-213c862ed34a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:55:30.215306 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa5dd6fe-52d3-42d9-bc81-213c862ed34a service neutron] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:55:30.235685 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.auth.core [None req-e4ed3ddd-f7a5-4afe-9424-14f5021ce554 None None] MFA Rules not processed for user `1007f8e379b74e169b98fb7c3c0ba5b4`. Rule list: `[]` (Enabled: `True`). {{(pid=87058) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:55:30.237522 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.fernet_utils [None req-e4ed3ddd-f7a5-4afe-9424-14f5021ce554 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:55:30.251302 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 926/1833] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 1 16:55:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 61 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:55:30.252577 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4ed3ddd-f7a5-4afe-9424-14f5021ce554 None None] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:55:30.256049 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 908/1834] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 1 16:55:30 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 219 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 16:55:30.259351 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf08e51-67a9-43af-8e08-e3b49253b24b None None] REQUEST_METHOD: `POST` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:55:30.265734 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf08e51-67a9-43af-8e08-e3b49253b24b None None] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:55:30.265734 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf08e51-67a9-43af-8e08-e3b49253b24b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:55:30.280605 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4ed3ddd-f7a5-4afe-9424-14f5021ce554 None None] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:55:30.288967 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG dbcounter [-] [87057] Writing DB stats keystone:SELECT=90,keystone:INSERT=2,keystone:DELETE=8 {{(pid=87057) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:55:30.297156 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b424665a-8ac9-4dfa-b2c3-03e96b2cf576 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:55:30.299577 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b424665a-8ac9-4dfa-b2c3-03e96b2cf576 admin admin] REQUEST_METHOD: `PUT` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:55:30.299577 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b424665a-8ac9-4dfa-b2c3-03e96b2cf576 admin admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:55:30.299577 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b424665a-8ac9-4dfa-b2c3-03e96b2cf576 admin admin] PATH_INFO: `/v3/projects/3fa4576551874b1692f62cc8d1b16704/users/22872d2368ae4407ba6230dede22fb25/roles/440bb943dd5b46fea038b903b6eaaf81` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:55:30.302642 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.auth.core [None req-ddf08e51-67a9-43af-8e08-e3b49253b24b None None] MFA Rules not processed for user `1007f8e379b74e169b98fb7c3c0ba5b4`. Rule list: `[]` (Enabled: `True`). {{(pid=87057) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:55:30.303873 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.fernet_utils [None req-ddf08e51-67a9-43af-8e08-e3b49253b24b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87057) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:55:30.311043 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b424665a-8ac9-4dfa-b2c3-03e96b2cf576 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3fa4576551874b1692f62cc8d1b16704, user_id=22872d2368ae4407ba6230dede22fb25, role_id=440bb943dd5b46fea038b903b6eaaf81)` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:55:30.316839 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 909/1835] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 1 16:55:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 56 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:55:30.317205 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b424665a-8ac9-4dfa-b2c3-03e96b2cf576 admin admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:55:30.317983 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddf08e51-67a9-43af-8e08-e3b49253b24b None None] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:55:30.341419 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 927/1836] 10.222.0.20 () {60 vars in 1483 bytes} [Mon Aug 1 16:55:30 2022] PUT /identity/v3/projects/3fa4576551874b1692f62cc8d1b16704/users/22872d2368ae4407ba6230dede22fb25/roles/440bb943dd5b46fea038b903b6eaaf81 => generated 0 bytes in 89 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 16:55:30.347209 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddf08e51-67a9-43af-8e08-e3b49253b24b None None] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:55:30.355610 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548504be-4eed-495b-8f12-ec5d1daef269 None None] REQUEST_METHOD: `POST` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:55:30.355984 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548504be-4eed-495b-8f12-ec5d1daef269 None None] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:55:30.356324 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548504be-4eed-495b-8f12-ec5d1daef269 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:55:30.402821 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.auth.core [None req-548504be-4eed-495b-8f12-ec5d1daef269 None None] MFA Rules not processed for user `22872d2368ae4407ba6230dede22fb25`. Rule list: `[]` (Enabled: `True`). {{(pid=87058) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:55:30.404191 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.fernet_utils [None req-548504be-4eed-495b-8f12-ec5d1daef269 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:55:30.492251 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a50debe2-bab7-4376-be5e-96d679a92b66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:55:30.494008 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50debe2-bab7-4376-be5e-96d679a92b66 admin admin] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:55:30.495115 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50debe2-bab7-4376-be5e-96d679a92b66 admin admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:55:30.495711 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50debe2-bab7-4376-be5e-96d679a92b66 admin admin] PATH_INFO: `/v3/domains` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:55:30.496788 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a50debe2-bab7-4376-be5e-96d679a92b66 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87057) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:55:30.497350 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a50debe2-bab7-4376-be5e-96d679a92b66 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:55:30.499490 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a50debe2-bab7-4376-be5e-96d679a92b66 admin admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:55:30.545409 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 910/1837] 10.222.0.20 () {58 vars in 1153 bytes} [Mon Aug 1 16:55:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:55:30.546800 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a50debe2-bab7-4376-be5e-96d679a92b66 admin admin] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:55:30.548212 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 928/1838] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 1 16:55:30 2022] POST /identity/v3/auth/tokens => generated 3303 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 16:55:30.558294 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a50debe2-bab7-4376-be5e-96d679a92b66 admin admin] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:55:30.561803 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-548504be-4eed-495b-8f12-ec5d1daef269 None None] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:55:30.564135 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c9d8f1c-d516-4fd2-b695-fb49a23047e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:55:30.564515 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.fernet_utils [None req-548504be-4eed-495b-8f12-ec5d1daef269 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:55:30.565148 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9d8f1c-d516-4fd2-b695-fb49a23047e2 admin admin] REQUEST_METHOD: `POST` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:55:30.565619 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9d8f1c-d516-4fd2-b695-fb49a23047e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:55:30.565962 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9d8f1c-d516-4fd2-b695-fb49a23047e2 admin admin] PATH_INFO: `/v3/projects` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:55:30.566862 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c9d8f1c-d516-4fd2-b695-fb49a23047e2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:55:30.568958 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c9d8f1c-d516-4fd2-b695-fb49a23047e2 admin admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:55:30.705262 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-548504be-4eed-495b-8f12-ec5d1daef269 None None] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:55:30.709229 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10df9f41-b41b-4c68-b8b3-824569bd0911 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25970b52ca15448fa1abb1f974d64038', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b992d6a65df4bf9b27c13192d57c5f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:55:30.715194 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10df9f41-b41b-4c68-b8b3-824569bd0911 service cinder] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:55:30.715670 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10df9f41-b41b-4c68-b8b3-824569bd0911 service cinder] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:55:30.720038 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10df9f41-b41b-4c68-b8b3-824569bd0911 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:55:30.723461 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 911/1839] 10.222.0.20 () {60 vars in 1153 bytes} [Mon Aug 1 16:55:30 2022] POST /identity/v3/projects => generated 365 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 16:55:30.729680 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c9d8f1c-d516-4fd2-b695-fb49a23047e2 admin admin] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:55:30.743422 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10df9f41-b41b-4c68-b8b3-824569bd0911 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:55:30.747132 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10df9f41-b41b-4c68-b8b3-824569bd0911 service cinder] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:55:30.773658 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 929/1840] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 1 16:55:30 2022] GET /identity/v3/auth/tokens => generated 3303 bytes in 212 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 01 16:55:30.814301 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG dbcounter [-] [87058] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:55:30.924442 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c9d8f1c-d516-4fd2-b695-fb49a23047e2 admin admin] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:55:30.961295 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10df9f41-b41b-4c68-b8b3-824569bd0911 service cinder] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:55:30.966053 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.fernet_utils [None req-10df9f41-b41b-4c68-b8b3-824569bd0911 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:55:30.976194 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b94a835-002f-4934-ad86-dcae76036e02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:55:30.979182 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b94a835-002f-4934-ad86-dcae76036e02 admin admin] REQUEST_METHOD: `POST` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:55:30.982122 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b94a835-002f-4934-ad86-dcae76036e02 admin admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:55:30.982493 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b94a835-002f-4934-ad86-dcae76036e02 admin admin] PATH_INFO: `/v3/users` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:55:30.983273 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b94a835-002f-4934-ad86-dcae76036e02 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:55:30.983444 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10df9f41-b41b-4c68-b8b3-824569bd0911 service cinder] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:55:30.985739 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b94a835-002f-4934-ad86-dcae76036e02 admin admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:55:31.074444 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-189b43c6-7d3e-42e3-bdba-b5aec0b31820 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25970b52ca15448fa1abb1f974d64038', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b992d6a65df4bf9b27c13192d57c5f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:55:31.075100 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189b43c6-7d3e-42e3-bdba-b5aec0b31820 service cinder] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:55:31.075578 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189b43c6-7d3e-42e3-bdba-b5aec0b31820 service cinder] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:55:31.075874 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189b43c6-7d3e-42e3-bdba-b5aec0b31820 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:55:31.078249 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.fernet_utils [None req-189b43c6-7d3e-42e3-bdba-b5aec0b31820 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:55:31.123138 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 912/1841] 10.222.0.20 () {60 vars in 1144 bytes} [Mon Aug 1 16:55:30 2022] POST /identity/v3/users => generated 283 bytes in 392 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 16:55:31.124700 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b94a835-002f-4934-ad86-dcae76036e02 admin admin] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:55:31.139602 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b94a835-002f-4934-ad86-dcae76036e02 admin admin] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:55:31.145422 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0667cd99-a303-4983-8d9a-aa129865fa79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:55:31.146246 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0667cd99-a303-4983-8d9a-aa129865fa79 admin admin] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:55:31.147560 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0667cd99-a303-4983-8d9a-aa129865fa79 admin admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:55:31.147560 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0667cd99-a303-4983-8d9a-aa129865fa79 admin admin] PATH_INFO: `/v3/roles` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:55:31.147806 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0667cd99-a303-4983-8d9a-aa129865fa79 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:55:31.149781 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0667cd99-a303-4983-8d9a-aa129865fa79 admin admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:55:31.176642 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-189b43c6-7d3e-42e3-bdba-b5aec0b31820 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:55:31.182634 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-189b43c6-7d3e-42e3-bdba-b5aec0b31820 service cinder] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:55:31.205238 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 913/1842] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Aug 1 16:55:31 2022] GET /identity/v3/roles => generated 1420 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 01 16:55:31.205238 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0667cd99-a303-4983-8d9a-aa129865fa79 admin admin] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:55:31.206247 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 930/1843] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 1 16:55:30 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 246 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 16:55:31.208074 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-189b43c6-7d3e-42e3-bdba-b5aec0b31820 service cinder] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:55:31.209548 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.fernet_utils [None req-0667cd99-a303-4983-8d9a-aa129865fa79 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87057) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:55:31.221155 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-189b43c6-7d3e-42e3-bdba-b5aec0b31820 service cinder] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:55:31.237258 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c734f721-9ca1-4e6d-bf1c-07440b49c0ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:55:31.238399 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c734f721-9ca1-4e6d-bf1c-07440b49c0ef admin admin] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:55:31.243467 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c734f721-9ca1-4e6d-bf1c-07440b49c0ef admin admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:55:31.244777 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c734f721-9ca1-4e6d-bf1c-07440b49c0ef admin admin] PATH_INFO: `/v3/roles` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:55:31.245747 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c734f721-9ca1-4e6d-bf1c-07440b49c0ef admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:55:31.247813 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c734f721-9ca1-4e6d-bf1c-07440b49c0ef admin admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:55:31.248821 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0667cd99-a303-4983-8d9a-aa129865fa79 admin admin] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:55:31.277357 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 931/1844] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Aug 1 16:55:31 2022] GET /identity/v3/roles => generated 1420 bytes in 70 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 16:55:31.281931 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c734f721-9ca1-4e6d-bf1c-07440b49c0ef admin admin] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:55:31.287735 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c40aec8c-1e44-4d46-bc70-f3462cc08ec8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:55:31.290269 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40aec8c-1e44-4d46-bc70-f3462cc08ec8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:55:31.290561 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40aec8c-1e44-4d46-bc70-f3462cc08ec8 admin admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:55:31.290952 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40aec8c-1e44-4d46-bc70-f3462cc08ec8 admin admin] PATH_INFO: `/v3/projects/14d7566fedb3472eb7bf7c2d6aef59b8` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:55:31.302618 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c40aec8c-1e44-4d46-bc70-f3462cc08ec8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=14d7566fedb3472eb7bf7c2d6aef59b8)` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:55:31.309010 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c40aec8c-1e44-4d46-bc70-f3462cc08ec8 admin admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:55:31.309504 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c734f721-9ca1-4e6d-bf1c-07440b49c0ef admin admin] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:55:31.322120 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db88786-2b75-4c0f-906f-6d6e68bc0f57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:55:31.323344 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db88786-2b75-4c0f-906f-6d6e68bc0f57 admin admin] REQUEST_METHOD: `PUT` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:55:31.323624 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db88786-2b75-4c0f-906f-6d6e68bc0f57 admin admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:55:31.323959 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db88786-2b75-4c0f-906f-6d6e68bc0f57 admin admin] PATH_INFO: `/v3/projects/7a6768f1a61941aa943ee21041280cb8/users/8377f3912caf4b6db3e939e4daeb80ff/roles/0999d3da63a545f18ff3252afdc90518` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:55:31.385929 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8db88786-2b75-4c0f-906f-6d6e68bc0f57 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7a6768f1a61941aa943ee21041280cb8, user_id=8377f3912caf4b6db3e939e4daeb80ff, role_id=0999d3da63a545f18ff3252afdc90518)` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:55:31.388939 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8db88786-2b75-4c0f-906f-6d6e68bc0f57 admin admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:55:31.490318 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 932/1845] 10.222.0.20 () {60 vars in 1483 bytes} [Mon Aug 1 16:55:31 2022] PUT /identity/v3/projects/7a6768f1a61941aa943ee21041280cb8/users/8377f3912caf4b6db3e939e4daeb80ff/roles/0999d3da63a545f18ff3252afdc90518 => generated 0 bytes in 208 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 16:55:31.502287 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d80605b-f20e-4632-928a-b3d8fe51a160 None None] REQUEST_METHOD: `POST` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:55:31.502583 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d80605b-f20e-4632-928a-b3d8fe51a160 None None] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:55:31.502868 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d80605b-f20e-4632-928a-b3d8fe51a160 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:55:31.515434 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.notifications [None req-c40aec8c-1e44-4d46-bc70-f3462cc08ec8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '14d7566fedb3472eb7bf7c2d6aef59b8', 'request_id': 'req-c40aec8c-1e44-4d46-bc70-f3462cc08ec8'} {{(pid=87057) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:55:31.532457 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.notifications [None req-c40aec8c-1e44-4d46-bc70-f3462cc08ec8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '14d7566fedb3472eb7bf7c2d6aef59b8', 'request_id': 'req-c40aec8c-1e44-4d46-bc70-f3462cc08ec8'} {{(pid=87057) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:55:31.545154 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.notifications [None req-c40aec8c-1e44-4d46-bc70-f3462cc08ec8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '14d7566fedb3472eb7bf7c2d6aef59b8', 'request_id': 'req-c40aec8c-1e44-4d46-bc70-f3462cc08ec8'} {{(pid=87057) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:55:31.545812 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.auth.core [None req-6d80605b-f20e-4632-928a-b3d8fe51a160 None None] MFA Rules not processed for user `8377f3912caf4b6db3e939e4daeb80ff`. Rule list: `[]` (Enabled: `True`). {{(pid=87058) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:55:31.547173 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.fernet_utils [None req-6d80605b-f20e-4632-928a-b3d8fe51a160 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:55:31.636933 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.notifications [None req-c40aec8c-1e44-4d46-bc70-f3462cc08ec8 admin admin] The token cache is being invalidate because project 14d7566fedb3472eb7bf7c2d6aef59b8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87057) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 16:55:31.637655 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.notifications [None req-c40aec8c-1e44-4d46-bc70-f3462cc08ec8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87057) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:55:31.638362 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.notifications [None req-c40aec8c-1e44-4d46-bc70-f3462cc08ec8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87057) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:55:31.642206 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 914/1846] 10.222.0.20 () {58 vars in 1233 bytes} [Mon Aug 1 16:55:31 2022] DELETE /identity/v3/projects/14d7566fedb3472eb7bf7c2d6aef59b8 => generated 0 bytes in 437 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 16:55:31.644013 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c40aec8c-1e44-4d46-bc70-f3462cc08ec8 admin admin] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:55:31.648186 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.fernet_utils [None req-c40aec8c-1e44-4d46-bc70-f3462cc08ec8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87057) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:55:31.714367 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 933/1847] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 1 16:55:31 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:55:31.716254 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d80605b-f20e-4632-928a-b3d8fe51a160 None None] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:55:31.718580 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.fernet_utils [None req-6d80605b-f20e-4632-928a-b3d8fe51a160 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:55:31.771146 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c40aec8c-1e44-4d46-bc70-f3462cc08ec8 admin admin] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:55:31.781362 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b655609-d0af-4906-bcfc-5154ce4ffd7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:55:31.782224 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b655609-d0af-4906-bcfc-5154ce4ffd7b admin admin] REQUEST_METHOD: `DELETE` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:55:31.782560 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b655609-d0af-4906-bcfc-5154ce4ffd7b admin admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:55:31.782864 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b655609-d0af-4906-bcfc-5154ce4ffd7b admin admin] PATH_INFO: `/v3/users/9f0da3bf8fd64382907da9412ddc3b37` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:55:31.785024 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b655609-d0af-4906-bcfc-5154ce4ffd7b admin admin] RBAC: Authorizing `identity:delete_user(user_id=9f0da3bf8fd64382907da9412ddc3b37)` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:55:31.787237 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b655609-d0af-4906-bcfc-5154ce4ffd7b admin admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:55:31.832541 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d80605b-f20e-4632-928a-b3d8fe51a160 None None] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:55:31.837668 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d273133d-473e-4f70-a116-0fa311be74b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:55:31.838816 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d273133d-473e-4f70-a116-0fa311be74b7 admin admin] REQUEST_METHOD: `POST` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:55:31.839274 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d273133d-473e-4f70-a116-0fa311be74b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:55:31.839702 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d273133d-473e-4f70-a116-0fa311be74b7 admin admin] PATH_INFO: `/v3/projects` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:55:31.841966 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d273133d-473e-4f70-a116-0fa311be74b7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:55:31.844098 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d273133d-473e-4f70-a116-0fa311be74b7 admin admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:55:31.954575 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 934/1848] 10.222.0.20 () {60 vars in 1153 bytes} [Mon Aug 1 16:55:31 2022] POST /identity/v3/projects => generated 365 bytes in 240 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 16:55:31.959380 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d273133d-473e-4f70-a116-0fa311be74b7 admin admin] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:55:31.985792 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.notifications [None req-2b655609-d0af-4906-bcfc-5154ce4ffd7b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9f0da3bf8fd64382907da9412ddc3b37'} {{(pid=87057) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:55:31.997737 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.notifications [None req-2b655609-d0af-4906-bcfc-5154ce4ffd7b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9f0da3bf8fd64382907da9412ddc3b37'} {{(pid=87057) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:55:32.037282 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d273133d-473e-4f70-a116-0fa311be74b7 admin admin] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:55:32.052134 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37729b1d-ee82-4ac6-81be-19e151b29ca4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:55:32.052803 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37729b1d-ee82-4ac6-81be-19e151b29ca4 admin admin] REQUEST_METHOD: `POST` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:55:32.053074 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37729b1d-ee82-4ac6-81be-19e151b29ca4 admin admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:55:32.053350 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37729b1d-ee82-4ac6-81be-19e151b29ca4 admin admin] PATH_INFO: `/v3/users` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:55:32.054074 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37729b1d-ee82-4ac6-81be-19e151b29ca4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:55:32.056162 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37729b1d-ee82-4ac6-81be-19e151b29ca4 admin admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:55:32.077468 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.notifications [None req-2b655609-d0af-4906-bcfc-5154ce4ffd7b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9f0da3bf8fd64382907da9412ddc3b37'} {{(pid=87057) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:55:32.078161 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.notifications [None req-2b655609-d0af-4906-bcfc-5154ce4ffd7b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9f0da3bf8fd64382907da9412ddc3b37'} {{(pid=87057) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:55:32.086532 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.notifications [None req-2b655609-d0af-4906-bcfc-5154ce4ffd7b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9f0da3bf8fd64382907da9412ddc3b37'} {{(pid=87057) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:55:32.105825 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 915/1849] 10.222.0.20 () {58 vars in 1224 bytes} [Mon Aug 1 16:55:31 2022] DELETE /identity/v3/users/9f0da3bf8fd64382907da9412ddc3b37 => generated 0 bytes in 462 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 16:55:32.174476 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 935/1850] 10.222.0.20 () {60 vars in 1144 bytes} [Mon Aug 1 16:55:31 2022] POST /identity/v3/users => generated 289 bytes in 215 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 16:55:32.175221 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b655609-d0af-4906-bcfc-5154ce4ffd7b admin admin] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:55:32.177131 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.fernet_utils [None req-2b655609-d0af-4906-bcfc-5154ce4ffd7b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87057) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:55:32.246735 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b655609-d0af-4906-bcfc-5154ce4ffd7b admin admin] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:55:32.251743 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff8613ba-3d63-4363-837d-5c6055c29410 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:55:32.252373 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8613ba-3d63-4363-837d-5c6055c29410 admin admin] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:55:32.252653 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8613ba-3d63-4363-837d-5c6055c29410 admin admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:55:32.252927 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8613ba-3d63-4363-837d-5c6055c29410 admin admin] PATH_INFO: `/v3/roles` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:55:32.258165 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff8613ba-3d63-4363-837d-5c6055c29410 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:55:32.258165 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff8613ba-3d63-4363-837d-5c6055c29410 admin admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:55:32.265102 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 916/1851] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Aug 1 16:55:32 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 16:55:32.269546 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37729b1d-ee82-4ac6-81be-19e151b29ca4 admin admin] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:55:32.280754 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37729b1d-ee82-4ac6-81be-19e151b29ca4 admin admin] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:55:32.288227 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-915fe078-8491-443d-8b3d-b0ffc29d1c59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:55:32.289055 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915fe078-8491-443d-8b3d-b0ffc29d1c59 admin admin] REQUEST_METHOD: `PUT` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:55:32.289337 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915fe078-8491-443d-8b3d-b0ffc29d1c59 admin admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:55:32.289609 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915fe078-8491-443d-8b3d-b0ffc29d1c59 admin admin] PATH_INFO: `/v3/projects/a9d4ce10ed224fe2b77fb79e6338aef8/users/d6faaa1f4f66409e813e677f4d4fa79a/roles/440bb943dd5b46fea038b903b6eaaf81` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:55:32.311425 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-915fe078-8491-443d-8b3d-b0ffc29d1c59 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a9d4ce10ed224fe2b77fb79e6338aef8, user_id=d6faaa1f4f66409e813e677f4d4fa79a, role_id=440bb943dd5b46fea038b903b6eaaf81)` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:55:32.314114 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-915fe078-8491-443d-8b3d-b0ffc29d1c59 admin admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:55:32.410070 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 936/1852] 10.222.0.20 () {60 vars in 1483 bytes} [Mon Aug 1 16:55:32 2022] PUT /identity/v3/projects/a9d4ce10ed224fe2b77fb79e6338aef8/users/d6faaa1f4f66409e813e677f4d4fa79a/roles/440bb943dd5b46fea038b903b6eaaf81 => generated 0 bytes in 141 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 16:55:32.414069 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff8613ba-3d63-4363-837d-5c6055c29410 admin admin] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:55:32.418778 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa5f3ec-3b01-4bfc-bf76-dee99285aac0 None None] REQUEST_METHOD: `POST` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:55:32.419091 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa5f3ec-3b01-4bfc-bf76-dee99285aac0 None None] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:55:32.419333 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa5f3ec-3b01-4bfc-bf76-dee99285aac0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:55:32.433360 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff8613ba-3d63-4363-837d-5c6055c29410 admin admin] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:55:32.453450 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.auth.core [None req-3aa5f3ec-3b01-4bfc-bf76-dee99285aac0 None None] MFA Rules not processed for user `20b77010606549cb8f5d23dad50f16d9`. Rule list: `[]` (Enabled: `True`). {{(pid=87058) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:55:32.454806 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.fernet_utils [None req-3aa5f3ec-3b01-4bfc-bf76-dee99285aac0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:55:32.518127 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65aa97dc-3e87-4e79-95ff-86608678fe79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:55:32.519516 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65aa97dc-3e87-4e79-95ff-86608678fe79 admin admin] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:55:32.519516 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65aa97dc-3e87-4e79-95ff-86608678fe79 admin admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:55:32.519516 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65aa97dc-3e87-4e79-95ff-86608678fe79 admin admin] PATH_INFO: `/v3/roles` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:55:32.520410 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65aa97dc-3e87-4e79-95ff-86608678fe79 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:55:32.521898 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65aa97dc-3e87-4e79-95ff-86608678fe79 admin admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:55:32.529601 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG dbcounter [-] [87058] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=87058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:55:32.545447 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 917/1853] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Aug 1 16:55:32 2022] GET /identity/v3/roles => generated 1420 bytes in 132 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 16:55:32.546719 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65aa97dc-3e87-4e79-95ff-86608678fe79 admin admin] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:55:32.559158 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65aa97dc-3e87-4e79-95ff-86608678fe79 admin admin] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:55:32.567519 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-632b3e15-71fc-4fc3-af01-cbc504c59501 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:55:32.568201 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632b3e15-71fc-4fc3-af01-cbc504c59501 admin admin] REQUEST_METHOD: `PUT` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:55:32.568483 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632b3e15-71fc-4fc3-af01-cbc504c59501 admin admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:55:32.568753 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632b3e15-71fc-4fc3-af01-cbc504c59501 admin admin] PATH_INFO: `/v3/projects/a9d4ce10ed224fe2b77fb79e6338aef8/users/d6faaa1f4f66409e813e677f4d4fa79a/roles/440bb943dd5b46fea038b903b6eaaf81` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:55:32.570992 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 937/1854] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 1 16:55:32 2022] POST /identity/v3/auth/tokens => generated 3305 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 16:55:32.580438 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-632b3e15-71fc-4fc3-af01-cbc504c59501 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a9d4ce10ed224fe2b77fb79e6338aef8, user_id=d6faaa1f4f66409e813e677f4d4fa79a, role_id=440bb943dd5b46fea038b903b6eaaf81)` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:55:32.585011 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-632b3e15-71fc-4fc3-af01-cbc504c59501 admin admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:55:32.586087 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa5f3ec-3b01-4bfc-bf76-dee99285aac0 None None] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:55:32.587577 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.fernet_utils [None req-3aa5f3ec-3b01-4bfc-bf76-dee99285aac0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:55:32.604339 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 918/1855] 10.222.0.20 () {60 vars in 1483 bytes} [Mon Aug 1 16:55:32 2022] PUT /identity/v3/projects/a9d4ce10ed224fe2b77fb79e6338aef8/users/d6faaa1f4f66409e813e677f4d4fa79a/roles/440bb943dd5b46fea038b903b6eaaf81 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 16:55:32.616075 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5edd43d-d1a9-4e5f-baaf-f12de512f2c0 None None] REQUEST_METHOD: `POST` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:55:32.616510 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5edd43d-d1a9-4e5f-baaf-f12de512f2c0 None None] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:55:32.616791 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5edd43d-d1a9-4e5f-baaf-f12de512f2c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:55:32.674660 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.auth.core [None req-d5edd43d-d1a9-4e5f-baaf-f12de512f2c0 None None] MFA Rules not processed for user `d6faaa1f4f66409e813e677f4d4fa79a`. Rule list: `[]` (Enabled: `True`). {{(pid=87057) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:55:32.674660 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.fernet_utils [None req-d5edd43d-d1a9-4e5f-baaf-f12de512f2c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87057) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:55:32.675144 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa5f3ec-3b01-4bfc-bf76-dee99285aac0 None None] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:55:32.694709 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG dbcounter [-] [87057] Writing DB stats keystone:SELECT=84,keystone:DELETE=12,keystone:INSERT=4 {{(pid=87057) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:55:32.725347 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3de739b-07e2-484b-9d0f-bf49fc5699d4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25970b52ca15448fa1abb1f974d64038', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b992d6a65df4bf9b27c13192d57c5f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:55:32.726140 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3de739b-07e2-484b-9d0f-bf49fc5699d4 service cinder] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:55:32.726552 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3de739b-07e2-484b-9d0f-bf49fc5699d4 service cinder] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:55:32.726829 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3de739b-07e2-484b-9d0f-bf49fc5699d4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:55:32.729305 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.fernet_utils [None req-a3de739b-07e2-484b-9d0f-bf49fc5699d4 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:55:32.749379 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 919/1856] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 1 16:55:32 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 16:55:32.749379 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5edd43d-d1a9-4e5f-baaf-f12de512f2c0 None None] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:55:32.749379 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.fernet_utils [None req-d5edd43d-d1a9-4e5f-baaf-f12de512f2c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87057) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:55:32.791462 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3de739b-07e2-484b-9d0f-bf49fc5699d4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:55:32.792916 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3de739b-07e2-484b-9d0f-bf49fc5699d4 service cinder] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:55:32.803348 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 938/1857] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 1 16:55:32 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 218 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 16:55:32.804231 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3de739b-07e2-484b-9d0f-bf49fc5699d4 service cinder] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:55:32.805601 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.fernet_utils [None req-a3de739b-07e2-484b-9d0f-bf49fc5699d4 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:55:32.817005 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3de739b-07e2-484b-9d0f-bf49fc5699d4 service cinder] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:55:32.820060 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5edd43d-d1a9-4e5f-baaf-f12de512f2c0 None None] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:55:32.821153 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-886e0722-0200-4cfe-9b6c-d524274c46c8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25970b52ca15448fa1abb1f974d64038', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b992d6a65df4bf9b27c13192d57c5f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:55:32.821856 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886e0722-0200-4cfe-9b6c-d524274c46c8 service cinder] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:55:32.822756 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886e0722-0200-4cfe-9b6c-d524274c46c8 service cinder] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:55:32.823029 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886e0722-0200-4cfe-9b6c-d524274c46c8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:55:32.826739 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9e250e9-343a-4b64-ade4-3265ac53d91e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:55:32.827506 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e250e9-343a-4b64-ade4-3265ac53d91e admin admin] REQUEST_METHOD: `DELETE` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:55:32.827864 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e250e9-343a-4b64-ade4-3265ac53d91e admin admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:55:32.828236 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e250e9-343a-4b64-ade4-3265ac53d91e admin admin] PATH_INFO: `/v3/projects/b2493237ab69425787f161b81b26407a` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:55:32.829768 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9e250e9-343a-4b64-ade4-3265ac53d91e admin admin] RBAC: Authorizing `identity:delete_project(project_id=b2493237ab69425787f161b81b26407a)` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:55:32.831850 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9e250e9-343a-4b64-ade4-3265ac53d91e admin admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:55:32.833346 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-886e0722-0200-4cfe-9b6c-d524274c46c8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:55:32.834826 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-886e0722-0200-4cfe-9b6c-d524274c46c8 service cinder] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:55:32.846450 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 939/1858] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 1 16:55:32 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 16:55:32.954013 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.notifications [None req-a9e250e9-343a-4b64-ade4-3265ac53d91e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b2493237ab69425787f161b81b26407a', 'request_id': 'req-a9e250e9-343a-4b64-ade4-3265ac53d91e'} {{(pid=87057) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:55:32.963543 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.notifications [None req-a9e250e9-343a-4b64-ade4-3265ac53d91e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b2493237ab69425787f161b81b26407a', 'request_id': 'req-a9e250e9-343a-4b64-ade4-3265ac53d91e'} {{(pid=87057) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:55:32.970446 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.notifications [None req-a9e250e9-343a-4b64-ade4-3265ac53d91e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b2493237ab69425787f161b81b26407a', 'request_id': 'req-a9e250e9-343a-4b64-ade4-3265ac53d91e'} {{(pid=87057) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:55:33.061173 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.notifications [None req-a9e250e9-343a-4b64-ade4-3265ac53d91e admin admin] The token cache is being invalidate because project b2493237ab69425787f161b81b26407a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87057) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 16:55:33.061689 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.notifications [None req-a9e250e9-343a-4b64-ade4-3265ac53d91e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87057) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:55:33.062510 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.notifications [None req-a9e250e9-343a-4b64-ade4-3265ac53d91e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87057) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:55:33.064384 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 920/1859] 10.222.0.20 () {58 vars in 1233 bytes} [Mon Aug 1 16:55:32 2022] DELETE /identity/v3/projects/b2493237ab69425787f161b81b26407a => generated 0 bytes in 318 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 16:55:33.069418 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f370b07-2672-4a71-bee4-2b5fa6be92fd None None] REQUEST_METHOD: `POST` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:55:33.069882 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f370b07-2672-4a71-bee4-2b5fa6be92fd None None] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:55:33.072417 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f370b07-2672-4a71-bee4-2b5fa6be92fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:55:33.096959 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9e250e9-343a-4b64-ade4-3265ac53d91e admin admin] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:55:33.098332 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.auth.core [None req-0f370b07-2672-4a71-bee4-2b5fa6be92fd None None] MFA Rules not processed for user `1007f8e379b74e169b98fb7c3c0ba5b4`. Rule list: `[]` (Enabled: `True`). {{(pid=87058) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:55:33.098732 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.fernet_utils [None req-a9e250e9-343a-4b64-ade4-3265ac53d91e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87057) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:55:33.099865 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.fernet_utils [None req-0f370b07-2672-4a71-bee4-2b5fa6be92fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:55:33.181433 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9e250e9-343a-4b64-ade4-3265ac53d91e admin admin] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:55:33.185211 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b3fa615-53de-4309-a821-0422f04cbe3c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25970b52ca15448fa1abb1f974d64038', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b992d6a65df4bf9b27c13192d57c5f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:55:33.185888 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3fa615-53de-4309-a821-0422f04cbe3c service cinder] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:55:33.186202 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3fa615-53de-4309-a821-0422f04cbe3c service cinder] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:55:33.186472 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3fa615-53de-4309-a821-0422f04cbe3c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:55:33.188225 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.fernet_utils [None req-3b3fa615-53de-4309-a821-0422f04cbe3c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87057) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:55:33.195048 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 940/1860] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 1 16:55:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:55:33.205198 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732ffd75-84a6-45d8-ac72-95c50b891be5 None None] REQUEST_METHOD: `POST` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:55:33.205563 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732ffd75-84a6-45d8-ac72-95c50b891be5 None None] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:55:33.205959 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732ffd75-84a6-45d8-ac72-95c50b891be5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:55:33.226411 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.auth.core [None req-732ffd75-84a6-45d8-ac72-95c50b891be5 None None] MFA Rules not processed for user `1007f8e379b74e169b98fb7c3c0ba5b4`. Rule list: `[]` (Enabled: `True`). {{(pid=87058) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:55:33.227765 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.fernet_utils [None req-732ffd75-84a6-45d8-ac72-95c50b891be5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:55:33.236600 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 941/1861] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 1 16:55:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 16:55:33.240646 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-732ffd75-84a6-45d8-ac72-95c50b891be5 None None] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:55:33.249401 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b3fa615-53de-4309-a821-0422f04cbe3c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:55:33.250909 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-732ffd75-84a6-45d8-ac72-95c50b891be5 None None] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:55:33.251069 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b3fa615-53de-4309-a821-0422f04cbe3c service cinder] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:55:33.257046 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d6c074-f53c-4f56-b065-2c592553f832 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:55:33.257527 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d6c074-f53c-4f56-b065-2c592553f832 admin admin] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:55:33.258073 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d6c074-f53c-4f56-b065-2c592553f832 admin admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:55:33.258073 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d6c074-f53c-4f56-b065-2c592553f832 admin admin] PATH_INFO: `/v3/domains` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:55:33.258769 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82d6c074-f53c-4f56-b065-2c592553f832 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87058) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:55:33.259229 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82d6c074-f53c-4f56-b065-2c592553f832 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:55:33.261468 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82d6c074-f53c-4f56-b065-2c592553f832 admin admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:55:33.262110 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 921/1862] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 1 16:55:33 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 166 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 01 16:55:33.274107 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 942/1863] 10.222.0.20 () {58 vars in 1153 bytes} [Mon Aug 1 16:55:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:55:33.279216 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b3fa615-53de-4309-a821-0422f04cbe3c service cinder] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:55:33.290419 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b3fa615-53de-4309-a821-0422f04cbe3c service cinder] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:55:33.295369 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75875141-6b88-4a9e-af5c-f73785e76615 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:55:33.296626 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75875141-6b88-4a9e-af5c-f73785e76615 admin admin] REQUEST_METHOD: `POST` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:55:33.296626 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75875141-6b88-4a9e-af5c-f73785e76615 admin admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:55:33.296859 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75875141-6b88-4a9e-af5c-f73785e76615 admin admin] PATH_INFO: `/v3/projects` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:55:33.297497 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75875141-6b88-4a9e-af5c-f73785e76615 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:55:33.300669 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75875141-6b88-4a9e-af5c-f73785e76615 admin admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:55:33.387137 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 922/1864] 10.222.0.20 () {60 vars in 1153 bytes} [Mon Aug 1 16:55:33 2022] POST /identity/v3/projects => generated 387 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 16:55:33.390957 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d6c074-f53c-4f56-b065-2c592553f832 admin admin] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:55:33.403805 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d6c074-f53c-4f56-b065-2c592553f832 admin admin] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:55:33.469068 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78811cbd-c408-4c4d-a11d-271f3b6f662c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:55:33.469689 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78811cbd-c408-4c4d-a11d-271f3b6f662c admin admin] REQUEST_METHOD: `POST` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:55:33.470767 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78811cbd-c408-4c4d-a11d-271f3b6f662c admin admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:55:33.470767 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78811cbd-c408-4c4d-a11d-271f3b6f662c admin admin] PATH_INFO: `/v3/users` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:55:33.470977 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78811cbd-c408-4c4d-a11d-271f3b6f662c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:55:33.472882 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78811cbd-c408-4c4d-a11d-271f3b6f662c admin admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:55:33.550833 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 943/1865] 10.222.0.20 () {60 vars in 1144 bytes} [Mon Aug 1 16:55:33 2022] POST /identity/v3/users => generated 294 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 16:55:33.554947 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75875141-6b88-4a9e-af5c-f73785e76615 admin admin] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:55:33.566241 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75875141-6b88-4a9e-af5c-f73785e76615 admin admin] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:55:33.570820 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba44c92-76af-4c3b-8d0b-0f1332264f2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:55:33.571417 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba44c92-76af-4c3b-8d0b-0f1332264f2d admin admin] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:55:33.571698 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba44c92-76af-4c3b-8d0b-0f1332264f2d admin admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:55:33.572071 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba44c92-76af-4c3b-8d0b-0f1332264f2d admin admin] PATH_INFO: `/v3/roles` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:55:33.573283 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ba44c92-76af-4c3b-8d0b-0f1332264f2d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:55:33.574931 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ba44c92-76af-4c3b-8d0b-0f1332264f2d admin admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:55:33.585367 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 923/1866] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Aug 1 16:55:33 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 16:55:33.589485 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78811cbd-c408-4c4d-a11d-271f3b6f662c admin admin] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:55:33.599658 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78811cbd-c408-4c4d-a11d-271f3b6f662c admin admin] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:55:33.604471 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-102942d6-cda4-4beb-9237-9daa7030e7a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:55:33.605077 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102942d6-cda4-4beb-9237-9daa7030e7a4 admin admin] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:55:33.605356 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102942d6-cda4-4beb-9237-9daa7030e7a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:55:33.605627 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102942d6-cda4-4beb-9237-9daa7030e7a4 admin admin] PATH_INFO: `/v3/roles` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:55:33.606358 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-102942d6-cda4-4beb-9237-9daa7030e7a4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:55:33.607986 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-102942d6-cda4-4beb-9237-9daa7030e7a4 admin admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:55:33.619375 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 944/1867] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Aug 1 16:55:33 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 16:55:33.623638 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba44c92-76af-4c3b-8d0b-0f1332264f2d admin admin] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:55:33.633745 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba44c92-76af-4c3b-8d0b-0f1332264f2d admin admin] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:55:33.641001 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a517ce-5931-4f92-8a8c-9051122a26fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:55:33.642210 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a517ce-5931-4f92-8a8c-9051122a26fe admin admin] REQUEST_METHOD: `PUT` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:55:33.642637 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a517ce-5931-4f92-8a8c-9051122a26fe admin admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:55:33.643080 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a517ce-5931-4f92-8a8c-9051122a26fe admin admin] PATH_INFO: `/v3/projects/d5bb252fc4584ab6b705d705a20a5e72/users/49c4981ff1bb41f88920b9c465997804/roles/0999d3da63a545f18ff3252afdc90518` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:55:33.669090 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54a517ce-5931-4f92-8a8c-9051122a26fe admin admin] RBAC: Authorizing `identity:create_grant(project_id=d5bb252fc4584ab6b705d705a20a5e72, user_id=49c4981ff1bb41f88920b9c465997804, role_id=0999d3da63a545f18ff3252afdc90518)` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:55:33.675815 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54a517ce-5931-4f92-8a8c-9051122a26fe admin admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:55:33.774770 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 924/1868] 10.222.0.20 () {60 vars in 1483 bytes} [Mon Aug 1 16:55:33 2022] PUT /identity/v3/projects/d5bb252fc4584ab6b705d705a20a5e72/users/49c4981ff1bb41f88920b9c465997804/roles/0999d3da63a545f18ff3252afdc90518 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 16:55:33.789716 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab607f4-a416-4cf5-bd17-645fffdf0203 None None] REQUEST_METHOD: `POST` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:55:33.790154 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab607f4-a416-4cf5-bd17-645fffdf0203 None None] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:55:33.790295 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab607f4-a416-4cf5-bd17-645fffdf0203 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:55:33.823855 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.auth.core [None req-6ab607f4-a416-4cf5-bd17-645fffdf0203 None None] MFA Rules not processed for user `49c4981ff1bb41f88920b9c465997804`. Rule list: `[]` (Enabled: `True`). {{(pid=87058) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:55:33.825380 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.fernet_utils [None req-6ab607f4-a416-4cf5-bd17-645fffdf0203 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:55:33.893115 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 945/1869] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 1 16:55:33 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:55:33.897920 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a517ce-5931-4f92-8a8c-9051122a26fe admin admin] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:55:33.964040 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a517ce-5931-4f92-8a8c-9051122a26fe admin admin] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:55:33.968776 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e97cea2c-d2b5-483e-892f-41d22e387ca7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:55:33.969390 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e97cea2c-d2b5-483e-892f-41d22e387ca7 admin admin] REQUEST_METHOD: `POST` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:55:33.969672 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e97cea2c-d2b5-483e-892f-41d22e387ca7 admin admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:55:33.969948 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e97cea2c-d2b5-483e-892f-41d22e387ca7 admin admin] PATH_INFO: `/v3/projects` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:55:33.970640 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e97cea2c-d2b5-483e-892f-41d22e387ca7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:55:33.972600 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e97cea2c-d2b5-483e-892f-41d22e387ca7 admin admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:55:34.055219 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 925/1870] 10.222.0.20 () {60 vars in 1153 bytes} [Mon Aug 1 16:55:33 2022] POST /identity/v3/projects => generated 389 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 16:55:34.060005 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ab607f4-a416-4cf5-bd17-645fffdf0203 None None] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:55:34.071375 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ab607f4-a416-4cf5-bd17-645fffdf0203 None None] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:55:34.128474 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb42d88-64cd-4af1-ad43-17768367627a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:55:34.129127 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb42d88-64cd-4af1-ad43-17768367627a admin admin] REQUEST_METHOD: `POST` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:55:34.129409 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb42d88-64cd-4af1-ad43-17768367627a admin admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:55:34.129683 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb42d88-64cd-4af1-ad43-17768367627a admin admin] PATH_INFO: `/v3/users` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:55:34.130392 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bb42d88-64cd-4af1-ad43-17768367627a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:55:34.132317 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bb42d88-64cd-4af1-ad43-17768367627a admin admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:55:34.279438 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 946/1871] 10.222.0.20 () {60 vars in 1144 bytes} [Mon Aug 1 16:55:34 2022] POST /identity/v3/users => generated 301 bytes in 220 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 16:55:34.288422 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e97cea2c-d2b5-483e-892f-41d22e387ca7 admin admin] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:55:34.299008 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e97cea2c-d2b5-483e-892f-41d22e387ca7 admin admin] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:55:34.303678 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab98a206-9cd2-4518-9a67-12968dcb2783 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:55:34.304304 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab98a206-9cd2-4518-9a67-12968dcb2783 admin admin] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:55:34.304590 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab98a206-9cd2-4518-9a67-12968dcb2783 admin admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:55:34.304878 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab98a206-9cd2-4518-9a67-12968dcb2783 admin admin] PATH_INFO: `/v3/roles` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:55:34.305562 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab98a206-9cd2-4518-9a67-12968dcb2783 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:55:34.306847 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG dbcounter [-] [87058] Writing DB stats keystone:SELECT=100 {{(pid=87058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:55:34.307241 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab98a206-9cd2-4518-9a67-12968dcb2783 admin admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:55:34.318100 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 926/1872] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Aug 1 16:55:34 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 16:55:34.329915 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb42d88-64cd-4af1-ad43-17768367627a admin admin] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:55:34.340861 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb42d88-64cd-4af1-ad43-17768367627a admin admin] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:55:34.345731 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92585b04-7b4e-4fa8-b657-5408161699e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:55:34.350071 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92585b04-7b4e-4fa8-b657-5408161699e7 admin admin] REQUEST_METHOD: `PUT` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:55:34.350071 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92585b04-7b4e-4fa8-b657-5408161699e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:55:34.350071 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92585b04-7b4e-4fa8-b657-5408161699e7 admin admin] PATH_INFO: `/v3/projects/cd3a50e0c3d046e4bd87718f49a7b53c/users/19c23472c55f47368d4c1ed5bb5eb186/roles/440bb943dd5b46fea038b903b6eaaf81` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:55:34.363207 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92585b04-7b4e-4fa8-b657-5408161699e7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cd3a50e0c3d046e4bd87718f49a7b53c, user_id=19c23472c55f47368d4c1ed5bb5eb186, role_id=440bb943dd5b46fea038b903b6eaaf81)` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:55:34.365928 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92585b04-7b4e-4fa8-b657-5408161699e7 admin admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:55:34.368146 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG dbcounter [-] [87057] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=87057) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:55:34.467814 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 947/1873] 10.222.0.20 () {60 vars in 1483 bytes} [Mon Aug 1 16:55:34 2022] PUT /identity/v3/projects/cd3a50e0c3d046e4bd87718f49a7b53c/users/19c23472c55f47368d4c1ed5bb5eb186/roles/440bb943dd5b46fea038b903b6eaaf81 => generated 0 bytes in 138 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 16:55:34.472281 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab98a206-9cd2-4518-9a67-12968dcb2783 admin admin] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:55:34.490380 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab98a206-9cd2-4518-9a67-12968dcb2783 admin admin] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:55:34.588033 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb0558a-8b5f-42c3-bcdc-3f31a86a9a31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:55:34.588033 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb0558a-8b5f-42c3-bcdc-3f31a86a9a31 admin admin] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:55:34.588033 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb0558a-8b5f-42c3-bcdc-3f31a86a9a31 admin admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:55:34.588033 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb0558a-8b5f-42c3-bcdc-3f31a86a9a31 admin admin] PATH_INFO: `/v3/roles` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:55:34.588900 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eb0558a-8b5f-42c3-bcdc-3f31a86a9a31 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:55:34.588900 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eb0558a-8b5f-42c3-bcdc-3f31a86a9a31 admin admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:55:34.599408 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 927/1874] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Aug 1 16:55:34 2022] GET /identity/v3/roles => generated 1420 bytes in 128 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 16:55:34.604297 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92585b04-7b4e-4fa8-b657-5408161699e7 admin admin] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:55:34.616331 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92585b04-7b4e-4fa8-b657-5408161699e7 admin admin] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:55:34.622501 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db257773-ba09-46b4-bcdb-6033ece8304b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:55:34.626662 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db257773-ba09-46b4-bcdb-6033ece8304b admin admin] REQUEST_METHOD: `PUT` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:55:34.627024 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db257773-ba09-46b4-bcdb-6033ece8304b admin admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:55:34.627380 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db257773-ba09-46b4-bcdb-6033ece8304b admin admin] PATH_INFO: `/v3/projects/cd3a50e0c3d046e4bd87718f49a7b53c/users/19c23472c55f47368d4c1ed5bb5eb186/roles/440bb943dd5b46fea038b903b6eaaf81` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:55:34.629738 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db257773-ba09-46b4-bcdb-6033ece8304b admin admin] RBAC: Authorizing `identity:create_grant(project_id=cd3a50e0c3d046e4bd87718f49a7b53c, user_id=19c23472c55f47368d4c1ed5bb5eb186, role_id=440bb943dd5b46fea038b903b6eaaf81)` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:55:34.632590 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db257773-ba09-46b4-bcdb-6033ece8304b admin admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:55:34.650610 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 948/1875] 10.222.0.20 () {60 vars in 1483 bytes} [Mon Aug 1 16:55:34 2022] PUT /identity/v3/projects/cd3a50e0c3d046e4bd87718f49a7b53c/users/19c23472c55f47368d4c1ed5bb5eb186/roles/440bb943dd5b46fea038b903b6eaaf81 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 16:55:34.662182 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dcbc5f1-e3c8-4530-a57a-eaf3730de157 None None] REQUEST_METHOD: `POST` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:55:34.662619 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dcbc5f1-e3c8-4530-a57a-eaf3730de157 None None] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:55:34.663003 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dcbc5f1-e3c8-4530-a57a-eaf3730de157 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:55:34.693925 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG dbcounter [-] [87058] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=87058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:55:34.701531 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.auth.core [None req-8dcbc5f1-e3c8-4530-a57a-eaf3730de157 None None] MFA Rules not processed for user `19c23472c55f47368d4c1ed5bb5eb186`. Rule list: `[]` (Enabled: `True`). {{(pid=87057) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:55:34.702699 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.fernet_utils [None req-8dcbc5f1-e3c8-4530-a57a-eaf3730de157 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87057) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:55:34.780896 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 928/1876] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 1 16:55:34 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:55:34.792688 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db257773-ba09-46b4-bcdb-6033ece8304b admin admin] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:55:34.794599 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.fernet_utils [None req-db257773-ba09-46b4-bcdb-6033ece8304b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:55:34.859708 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db257773-ba09-46b4-bcdb-6033ece8304b admin admin] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:55:34.864242 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7405fb59-805d-4bec-a000-3b81d5aaf85a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25970b52ca15448fa1abb1f974d64038', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b992d6a65df4bf9b27c13192d57c5f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:55:34.865010 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7405fb59-805d-4bec-a000-3b81d5aaf85a service cinder] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:55:34.865296 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7405fb59-805d-4bec-a000-3b81d5aaf85a service cinder] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:55:34.865596 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7405fb59-805d-4bec-a000-3b81d5aaf85a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:55:34.880617 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7405fb59-805d-4bec-a000-3b81d5aaf85a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:55:34.882186 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7405fb59-805d-4bec-a000-3b81d5aaf85a service cinder] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:55:34.894260 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 949/1877] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 1 16:55:34 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 16:55:35.036359 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG dbcounter [-] [87057] Writing DB stats keystone:SELECT=93,keystone:INSERT=4,keystone:DELETE=3 {{(pid=87057) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:55:35.098962 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dcbc5f1-e3c8-4530-a57a-eaf3730de157 None None] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:55:35.160318 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dcbc5f1-e3c8-4530-a57a-eaf3730de157 None None] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:55:35.165225 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aacd4ed-abdc-4967-a9d3-dfaac10c18ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:55:35.166540 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aacd4ed-abdc-4967-a9d3-dfaac10c18ab admin admin] REQUEST_METHOD: `DELETE` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:55:35.166822 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aacd4ed-abdc-4967-a9d3-dfaac10c18ab admin admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:55:35.167095 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aacd4ed-abdc-4967-a9d3-dfaac10c18ab admin admin] PATH_INFO: `/v3/users/49c4981ff1bb41f88920b9c465997804` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:55:35.168388 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aacd4ed-abdc-4967-a9d3-dfaac10c18ab admin admin] RBAC: Authorizing `identity:delete_user(user_id=49c4981ff1bb41f88920b9c465997804)` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:55:35.170378 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aacd4ed-abdc-4967-a9d3-dfaac10c18ab admin admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:55:35.287975 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG dbcounter [-] [87058] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=87058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:55:35.344055 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.notifications [None req-1aacd4ed-abdc-4967-a9d3-dfaac10c18ab admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '49c4981ff1bb41f88920b9c465997804'} {{(pid=87057) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:55:35.344979 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.notifications [None req-1aacd4ed-abdc-4967-a9d3-dfaac10c18ab admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '49c4981ff1bb41f88920b9c465997804'} {{(pid=87057) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:55:35.413311 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.notifications [None req-1aacd4ed-abdc-4967-a9d3-dfaac10c18ab admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '49c4981ff1bb41f88920b9c465997804'} {{(pid=87057) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:55:35.413957 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.notifications [None req-1aacd4ed-abdc-4967-a9d3-dfaac10c18ab admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '49c4981ff1bb41f88920b9c465997804'} {{(pid=87057) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:55:35.421747 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.notifications [None req-1aacd4ed-abdc-4967-a9d3-dfaac10c18ab admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '49c4981ff1bb41f88920b9c465997804'} {{(pid=87057) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:55:35.440538 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 929/1878] 10.222.0.20 () {58 vars in 1224 bytes} [Mon Aug 1 16:55:35 2022] DELETE /identity/v3/users/49c4981ff1bb41f88920b9c465997804 => generated 0 bytes in 342 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 16:55:35.444983 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93f9a42-1b74-48f8-8eb5-430e2e8f3450 None None] REQUEST_METHOD: `POST` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:55:35.445269 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93f9a42-1b74-48f8-8eb5-430e2e8f3450 None None] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:55:35.445548 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93f9a42-1b74-48f8-8eb5-430e2e8f3450 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:55:35.465914 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.auth.core [None req-e93f9a42-1b74-48f8-8eb5-430e2e8f3450 None None] MFA Rules not processed for user `1007f8e379b74e169b98fb7c3c0ba5b4`. Rule list: `[]` (Enabled: `True`). {{(pid=87058) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:55:35.467155 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.fernet_utils [None req-e93f9a42-1b74-48f8-8eb5-430e2e8f3450 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:55:35.532938 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 950/1879] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 1 16:55:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 16:55:35.542780 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aacd4ed-abdc-4967-a9d3-dfaac10c18ab admin admin] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:55:35.544200 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.fernet_utils [None req-1aacd4ed-abdc-4967-a9d3-dfaac10c18ab admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87057) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:55:35.573759 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG dbcounter [-] [87058] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:55:35.602812 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG dbcounter [-] [87057] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=87057) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:55:35.621268 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aacd4ed-abdc-4967-a9d3-dfaac10c18ab admin admin] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:55:35.621268 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b150e732-d82d-4160-8907-04e9631d0460 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d9bd8d649154c979be3744bfd016f95', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b992d6a65df4bf9b27c13192d57c5f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:55:35.621268 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b150e732-d82d-4160-8907-04e9631d0460 service neutron] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:55:35.621268 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b150e732-d82d-4160-8907-04e9631d0460 service neutron] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:55:35.621626 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b150e732-d82d-4160-8907-04e9631d0460 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:55:35.634760 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b150e732-d82d-4160-8907-04e9631d0460 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:55:35.636792 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b150e732-d82d-4160-8907-04e9631d0460 service neutron] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:55:35.646971 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 930/1880] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 1 16:55:35 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 16:55:35.733809 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG dbcounter [-] [87058] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=87058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:55:35.985173 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG dbcounter [-] [87057] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87057) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:55:36.058239 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e93f9a42-1b74-48f8-8eb5-430e2e8f3450 None None] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:55:36.059687 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.fernet_utils [None req-e93f9a42-1b74-48f8-8eb5-430e2e8f3450 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:55:36.072871 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e93f9a42-1b74-48f8-8eb5-430e2e8f3450 None None] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:55:36.078585 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c86cbd25-d89c-4cd4-bcbb-dd8aeddbf3b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:55:36.079244 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86cbd25-d89c-4cd4-bcbb-dd8aeddbf3b2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:55:36.079676 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86cbd25-d89c-4cd4-bcbb-dd8aeddbf3b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:55:36.080223 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86cbd25-d89c-4cd4-bcbb-dd8aeddbf3b2 admin admin] PATH_INFO: `/v3/projects/d5bb252fc4584ab6b705d705a20a5e72` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:55:36.098355 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c86cbd25-d89c-4cd4-bcbb-dd8aeddbf3b2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d5bb252fc4584ab6b705d705a20a5e72)` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:55:36.104315 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c86cbd25-d89c-4cd4-bcbb-dd8aeddbf3b2 admin admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:55:36.186234 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.notifications [None req-c86cbd25-d89c-4cd4-bcbb-dd8aeddbf3b2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd5bb252fc4584ab6b705d705a20a5e72', 'request_id': 'req-c86cbd25-d89c-4cd4-bcbb-dd8aeddbf3b2'} {{(pid=87058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:55:36.193079 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.notifications [None req-c86cbd25-d89c-4cd4-bcbb-dd8aeddbf3b2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd5bb252fc4584ab6b705d705a20a5e72', 'request_id': 'req-c86cbd25-d89c-4cd4-bcbb-dd8aeddbf3b2'} {{(pid=87058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:55:36.199967 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.notifications [None req-c86cbd25-d89c-4cd4-bcbb-dd8aeddbf3b2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd5bb252fc4584ab6b705d705a20a5e72', 'request_id': 'req-c86cbd25-d89c-4cd4-bcbb-dd8aeddbf3b2'} {{(pid=87058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:55:36.270063 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.notifications [None req-c86cbd25-d89c-4cd4-bcbb-dd8aeddbf3b2 admin admin] The token cache is being invalidate because project d5bb252fc4584ab6b705d705a20a5e72 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87058) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 16:55:36.270351 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.notifications [None req-c86cbd25-d89c-4cd4-bcbb-dd8aeddbf3b2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:55:36.271147 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.notifications [None req-c86cbd25-d89c-4cd4-bcbb-dd8aeddbf3b2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:55:36.272724 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 951/1881] 10.222.0.20 () {58 vars in 1233 bytes} [Mon Aug 1 16:55:36 2022] DELETE /identity/v3/projects/d5bb252fc4584ab6b705d705a20a5e72 => generated 0 bytes in 215 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 16:55:36.279427 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b150e732-d82d-4160-8907-04e9631d0460 service neutron] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:55:36.289989 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b150e732-d82d-4160-8907-04e9631d0460 service neutron] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:55:36.345524 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a06fcd-f094-4500-b9a6-2cfb10becb8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:55:36.346203 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a06fcd-f094-4500-b9a6-2cfb10becb8d admin admin] REQUEST_METHOD: `DELETE` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:55:36.346489 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a06fcd-f094-4500-b9a6-2cfb10becb8d admin admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:55:36.346768 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a06fcd-f094-4500-b9a6-2cfb10becb8d admin admin] PATH_INFO: `/v3/users/19c23472c55f47368d4c1ed5bb5eb186` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:55:36.347971 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2a06fcd-f094-4500-b9a6-2cfb10becb8d admin admin] RBAC: Authorizing `identity:delete_user(user_id=19c23472c55f47368d4c1ed5bb5eb186)` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:55:36.349917 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2a06fcd-f094-4500-b9a6-2cfb10becb8d admin admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:55:36.479255 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG dbcounter [-] [87057] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=87057) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:55:36.571143 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.notifications [None req-c2a06fcd-f094-4500-b9a6-2cfb10becb8d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '19c23472c55f47368d4c1ed5bb5eb186'} {{(pid=87057) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:55:36.572289 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.notifications [None req-c2a06fcd-f094-4500-b9a6-2cfb10becb8d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '19c23472c55f47368d4c1ed5bb5eb186'} {{(pid=87057) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:55:36.642733 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.notifications [None req-c2a06fcd-f094-4500-b9a6-2cfb10becb8d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '19c23472c55f47368d4c1ed5bb5eb186'} {{(pid=87057) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:55:36.643749 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.notifications [None req-c2a06fcd-f094-4500-b9a6-2cfb10becb8d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '19c23472c55f47368d4c1ed5bb5eb186'} {{(pid=87057) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:55:36.653776 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.notifications [None req-c2a06fcd-f094-4500-b9a6-2cfb10becb8d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '19c23472c55f47368d4c1ed5bb5eb186'} {{(pid=87057) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:55:36.680670 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 931/1882] 10.222.0.20 () {58 vars in 1224 bytes} [Mon Aug 1 16:55:36 2022] DELETE /identity/v3/users/19c23472c55f47368d4c1ed5bb5eb186 => generated 0 bytes in 401 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 16:55:36.779053 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c86cbd25-d89c-4cd4-bcbb-dd8aeddbf3b2 admin admin] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:55:36.782155 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.fernet_utils [None req-c86cbd25-d89c-4cd4-bcbb-dd8aeddbf3b2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:55:36.799347 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c86cbd25-d89c-4cd4-bcbb-dd8aeddbf3b2 admin admin] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:55:36.870636 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG dbcounter [-] [87058] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=87058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:55:36.879967 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0234c547-f91e-41be-91ec-d3a496fb1756 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:55:36.880797 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0234c547-f91e-41be-91ec-d3a496fb1756 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:55:36.881308 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0234c547-f91e-41be-91ec-d3a496fb1756 admin admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:55:36.881713 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0234c547-f91e-41be-91ec-d3a496fb1756 admin admin] PATH_INFO: `/v3/users/4fa2710924b444188db11766e97762db` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:55:36.883089 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0234c547-f91e-41be-91ec-d3a496fb1756 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4fa2710924b444188db11766e97762db)` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:55:36.885900 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0234c547-f91e-41be-91ec-d3a496fb1756 admin admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:55:37.137086 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.notifications [None req-0234c547-f91e-41be-91ec-d3a496fb1756 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4fa2710924b444188db11766e97762db'} {{(pid=87058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:55:37.138143 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.notifications [None req-0234c547-f91e-41be-91ec-d3a496fb1756 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4fa2710924b444188db11766e97762db'} {{(pid=87058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:55:37.139009 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.notifications [None req-0234c547-f91e-41be-91ec-d3a496fb1756 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4fa2710924b444188db11766e97762db'} {{(pid=87058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:55:37.154762 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.notifications [None req-0234c547-f91e-41be-91ec-d3a496fb1756 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4fa2710924b444188db11766e97762db'} {{(pid=87058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:55:37.163147 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.notifications [None req-0234c547-f91e-41be-91ec-d3a496fb1756 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4fa2710924b444188db11766e97762db'} {{(pid=87058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:55:37.265871 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 952/1883] 10.222.0.20 () {58 vars in 1224 bytes} [Mon Aug 1 16:55:36 2022] DELETE /identity/v3/users/4fa2710924b444188db11766e97762db => generated 0 bytes in 487 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 16:55:37.271346 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-675fea35-c7f4-4751-8e49-31c68256a01d None None] REQUEST_METHOD: `POST` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:55:37.272279 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-675fea35-c7f4-4751-8e49-31c68256a01d None None] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:55:37.272792 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-675fea35-c7f4-4751-8e49-31c68256a01d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:55:37.296799 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG dbcounter [-] [87057] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=87057) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:55:37.317873 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.auth.core [None req-675fea35-c7f4-4751-8e49-31c68256a01d None None] MFA Rules not processed for user `1007f8e379b74e169b98fb7c3c0ba5b4`. Rule list: `[]` (Enabled: `True`). {{(pid=87057) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:55:37.320286 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.fernet_utils [None req-675fea35-c7f4-4751-8e49-31c68256a01d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87057) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:55:37.416990 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 932/1884] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 1 16:55:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:55:37.456949 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0234c547-f91e-41be-91ec-d3a496fb1756 admin admin] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:55:37.458568 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.fernet_utils [None req-0234c547-f91e-41be-91ec-d3a496fb1756 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:55:37.549031 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0234c547-f91e-41be-91ec-d3a496fb1756 admin admin] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:55:37.563666 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0388a3f8-55df-4b32-bce3-c9727691efc6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d9bd8d649154c979be3744bfd016f95', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b992d6a65df4bf9b27c13192d57c5f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:55:37.564405 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0388a3f8-55df-4b32-bce3-c9727691efc6 service neutron] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:55:37.564758 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0388a3f8-55df-4b32-bce3-c9727691efc6 service neutron] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:55:37.565133 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0388a3f8-55df-4b32-bce3-c9727691efc6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:55:37.576684 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0388a3f8-55df-4b32-bce3-c9727691efc6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:55:37.578388 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0388a3f8-55df-4b32-bce3-c9727691efc6 service neutron] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:55:37.591282 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 953/1885] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 1 16:55:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 16:55:37.894871 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-675fea35-c7f4-4751-8e49-31c68256a01d None None] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:55:37.896719 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.fernet_utils [None req-675fea35-c7f4-4751-8e49-31c68256a01d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87057) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:55:37.921590 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-675fea35-c7f4-4751-8e49-31c68256a01d None None] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:55:37.934096 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23fcb5a2-eee5-4391-91c8-fd20e33679c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:55:37.934896 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23fcb5a2-eee5-4391-91c8-fd20e33679c5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:55:37.935180 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23fcb5a2-eee5-4391-91c8-fd20e33679c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:55:37.935457 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23fcb5a2-eee5-4391-91c8-fd20e33679c5 admin admin] PATH_INFO: `/v3/projects/cd3a50e0c3d046e4bd87718f49a7b53c` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:55:37.936683 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23fcb5a2-eee5-4391-91c8-fd20e33679c5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cd3a50e0c3d046e4bd87718f49a7b53c)` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:55:37.940443 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23fcb5a2-eee5-4391-91c8-fd20e33679c5 admin admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:55:38.088832 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.notifications [None req-23fcb5a2-eee5-4391-91c8-fd20e33679c5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cd3a50e0c3d046e4bd87718f49a7b53c', 'request_id': 'req-23fcb5a2-eee5-4391-91c8-fd20e33679c5'} {{(pid=87057) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:55:38.099986 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.notifications [None req-23fcb5a2-eee5-4391-91c8-fd20e33679c5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cd3a50e0c3d046e4bd87718f49a7b53c', 'request_id': 'req-23fcb5a2-eee5-4391-91c8-fd20e33679c5'} {{(pid=87057) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:55:38.107672 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.notifications [None req-23fcb5a2-eee5-4391-91c8-fd20e33679c5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cd3a50e0c3d046e4bd87718f49a7b53c', 'request_id': 'req-23fcb5a2-eee5-4391-91c8-fd20e33679c5'} {{(pid=87057) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:55:38.162938 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG dbcounter [-] [87057] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87057) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:55:38.224502 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.notifications [None req-23fcb5a2-eee5-4391-91c8-fd20e33679c5 admin admin] The token cache is being invalidate because project cd3a50e0c3d046e4bd87718f49a7b53c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87057) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 16:55:38.224502 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.notifications [None req-23fcb5a2-eee5-4391-91c8-fd20e33679c5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87057) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:55:38.226552 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.notifications [None req-23fcb5a2-eee5-4391-91c8-fd20e33679c5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87057) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:55:38.229915 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 933/1886] 10.222.0.20 () {58 vars in 1233 bytes} [Mon Aug 1 16:55:37 2022] DELETE /identity/v3/projects/cd3a50e0c3d046e4bd87718f49a7b53c => generated 0 bytes in 336 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 16:55:38.235581 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6551fd45-a178-4370-aa54-9de4f77b6fd9 None None] REQUEST_METHOD: `POST` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:55:38.236347 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6551fd45-a178-4370-aa54-9de4f77b6fd9 None None] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:55:38.236738 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6551fd45-a178-4370-aa54-9de4f77b6fd9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:55:38.274612 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.auth.core [None req-6551fd45-a178-4370-aa54-9de4f77b6fd9 None None] MFA Rules not processed for user `1007f8e379b74e169b98fb7c3c0ba5b4`. Rule list: `[]` (Enabled: `True`). {{(pid=87058) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:55:38.277196 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.fernet_utils [None req-6551fd45-a178-4370-aa54-9de4f77b6fd9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:55:38.362139 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 954/1887] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 1 16:55:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:55:38.366971 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8be5c2-3009-4376-9b4e-be66d5480968 None None] REQUEST_METHOD: `POST` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:55:38.367253 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8be5c2-3009-4376-9b4e-be66d5480968 None None] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:55:38.385582 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8be5c2-3009-4376-9b4e-be66d5480968 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:55:38.388762 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.auth.core [None req-3b8be5c2-3009-4376-9b4e-be66d5480968 None None] MFA Rules not processed for user `1007f8e379b74e169b98fb7c3c0ba5b4`. Rule list: `[]` (Enabled: `True`). {{(pid=87057) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:55:38.389974 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.fernet_utils [None req-3b8be5c2-3009-4376-9b4e-be66d5480968 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87057) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:55:38.400812 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 934/1888] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 1 16:55:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:55:38.407440 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6551fd45-a178-4370-aa54-9de4f77b6fd9 None None] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:55:38.415912 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6551fd45-a178-4370-aa54-9de4f77b6fd9 None None] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:55:38.421658 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1af6678-f78d-4678-97b4-6e94d71bbb09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:55:38.422322 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1af6678-f78d-4678-97b4-6e94d71bbb09 admin admin] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:55:38.422605 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1af6678-f78d-4678-97b4-6e94d71bbb09 admin admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:55:38.422881 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1af6678-f78d-4678-97b4-6e94d71bbb09 admin admin] PATH_INFO: `/v3/domains` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:55:38.423591 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1af6678-f78d-4678-97b4-6e94d71bbb09 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87058) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:55:38.423956 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1af6678-f78d-4678-97b4-6e94d71bbb09 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:55:38.425498 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1af6678-f78d-4678-97b4-6e94d71bbb09 admin admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:55:38.437944 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 955/1889] 10.222.0.20 () {58 vars in 1153 bytes} [Mon Aug 1 16:55:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:55:38.442421 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b8be5c2-3009-4376-9b4e-be66d5480968 None None] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:55:38.452368 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b8be5c2-3009-4376-9b4e-be66d5480968 None None] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:55:38.458744 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d20ff887-f5b0-4b0c-9263-20fac73d92b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:55:38.459612 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d20ff887-f5b0-4b0c-9263-20fac73d92b1 admin admin] REQUEST_METHOD: `POST` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:55:38.460438 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d20ff887-f5b0-4b0c-9263-20fac73d92b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:55:38.460438 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d20ff887-f5b0-4b0c-9263-20fac73d92b1 admin admin] PATH_INFO: `/v3/projects` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:55:38.460599 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1af6678-f78d-4678-97b4-6e94d71bbb09 admin admin] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:55:38.460912 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d20ff887-f5b0-4b0c-9263-20fac73d92b1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:55:38.462479 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.fernet_utils [None req-c1af6678-f78d-4678-97b4-6e94d71bbb09 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:55:38.463475 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d20ff887-f5b0-4b0c-9263-20fac73d92b1 admin admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:55:38.476238 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1af6678-f78d-4678-97b4-6e94d71bbb09 admin admin] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:55:38.482402 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-814497a6-c174-48be-b47a-9c50cdc4d32f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:55:38.483134 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814497a6-c174-48be-b47a-9c50cdc4d32f admin admin] REQUEST_METHOD: `DELETE` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:55:38.483475 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814497a6-c174-48be-b47a-9c50cdc4d32f admin admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:55:38.483754 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814497a6-c174-48be-b47a-9c50cdc4d32f admin admin] PATH_INFO: `/v3/projects/7e6dba3c3e624cf090439b3545d22944` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:55:38.484943 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-814497a6-c174-48be-b47a-9c50cdc4d32f admin admin] RBAC: Authorizing `identity:delete_project(project_id=7e6dba3c3e624cf090439b3545d22944)` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:55:38.486925 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-814497a6-c174-48be-b47a-9c50cdc4d32f admin admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:55:38.488815 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG dbcounter [-] [87058] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:55:38.541137 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 935/1890] 10.222.0.20 () {60 vars in 1153 bytes} [Mon Aug 1 16:55:38 2022] POST /identity/v3/projects => generated 383 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 16:55:38.545343 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d20ff887-f5b0-4b0c-9263-20fac73d92b1 admin admin] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:55:38.605816 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d20ff887-f5b0-4b0c-9263-20fac73d92b1 admin admin] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:55:38.608251 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.notifications [None req-814497a6-c174-48be-b47a-9c50cdc4d32f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7e6dba3c3e624cf090439b3545d22944', 'request_id': 'req-814497a6-c174-48be-b47a-9c50cdc4d32f'} {{(pid=87058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:55:38.615563 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.notifications [None req-814497a6-c174-48be-b47a-9c50cdc4d32f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7e6dba3c3e624cf090439b3545d22944', 'request_id': 'req-814497a6-c174-48be-b47a-9c50cdc4d32f'} {{(pid=87058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:55:38.622106 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.notifications [None req-814497a6-c174-48be-b47a-9c50cdc4d32f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7e6dba3c3e624cf090439b3545d22944', 'request_id': 'req-814497a6-c174-48be-b47a-9c50cdc4d32f'} {{(pid=87058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:55:38.663466 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd634932-d2b5-4416-9518-39c1c0d5a73c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:55:38.664094 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd634932-d2b5-4416-9518-39c1c0d5a73c admin admin] REQUEST_METHOD: `POST` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:55:38.664439 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd634932-d2b5-4416-9518-39c1c0d5a73c admin admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:55:38.664724 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd634932-d2b5-4416-9518-39c1c0d5a73c admin admin] PATH_INFO: `/v3/users` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:55:38.665436 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd634932-d2b5-4416-9518-39c1c0d5a73c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:55:38.669707 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd634932-d2b5-4416-9518-39c1c0d5a73c admin admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:55:38.693315 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.notifications [None req-814497a6-c174-48be-b47a-9c50cdc4d32f admin admin] The token cache is being invalidate because project 7e6dba3c3e624cf090439b3545d22944 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87058) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 16:55:38.693618 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.notifications [None req-814497a6-c174-48be-b47a-9c50cdc4d32f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:55:38.694307 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.notifications [None req-814497a6-c174-48be-b47a-9c50cdc4d32f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:55:38.699393 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 956/1891] 10.222.0.20 () {58 vars in 1233 bytes} [Mon Aug 1 16:55:38 2022] DELETE /identity/v3/projects/7e6dba3c3e624cf090439b3545d22944 => generated 0 bytes in 237 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 16:55:38.700195 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-814497a6-c174-48be-b47a-9c50cdc4d32f admin admin] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:55:38.702018 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.fernet_utils [None req-814497a6-c174-48be-b47a-9c50cdc4d32f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:55:38.718360 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-814497a6-c174-48be-b47a-9c50cdc4d32f admin admin] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:55:38.756251 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ac6a17-d623-4cc3-b4a6-eb7520f62880 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:55:38.756251 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ac6a17-d623-4cc3-b4a6-eb7520f62880 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:55:38.756251 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ac6a17-d623-4cc3-b4a6-eb7520f62880 admin admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:55:38.756251 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ac6a17-d623-4cc3-b4a6-eb7520f62880 admin admin] PATH_INFO: `/v3/users/22872d2368ae4407ba6230dede22fb25` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:55:38.756594 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17ac6a17-d623-4cc3-b4a6-eb7520f62880 admin admin] RBAC: Authorizing `identity:delete_user(user_id=22872d2368ae4407ba6230dede22fb25)` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:55:38.756594 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17ac6a17-d623-4cc3-b4a6-eb7520f62880 admin admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:55:39.009910 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 936/1892] 10.222.0.20 () {60 vars in 1144 bytes} [Mon Aug 1 16:55:38 2022] POST /identity/v3/users => generated 292 bytes in 464 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 16:55:39.026519 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd634932-d2b5-4416-9518-39c1c0d5a73c admin admin] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:55:39.026519 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.fernet_utils [None req-cd634932-d2b5-4416-9518-39c1c0d5a73c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87057) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:55:39.069803 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd634932-d2b5-4416-9518-39c1c0d5a73c admin admin] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:55:39.074183 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-711a10ec-7b6e-41c6-a3e1-ad64d9cf0463 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25970b52ca15448fa1abb1f974d64038', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b992d6a65df4bf9b27c13192d57c5f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:55:39.074789 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711a10ec-7b6e-41c6-a3e1-ad64d9cf0463 service cinder] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:55:39.075065 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711a10ec-7b6e-41c6-a3e1-ad64d9cf0463 service cinder] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:55:39.075349 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711a10ec-7b6e-41c6-a3e1-ad64d9cf0463 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:55:39.077118 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.fernet_utils [None req-711a10ec-7b6e-41c6-a3e1-ad64d9cf0463 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87057) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:55:39.139290 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-711a10ec-7b6e-41c6-a3e1-ad64d9cf0463 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:55:39.140808 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-711a10ec-7b6e-41c6-a3e1-ad64d9cf0463 service cinder] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:55:39.150949 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 937/1893] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 1 16:55:39 2022] GET /identity/v3/auth/tokens => generated 3305 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 16:55:39.151833 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-711a10ec-7b6e-41c6-a3e1-ad64d9cf0463 service cinder] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:55:39.153974 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.fernet_utils [None req-711a10ec-7b6e-41c6-a3e1-ad64d9cf0463 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87057) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:55:39.172799 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-711a10ec-7b6e-41c6-a3e1-ad64d9cf0463 service cinder] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:55:39.177641 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca920ff3-b4c4-4e47-9baf-d72eb7ea411e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:55:39.178326 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca920ff3-b4c4-4e47-9baf-d72eb7ea411e admin admin] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:55:39.178603 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca920ff3-b4c4-4e47-9baf-d72eb7ea411e admin admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:55:39.178873 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca920ff3-b4c4-4e47-9baf-d72eb7ea411e admin admin] PATH_INFO: `/v3/roles` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:55:39.179604 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca920ff3-b4c4-4e47-9baf-d72eb7ea411e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:55:39.181493 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca920ff3-b4c4-4e47-9baf-d72eb7ea411e admin admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:55:39.191842 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 938/1894] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Aug 1 16:55:39 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 16:55:39.195798 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca920ff3-b4c4-4e47-9baf-d72eb7ea411e admin admin] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:55:39.206583 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca920ff3-b4c4-4e47-9baf-d72eb7ea411e admin admin] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:55:39.211526 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9daf6ff3-5359-4cb5-8433-ae002542d8ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:55:39.212081 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9daf6ff3-5359-4cb5-8433-ae002542d8ab admin admin] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:55:39.213040 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9daf6ff3-5359-4cb5-8433-ae002542d8ab admin admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:55:39.213413 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9daf6ff3-5359-4cb5-8433-ae002542d8ab admin admin] PATH_INFO: `/v3/roles` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:55:39.214329 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9daf6ff3-5359-4cb5-8433-ae002542d8ab admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:55:39.215927 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9daf6ff3-5359-4cb5-8433-ae002542d8ab admin admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:55:39.226149 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 939/1895] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Aug 1 16:55:39 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 01 16:55:39.230441 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9daf6ff3-5359-4cb5-8433-ae002542d8ab admin admin] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:55:39.240093 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9daf6ff3-5359-4cb5-8433-ae002542d8ab admin admin] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:55:39.245262 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61fcd66d-5cd1-4b7a-aa82-4ad5f546605d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:55:39.245906 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61fcd66d-5cd1-4b7a-aa82-4ad5f546605d admin admin] REQUEST_METHOD: `PUT` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:55:39.246221 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61fcd66d-5cd1-4b7a-aa82-4ad5f546605d admin admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:55:39.246499 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61fcd66d-5cd1-4b7a-aa82-4ad5f546605d admin admin] PATH_INFO: `/v3/projects/3a97c42aae924d89b5a59d95daf88738/users/feb90f07095846f1a26f3c2c23b50420/roles/0999d3da63a545f18ff3252afdc90518` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:55:39.263435 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61fcd66d-5cd1-4b7a-aa82-4ad5f546605d admin admin] RBAC: Authorizing `identity:create_grant(project_id=3a97c42aae924d89b5a59d95daf88738, user_id=feb90f07095846f1a26f3c2c23b50420, role_id=0999d3da63a545f18ff3252afdc90518)` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:55:39.266951 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61fcd66d-5cd1-4b7a-aa82-4ad5f546605d admin admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:55:39.857468 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 940/1896] 10.222.0.20 () {60 vars in 1483 bytes} [Mon Aug 1 16:55:39 2022] PUT /identity/v3/projects/3a97c42aae924d89b5a59d95daf88738/users/feb90f07095846f1a26f3c2c23b50420/roles/0999d3da63a545f18ff3252afdc90518 => generated 0 bytes in 627 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 16:55:39.880373 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a264e066-305d-4092-9243-a265a916f947 None None] REQUEST_METHOD: `POST` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:55:39.880480 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a264e066-305d-4092-9243-a265a916f947 None None] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:55:39.880729 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a264e066-305d-4092-9243-a265a916f947 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:55:39.931206 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.auth.core [None req-a264e066-305d-4092-9243-a265a916f947 None None] MFA Rules not processed for user `feb90f07095846f1a26f3c2c23b50420`. Rule list: `[]` (Enabled: `True`). {{(pid=87057) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:55:39.933661 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.fernet_utils [None req-a264e066-305d-4092-9243-a265a916f947 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87057) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:55:39.951292 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.notifications [None req-17ac6a17-d623-4cc3-b4a6-eb7520f62880 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '22872d2368ae4407ba6230dede22fb25'} {{(pid=87058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:55:39.953088 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.notifications [None req-17ac6a17-d623-4cc3-b4a6-eb7520f62880 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '22872d2368ae4407ba6230dede22fb25'} {{(pid=87058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:55:39.953867 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.notifications [None req-17ac6a17-d623-4cc3-b4a6-eb7520f62880 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '22872d2368ae4407ba6230dede22fb25'} {{(pid=87058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:55:39.971726 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.notifications [None req-17ac6a17-d623-4cc3-b4a6-eb7520f62880 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '22872d2368ae4407ba6230dede22fb25'} {{(pid=87058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:55:39.983726 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.notifications [None req-17ac6a17-d623-4cc3-b4a6-eb7520f62880 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '22872d2368ae4407ba6230dede22fb25'} {{(pid=87058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:55:40.018122 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 941/1897] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 1 16:55:39 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:55:40.021319 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a264e066-305d-4092-9243-a265a916f947 None None] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:55:40.022671 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.fernet_utils [None req-a264e066-305d-4092-9243-a265a916f947 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87057) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:55:40.106988 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 957/1898] 10.222.0.20 () {58 vars in 1224 bytes} [Mon Aug 1 16:55:38 2022] DELETE /identity/v3/users/22872d2368ae4407ba6230dede22fb25 => generated 0 bytes in 1402 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 16:55:40.112336 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a264e066-305d-4092-9243-a265a916f947 None None] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:55:40.187171 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a289ee7-7675-47c7-a9f4-2af2ccd3bf8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:55:40.188215 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a289ee7-7675-47c7-a9f4-2af2ccd3bf8d admin admin] REQUEST_METHOD: `POST` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:55:40.188499 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a289ee7-7675-47c7-a9f4-2af2ccd3bf8d admin admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:55:40.192051 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a289ee7-7675-47c7-a9f4-2af2ccd3bf8d admin admin] PATH_INFO: `/v3/projects` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:55:40.192051 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a289ee7-7675-47c7-a9f4-2af2ccd3bf8d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:55:40.192051 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a289ee7-7675-47c7-a9f4-2af2ccd3bf8d admin admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:55:40.347817 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG dbcounter [-] [87057] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87057) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:55:40.358421 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 942/1899] 10.222.0.20 () {60 vars in 1153 bytes} [Mon Aug 1 16:55:40 2022] POST /identity/v3/projects => generated 383 bytes in 335 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 16:55:40.359607 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ac6a17-d623-4cc3-b4a6-eb7520f62880 admin admin] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:55:40.373550 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ac6a17-d623-4cc3-b4a6-eb7520f62880 admin admin] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:55:40.448549 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6b64559-d699-4ebd-a886-a69a0c87e8b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:55:40.449391 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b64559-d699-4ebd-a886-a69a0c87e8b7 admin admin] REQUEST_METHOD: `POST` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:55:40.449899 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b64559-d699-4ebd-a886-a69a0c87e8b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:55:40.450207 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b64559-d699-4ebd-a886-a69a0c87e8b7 admin admin] PATH_INFO: `/v3/users` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:55:40.450888 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6b64559-d699-4ebd-a886-a69a0c87e8b7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:55:40.454388 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6b64559-d699-4ebd-a886-a69a0c87e8b7 admin admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:55:40.596925 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 958/1900] 10.222.0.20 () {60 vars in 1144 bytes} [Mon Aug 1 16:55:40 2022] POST /identity/v3/users => generated 298 bytes in 238 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 16:55:40.600752 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a289ee7-7675-47c7-a9f4-2af2ccd3bf8d admin admin] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:55:40.612431 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a289ee7-7675-47c7-a9f4-2af2ccd3bf8d admin admin] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:55:40.635635 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6b64559-d699-4ebd-a886-a69a0c87e8b7 admin admin] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:55:40.640259 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-384b122e-1bef-4244-9e78-a24ab01596d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:55:40.640525 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.fernet_utils [None req-b6b64559-d699-4ebd-a886-a69a0c87e8b7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:55:40.640932 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384b122e-1bef-4244-9e78-a24ab01596d8 admin admin] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:55:40.641210 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384b122e-1bef-4244-9e78-a24ab01596d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:55:40.641528 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384b122e-1bef-4244-9e78-a24ab01596d8 admin admin] PATH_INFO: `/v3/roles` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:55:40.642575 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-384b122e-1bef-4244-9e78-a24ab01596d8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:55:40.647256 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-384b122e-1bef-4244-9e78-a24ab01596d8 admin admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:55:40.675669 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 943/1901] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Aug 1 16:55:40 2022] GET /identity/v3/roles => generated 1420 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 16:55:40.675756 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6b64559-d699-4ebd-a886-a69a0c87e8b7 admin admin] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:55:40.683583 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-384b122e-1bef-4244-9e78-a24ab01596d8 admin admin] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:55:40.696611 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-220b6283-ab33-4dc9-b6cc-7bba45ba658f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:55:40.698372 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220b6283-ab33-4dc9-b6cc-7bba45ba658f admin admin] REQUEST_METHOD: `DELETE` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:55:40.703323 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220b6283-ab33-4dc9-b6cc-7bba45ba658f admin admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:55:40.703323 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220b6283-ab33-4dc9-b6cc-7bba45ba658f admin admin] PATH_INFO: `/v3/projects/3fa4576551874b1692f62cc8d1b16704` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:55:40.703323 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-220b6283-ab33-4dc9-b6cc-7bba45ba658f admin admin] RBAC: Authorizing `identity:delete_project(project_id=3fa4576551874b1692f62cc8d1b16704)` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:55:40.703770 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-384b122e-1bef-4244-9e78-a24ab01596d8 admin admin] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:55:40.704598 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-220b6283-ab33-4dc9-b6cc-7bba45ba658f admin admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:55:40.708470 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fc42211-a4b4-40e5-b4d7-a164f2d9e2b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:55:40.709124 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc42211-a4b4-40e5-b4d7-a164f2d9e2b0 admin admin] REQUEST_METHOD: `PUT` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:55:40.709473 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc42211-a4b4-40e5-b4d7-a164f2d9e2b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:55:40.710046 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc42211-a4b4-40e5-b4d7-a164f2d9e2b0 admin admin] PATH_INFO: `/v3/projects/c5848bd3203548e59309a80c31a72350/users/4e5312336075410e8ad0e1e93461628e/roles/440bb943dd5b46fea038b903b6eaaf81` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:55:40.755168 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fc42211-a4b4-40e5-b4d7-a164f2d9e2b0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c5848bd3203548e59309a80c31a72350, user_id=4e5312336075410e8ad0e1e93461628e, role_id=440bb943dd5b46fea038b903b6eaaf81)` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:55:40.764156 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fc42211-a4b4-40e5-b4d7-a164f2d9e2b0 admin admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:55:40.866292 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG dbcounter [-] [87058] Writing DB stats keystone:SELECT=80,keystone:DELETE=17,keystone:INSERT=3 {{(pid=87058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:55:40.874928 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 944/1902] 10.222.0.20 () {60 vars in 1483 bytes} [Mon Aug 1 16:55:40 2022] PUT /identity/v3/projects/c5848bd3203548e59309a80c31a72350/users/4e5312336075410e8ad0e1e93461628e/roles/440bb943dd5b46fea038b903b6eaaf81 => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 16:55:40.880206 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fc42211-a4b4-40e5-b4d7-a164f2d9e2b0 admin admin] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:55:40.893331 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.notifications [None req-220b6283-ab33-4dc9-b6cc-7bba45ba658f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3fa4576551874b1692f62cc8d1b16704', 'request_id': 'req-220b6283-ab33-4dc9-b6cc-7bba45ba658f'} {{(pid=87058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:55:40.903257 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.notifications [None req-220b6283-ab33-4dc9-b6cc-7bba45ba658f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3fa4576551874b1692f62cc8d1b16704', 'request_id': 'req-220b6283-ab33-4dc9-b6cc-7bba45ba658f'} {{(pid=87058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:55:40.912287 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.notifications [None req-220b6283-ab33-4dc9-b6cc-7bba45ba658f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3fa4576551874b1692f62cc8d1b16704', 'request_id': 'req-220b6283-ab33-4dc9-b6cc-7bba45ba658f'} {{(pid=87058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:55:40.956836 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fc42211-a4b4-40e5-b4d7-a164f2d9e2b0 admin admin] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:55:41.014924 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8440e623-f236-4657-a3ed-c8da4727f422 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:55:41.015187 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.notifications [None req-220b6283-ab33-4dc9-b6cc-7bba45ba658f admin admin] The token cache is being invalidate because project 3fa4576551874b1692f62cc8d1b16704 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87058) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 16:55:41.019335 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.notifications [None req-220b6283-ab33-4dc9-b6cc-7bba45ba658f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:55:41.019452 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8440e623-f236-4657-a3ed-c8da4727f422 admin admin] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:55:41.019452 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8440e623-f236-4657-a3ed-c8da4727f422 admin admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:55:41.019452 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8440e623-f236-4657-a3ed-c8da4727f422 admin admin] PATH_INFO: `/v3/roles` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:55:41.020007 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.notifications [None req-220b6283-ab33-4dc9-b6cc-7bba45ba658f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:55:41.020104 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8440e623-f236-4657-a3ed-c8da4727f422 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:55:41.020187 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 959/1903] 10.222.0.20 () {58 vars in 1233 bytes} [Mon Aug 1 16:55:40 2022] DELETE /identity/v3/projects/3fa4576551874b1692f62cc8d1b16704 => generated 0 bytes in 384 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 16:55:41.020269 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8440e623-f236-4657-a3ed-c8da4727f422 admin admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:55:41.024585 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd03c265-6af3-419c-a1d0-1fca16d8d459 None None] REQUEST_METHOD: `POST` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:55:41.025402 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd03c265-6af3-419c-a1d0-1fca16d8d459 None None] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:55:41.025758 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd03c265-6af3-419c-a1d0-1fca16d8d459 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:55:41.034533 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 945/1904] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Aug 1 16:55:40 2022] GET /identity/v3/roles => generated 1420 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 16:55:41.035578 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8440e623-f236-4657-a3ed-c8da4727f422 admin admin] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:55:41.037542 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.fernet_utils [None req-8440e623-f236-4657-a3ed-c8da4727f422 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87057) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:55:41.050957 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8440e623-f236-4657-a3ed-c8da4727f422 admin admin] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:55:41.051537 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.auth.core [None req-dd03c265-6af3-419c-a1d0-1fca16d8d459 None None] MFA Rules not processed for user `1007f8e379b74e169b98fb7c3c0ba5b4`. Rule list: `[]` (Enabled: `True`). {{(pid=87058) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:55:41.052767 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.fernet_utils [None req-dd03c265-6af3-419c-a1d0-1fca16d8d459 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:55:41.057229 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93d68666-3db1-4a80-a17f-0fe434a2c894 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:55:41.057944 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d68666-3db1-4a80-a17f-0fe434a2c894 admin admin] REQUEST_METHOD: `PUT` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:55:41.058319 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d68666-3db1-4a80-a17f-0fe434a2c894 admin admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:55:41.058661 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d68666-3db1-4a80-a17f-0fe434a2c894 admin admin] PATH_INFO: `/v3/projects/c5848bd3203548e59309a80c31a72350/users/4e5312336075410e8ad0e1e93461628e/roles/440bb943dd5b46fea038b903b6eaaf81` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:55:41.060824 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93d68666-3db1-4a80-a17f-0fe434a2c894 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c5848bd3203548e59309a80c31a72350, user_id=4e5312336075410e8ad0e1e93461628e, role_id=440bb943dd5b46fea038b903b6eaaf81)` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:55:41.062308 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 960/1905] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 1 16:55:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 16:55:41.069237 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93d68666-3db1-4a80-a17f-0fe434a2c894 admin admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:55:41.070222 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b466732d-b7e4-4537-8a37-227fa0b7ebf9 None None] REQUEST_METHOD: `POST` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:55:41.070532 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b466732d-b7e4-4537-8a37-227fa0b7ebf9 None None] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:55:41.070787 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b466732d-b7e4-4537-8a37-227fa0b7ebf9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:55:41.079544 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 946/1906] 10.222.0.20 () {60 vars in 1483 bytes} [Mon Aug 1 16:55:41 2022] PUT /identity/v3/projects/c5848bd3203548e59309a80c31a72350/users/4e5312336075410e8ad0e1e93461628e/roles/440bb943dd5b46fea038b903b6eaaf81 => generated 0 bytes in 44 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 16:55:41.092916 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9776a99-bd00-4faa-9917-30c6121d34e0 None None] REQUEST_METHOD: `POST` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:55:41.093382 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9776a99-bd00-4faa-9917-30c6121d34e0 None None] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:55:41.093789 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.auth.core [None req-b466732d-b7e4-4537-8a37-227fa0b7ebf9 None None] MFA Rules not processed for user `1007f8e379b74e169b98fb7c3c0ba5b4`. Rule list: `[]` (Enabled: `True`). {{(pid=87058) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:55:41.094241 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9776a99-bd00-4faa-9917-30c6121d34e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:55:41.095413 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.fernet_utils [None req-b466732d-b7e4-4537-8a37-227fa0b7ebf9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:55:41.136641 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.auth.core [None req-f9776a99-bd00-4faa-9917-30c6121d34e0 None None] MFA Rules not processed for user `4e5312336075410e8ad0e1e93461628e`. Rule list: `[]` (Enabled: `True`). {{(pid=87057) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:55:41.138535 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.fernet_utils [None req-f9776a99-bd00-4faa-9917-30c6121d34e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87057) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:55:41.181788 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 961/1907] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 1 16:55:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 16:55:41.195336 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b466732d-b7e4-4537-8a37-227fa0b7ebf9 None None] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:55:41.205749 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b466732d-b7e4-4537-8a37-227fa0b7ebf9 None None] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:55:41.210888 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b41e8bf-bb7a-47ab-86ae-7e0077c702d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:55:41.211941 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b41e8bf-bb7a-47ab-86ae-7e0077c702d7 admin admin] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:55:41.212993 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b41e8bf-bb7a-47ab-86ae-7e0077c702d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:55:41.213557 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b41e8bf-bb7a-47ab-86ae-7e0077c702d7 admin admin] PATH_INFO: `/v3/domains` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:55:41.214694 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b41e8bf-bb7a-47ab-86ae-7e0077c702d7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87058) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:55:41.215151 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b41e8bf-bb7a-47ab-86ae-7e0077c702d7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:55:41.219223 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b41e8bf-bb7a-47ab-86ae-7e0077c702d7 admin admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:55:41.234447 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 962/1908] 10.222.0.20 () {58 vars in 1153 bytes} [Mon Aug 1 16:55:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 16:55:41.242761 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b41e8bf-bb7a-47ab-86ae-7e0077c702d7 admin admin] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:55:41.253742 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 947/1909] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 1 16:55:41 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 16:55:41.283148 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b41e8bf-bb7a-47ab-86ae-7e0077c702d7 admin admin] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:55:41.295831 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd053483-2de5-4abb-b4cb-5b2e3179eaa5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:55:41.296460 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd053483-2de5-4abb-b4cb-5b2e3179eaa5 admin admin] REQUEST_METHOD: `POST` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:55:41.297095 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd053483-2de5-4abb-b4cb-5b2e3179eaa5 admin admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:55:41.297664 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd053483-2de5-4abb-b4cb-5b2e3179eaa5 admin admin] PATH_INFO: `/v3/projects` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:55:41.298524 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd053483-2de5-4abb-b4cb-5b2e3179eaa5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:55:41.300412 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd053483-2de5-4abb-b4cb-5b2e3179eaa5 admin admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:55:41.312331 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9776a99-bd00-4faa-9917-30c6121d34e0 None None] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:55:41.314574 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.fernet_utils [None req-f9776a99-bd00-4faa-9917-30c6121d34e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87057) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:55:41.409504 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9776a99-bd00-4faa-9917-30c6121d34e0 None None] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:55:41.424430 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2ff7fd0-1e27-48b4-884b-f9a86a85dd62 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25970b52ca15448fa1abb1f974d64038', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b992d6a65df4bf9b27c13192d57c5f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:55:41.426152 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ff7fd0-1e27-48b4-884b-f9a86a85dd62 service cinder] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:55:41.426580 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ff7fd0-1e27-48b4-884b-f9a86a85dd62 service cinder] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:55:41.427026 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ff7fd0-1e27-48b4-884b-f9a86a85dd62 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:55:41.435718 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 963/1910] 10.222.0.20 () {60 vars in 1153 bytes} [Mon Aug 1 16:55:41 2022] POST /identity/v3/projects => generated 377 bytes in 198 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 16:55:41.436776 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd053483-2de5-4abb-b4cb-5b2e3179eaa5 admin admin] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:55:41.449230 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2ff7fd0-1e27-48b4-884b-f9a86a85dd62 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:55:41.451527 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2ff7fd0-1e27-48b4-884b-f9a86a85dd62 service cinder] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:55:41.487571 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 948/1911] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 1 16:55:41 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 176 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 16:55:41.523536 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG dbcounter [-] [87057] Writing DB stats keystone:SELECT=88,keystone:DELETE=7,keystone:INSERT=5 {{(pid=87057) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:55:41.544993 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd053483-2de5-4abb-b4cb-5b2e3179eaa5 admin admin] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:55:41.551821 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dbe07e1-7afd-4fe3-8705-9309d3298348 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:55:41.553345 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dbe07e1-7afd-4fe3-8705-9309d3298348 admin admin] REQUEST_METHOD: `POST` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:55:41.553741 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dbe07e1-7afd-4fe3-8705-9309d3298348 admin admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:55:41.554173 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dbe07e1-7afd-4fe3-8705-9309d3298348 admin admin] PATH_INFO: `/v3/users` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:55:41.554977 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dbe07e1-7afd-4fe3-8705-9309d3298348 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:55:41.558517 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dbe07e1-7afd-4fe3-8705-9309d3298348 admin admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:55:41.695665 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 964/1912] 10.222.0.20 () {60 vars in 1144 bytes} [Mon Aug 1 16:55:41 2022] POST /identity/v3/users => generated 289 bytes in 259 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 16:55:41.705162 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2ff7fd0-1e27-48b4-884b-f9a86a85dd62 service cinder] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:55:41.712876 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dbe07e1-7afd-4fe3-8705-9309d3298348 admin admin] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:55:41.718858 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2ff7fd0-1e27-48b4-884b-f9a86a85dd62 service cinder] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:55:41.725696 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e88058b-6ddd-4bd2-84eb-f8884ee4e90c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:55:41.726723 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e88058b-6ddd-4bd2-84eb-f8884ee4e90c admin admin] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:55:41.727076 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e88058b-6ddd-4bd2-84eb-f8884ee4e90c admin admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:55:41.727506 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e88058b-6ddd-4bd2-84eb-f8884ee4e90c admin admin] PATH_INFO: `/v3/roles` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:55:41.728929 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e88058b-6ddd-4bd2-84eb-f8884ee4e90c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:55:41.730576 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e88058b-6ddd-4bd2-84eb-f8884ee4e90c admin admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:55:41.742084 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 949/1913] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Aug 1 16:55:41 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 16:55:41.746254 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e88058b-6ddd-4bd2-84eb-f8884ee4e90c admin admin] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:55:41.760207 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e88058b-6ddd-4bd2-84eb-f8884ee4e90c admin admin] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:55:41.769974 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be14224f-daed-4f82-add8-9f3280f08fca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:55:41.770610 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be14224f-daed-4f82-add8-9f3280f08fca admin admin] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:55:41.770888 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be14224f-daed-4f82-add8-9f3280f08fca admin admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:55:41.771197 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be14224f-daed-4f82-add8-9f3280f08fca admin admin] PATH_INFO: `/v3/roles` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:55:41.771864 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be14224f-daed-4f82-add8-9f3280f08fca admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:55:41.773648 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be14224f-daed-4f82-add8-9f3280f08fca admin admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:55:41.781498 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dbe07e1-7afd-4fe3-8705-9309d3298348 admin admin] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:55:41.785471 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6adce25-f7a7-437c-b755-3c3b93f60790 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25970b52ca15448fa1abb1f974d64038', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b992d6a65df4bf9b27c13192d57c5f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:55:41.786137 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6adce25-f7a7-437c-b755-3c3b93f60790 service cinder] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:55:41.786453 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6adce25-f7a7-437c-b755-3c3b93f60790 service cinder] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:55:41.786632 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 950/1914] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Aug 1 16:55:41 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 16:55:41.786804 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6adce25-f7a7-437c-b755-3c3b93f60790 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:55:41.788763 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.fernet_utils [None req-f6adce25-f7a7-437c-b755-3c3b93f60790 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:55:41.792529 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be14224f-daed-4f82-add8-9f3280f08fca admin admin] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:55:41.810433 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be14224f-daed-4f82-add8-9f3280f08fca admin admin] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:55:41.823874 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b610a42f-b237-4787-8591-a8e26b0c41a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:55:41.824666 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b610a42f-b237-4787-8591-a8e26b0c41a1 admin admin] REQUEST_METHOD: `PUT` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:55:41.825058 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b610a42f-b237-4787-8591-a8e26b0c41a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:55:41.825443 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b610a42f-b237-4787-8591-a8e26b0c41a1 admin admin] PATH_INFO: `/v3/projects/6b4d40c3c1c8441ca60aa8df7bc1a5c4/users/d115e68b759b452f9482d1b759f75aac/roles/0999d3da63a545f18ff3252afdc90518` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:55:41.851579 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b610a42f-b237-4787-8591-a8e26b0c41a1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6b4d40c3c1c8441ca60aa8df7bc1a5c4, user_id=d115e68b759b452f9482d1b759f75aac, role_id=0999d3da63a545f18ff3252afdc90518)` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:55:41.855222 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b610a42f-b237-4787-8591-a8e26b0c41a1 admin admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:55:41.875605 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6adce25-f7a7-437c-b755-3c3b93f60790 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:55:41.877044 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6adce25-f7a7-437c-b755-3c3b93f60790 service cinder] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:55:41.886307 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 965/1915] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 1 16:55:41 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 174 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 01 16:55:41.965422 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 951/1916] 10.222.0.20 () {60 vars in 1483 bytes} [Mon Aug 1 16:55:41 2022] PUT /identity/v3/projects/6b4d40c3c1c8441ca60aa8df7bc1a5c4/users/d115e68b759b452f9482d1b759f75aac/roles/0999d3da63a545f18ff3252afdc90518 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 16:55:41.975357 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a21300c-f904-4130-b051-675810b23536 None None] REQUEST_METHOD: `POST` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:55:41.975643 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a21300c-f904-4130-b051-675810b23536 None None] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:55:41.975928 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a21300c-f904-4130-b051-675810b23536 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:55:42.007701 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.auth.core [None req-6a21300c-f904-4130-b051-675810b23536 None None] MFA Rules not processed for user `d115e68b759b452f9482d1b759f75aac`. Rule list: `[]` (Enabled: `True`). {{(pid=87058) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:55:42.008918 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.fernet_utils [None req-6a21300c-f904-4130-b051-675810b23536 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:55:42.077353 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 966/1917] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 1 16:55:41 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 16:55:42.082635 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b610a42f-b237-4787-8591-a8e26b0c41a1 admin admin] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:55:42.110954 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG dbcounter [-] [87058] Writing DB stats keystone:INSERT=4,keystone:SELECT=89,keystone:DELETE=7 {{(pid=87058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:55:42.194765 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b610a42f-b237-4787-8591-a8e26b0c41a1 admin admin] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:55:42.202425 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cff1de8-cf5d-430e-89ed-cbaa54d2b283 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:55:42.207180 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cff1de8-cf5d-430e-89ed-cbaa54d2b283 admin admin] REQUEST_METHOD: `POST` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:55:42.207180 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cff1de8-cf5d-430e-89ed-cbaa54d2b283 admin admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:55:42.207180 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cff1de8-cf5d-430e-89ed-cbaa54d2b283 admin admin] PATH_INFO: `/v3/projects` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:55:42.207180 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cff1de8-cf5d-430e-89ed-cbaa54d2b283 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:55:42.207180 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cff1de8-cf5d-430e-89ed-cbaa54d2b283 admin admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:55:42.375868 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 952/1918] 10.222.0.20 () {60 vars in 1153 bytes} [Mon Aug 1 16:55:42 2022] POST /identity/v3/projects => generated 379 bytes in 289 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 16:55:42.379147 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a21300c-f904-4130-b051-675810b23536 None None] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:55:42.392300 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a21300c-f904-4130-b051-675810b23536 None None] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:55:42.474024 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de437d2-c641-4caf-9fcf-308ac928023c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:55:42.474681 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de437d2-c641-4caf-9fcf-308ac928023c admin admin] REQUEST_METHOD: `POST` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:55:42.474962 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de437d2-c641-4caf-9fcf-308ac928023c admin admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:55:42.475237 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de437d2-c641-4caf-9fcf-308ac928023c admin admin] PATH_INFO: `/v3/users` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:55:42.475907 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0de437d2-c641-4caf-9fcf-308ac928023c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:55:42.477824 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0de437d2-c641-4caf-9fcf-308ac928023c admin admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:55:42.579140 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 967/1919] 10.222.0.20 () {60 vars in 1144 bytes} [Mon Aug 1 16:55:42 2022] POST /identity/v3/users => generated 296 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 16:55:42.582824 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cff1de8-cf5d-430e-89ed-cbaa54d2b283 admin admin] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:55:42.593649 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cff1de8-cf5d-430e-89ed-cbaa54d2b283 admin admin] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:55:42.598271 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e160bd-3ea5-49c5-a05a-24a0efc6309a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:55:42.598882 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e160bd-3ea5-49c5-a05a-24a0efc6309a admin admin] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:55:42.599164 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e160bd-3ea5-49c5-a05a-24a0efc6309a admin admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:55:42.599441 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e160bd-3ea5-49c5-a05a-24a0efc6309a admin admin] PATH_INFO: `/v3/roles` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:55:42.600240 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7e160bd-3ea5-49c5-a05a-24a0efc6309a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:55:42.602350 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7e160bd-3ea5-49c5-a05a-24a0efc6309a admin admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:55:42.620002 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 953/1920] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Aug 1 16:55:42 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 16:55:42.625219 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de437d2-c641-4caf-9fcf-308ac928023c admin admin] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:55:42.627198 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e160bd-3ea5-49c5-a05a-24a0efc6309a admin admin] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:55:42.636593 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de437d2-c641-4caf-9fcf-308ac928023c admin admin] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:55:42.638252 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e160bd-3ea5-49c5-a05a-24a0efc6309a admin admin] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:55:42.643133 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5406e3d0-76dc-4d91-8538-594a61f01e1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:55:42.643731 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b049fb78-fc70-4582-b2d4-d755678ca11a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:55:42.643852 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5406e3d0-76dc-4d91-8538-594a61f01e1b admin admin] REQUEST_METHOD: `DELETE` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:55:42.644047 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5406e3d0-76dc-4d91-8538-594a61f01e1b admin admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:55:42.644326 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5406e3d0-76dc-4d91-8538-594a61f01e1b admin admin] PATH_INFO: `/v3/users/feb90f07095846f1a26f3c2c23b50420` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:55:42.645335 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b049fb78-fc70-4582-b2d4-d755678ca11a admin admin] REQUEST_METHOD: `PUT` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:55:42.645520 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5406e3d0-76dc-4d91-8538-594a61f01e1b admin admin] RBAC: Authorizing `identity:delete_user(user_id=feb90f07095846f1a26f3c2c23b50420)` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:55:42.645973 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b049fb78-fc70-4582-b2d4-d755678ca11a admin admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:55:42.646516 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b049fb78-fc70-4582-b2d4-d755678ca11a admin admin] PATH_INFO: `/v3/projects/27be61b212e24589ad5a6034f0d205e0/users/d2fbbb9ad49c4133bac2d2f02feefa25/roles/440bb943dd5b46fea038b903b6eaaf81` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:55:42.647565 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5406e3d0-76dc-4d91-8538-594a61f01e1b admin admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:55:42.663144 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b049fb78-fc70-4582-b2d4-d755678ca11a admin admin] RBAC: Authorizing `identity:create_grant(project_id=27be61b212e24589ad5a6034f0d205e0, user_id=d2fbbb9ad49c4133bac2d2f02feefa25, role_id=440bb943dd5b46fea038b903b6eaaf81)` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:55:42.666525 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b049fb78-fc70-4582-b2d4-d755678ca11a admin admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:55:42.759717 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 968/1921] 10.222.0.20 () {60 vars in 1483 bytes} [Mon Aug 1 16:55:42 2022] PUT /identity/v3/projects/27be61b212e24589ad5a6034f0d205e0/users/d2fbbb9ad49c4133bac2d2f02feefa25/roles/440bb943dd5b46fea038b903b6eaaf81 => generated 0 bytes in 135 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 16:55:42.764254 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b049fb78-fc70-4582-b2d4-d755678ca11a admin admin] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:55:42.823681 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.notifications [None req-5406e3d0-76dc-4d91-8538-594a61f01e1b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'feb90f07095846f1a26f3c2c23b50420'} {{(pid=87057) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:55:42.824439 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.notifications [None req-5406e3d0-76dc-4d91-8538-594a61f01e1b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'feb90f07095846f1a26f3c2c23b50420'} {{(pid=87057) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:55:42.835695 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b049fb78-fc70-4582-b2d4-d755678ca11a admin admin] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:55:42.840995 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4591702a-f2f2-413d-9378-71cd5f8635f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:55:42.841674 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4591702a-f2f2-413d-9378-71cd5f8635f9 admin admin] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:55:42.844625 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4591702a-f2f2-413d-9378-71cd5f8635f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:55:42.844625 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4591702a-f2f2-413d-9378-71cd5f8635f9 admin admin] PATH_INFO: `/v3/roles` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:55:42.844625 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4591702a-f2f2-413d-9378-71cd5f8635f9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:55:42.845574 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4591702a-f2f2-413d-9378-71cd5f8635f9 admin admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:55:42.856547 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 969/1922] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Aug 1 16:55:42 2022] GET /identity/v3/roles => generated 1420 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 01 16:55:42.860552 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4591702a-f2f2-413d-9378-71cd5f8635f9 admin admin] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:55:42.864495 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG dbcounter [-] [87057] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87057) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:55:42.870663 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4591702a-f2f2-413d-9378-71cd5f8635f9 admin admin] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:55:42.875299 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5796abe8-3e4d-49ba-9ad4-e791a3663f23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:55:42.875932 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5796abe8-3e4d-49ba-9ad4-e791a3663f23 admin admin] REQUEST_METHOD: `PUT` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:55:42.876211 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5796abe8-3e4d-49ba-9ad4-e791a3663f23 admin admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:55:42.876482 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5796abe8-3e4d-49ba-9ad4-e791a3663f23 admin admin] PATH_INFO: `/v3/projects/27be61b212e24589ad5a6034f0d205e0/users/d2fbbb9ad49c4133bac2d2f02feefa25/roles/440bb943dd5b46fea038b903b6eaaf81` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:55:42.878446 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5796abe8-3e4d-49ba-9ad4-e791a3663f23 admin admin] RBAC: Authorizing `identity:create_grant(project_id=27be61b212e24589ad5a6034f0d205e0, user_id=d2fbbb9ad49c4133bac2d2f02feefa25, role_id=440bb943dd5b46fea038b903b6eaaf81)` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:55:42.881046 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5796abe8-3e4d-49ba-9ad4-e791a3663f23 admin admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:55:42.891301 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 970/1923] 10.222.0.20 () {60 vars in 1483 bytes} [Mon Aug 1 16:55:42 2022] PUT /identity/v3/projects/27be61b212e24589ad5a6034f0d205e0/users/d2fbbb9ad49c4133bac2d2f02feefa25/roles/440bb943dd5b46fea038b903b6eaaf81 => generated 0 bytes in 31 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 16:55:42.892624 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.notifications [None req-5406e3d0-76dc-4d91-8538-594a61f01e1b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'feb90f07095846f1a26f3c2c23b50420'} {{(pid=87057) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:55:42.893255 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.notifications [None req-5406e3d0-76dc-4d91-8538-594a61f01e1b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'feb90f07095846f1a26f3c2c23b50420'} {{(pid=87057) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:55:42.901242 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.notifications [None req-5406e3d0-76dc-4d91-8538-594a61f01e1b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'feb90f07095846f1a26f3c2c23b50420'} {{(pid=87057) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:55:42.902876 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6c7605-60b0-464b-9416-ee0ef9b84dd8 None None] REQUEST_METHOD: `POST` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:55:42.903499 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6c7605-60b0-464b-9416-ee0ef9b84dd8 None None] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:55:42.903499 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6c7605-60b0-464b-9416-ee0ef9b84dd8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:55:42.922010 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 954/1924] 10.222.0.20 () {58 vars in 1224 bytes} [Mon Aug 1 16:55:42 2022] DELETE /identity/v3/users/feb90f07095846f1a26f3c2c23b50420 => generated 0 bytes in 295 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 16:55:42.926421 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1bf7ab6-da31-4a37-82fc-baad4945c400 None None] REQUEST_METHOD: `POST` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:55:42.926772 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1bf7ab6-da31-4a37-82fc-baad4945c400 None None] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:55:42.927536 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1bf7ab6-da31-4a37-82fc-baad4945c400 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:55:42.935610 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.auth.core [None req-2d6c7605-60b0-464b-9416-ee0ef9b84dd8 None None] MFA Rules not processed for user `d2fbbb9ad49c4133bac2d2f02feefa25`. Rule list: `[]` (Enabled: `True`). {{(pid=87058) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:55:42.936793 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.fernet_utils [None req-2d6c7605-60b0-464b-9416-ee0ef9b84dd8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:55:42.947670 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.auth.core [None req-a1bf7ab6-da31-4a37-82fc-baad4945c400 None None] MFA Rules not processed for user `1007f8e379b74e169b98fb7c3c0ba5b4`. Rule list: `[]` (Enabled: `True`). {{(pid=87057) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:55:42.948870 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.fernet_utils [None req-a1bf7ab6-da31-4a37-82fc-baad4945c400 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87057) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:55:43.007993 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 971/1925] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 1 16:55:42 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:55:43.020158 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d6c7605-60b0-464b-9416-ee0ef9b84dd8 None None] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:55:43.021544 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.fernet_utils [None req-2d6c7605-60b0-464b-9416-ee0ef9b84dd8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:55:43.026522 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG dbcounter [-] [87058] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:55:43.032349 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 955/1926] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 1 16:55:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:55:43.044494 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1bf7ab6-da31-4a37-82fc-baad4945c400 None None] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:55:43.046374 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.fernet_utils [None req-a1bf7ab6-da31-4a37-82fc-baad4945c400 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87057) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:55:43.082624 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG dbcounter [-] [87057] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=87057) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:55:43.086279 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d6c7605-60b0-464b-9416-ee0ef9b84dd8 None None] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:55:43.091268 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cb65baa-880f-4848-8fdb-c151460f4039 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25970b52ca15448fa1abb1f974d64038', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b992d6a65df4bf9b27c13192d57c5f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:55:43.091906 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb65baa-880f-4848-8fdb-c151460f4039 service cinder] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:55:43.092190 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb65baa-880f-4848-8fdb-c151460f4039 service cinder] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:55:43.092941 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb65baa-880f-4848-8fdb-c151460f4039 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:55:43.107389 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cb65baa-880f-4848-8fdb-c151460f4039 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:55:43.109467 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cb65baa-880f-4848-8fdb-c151460f4039 service cinder] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:55:43.112866 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1bf7ab6-da31-4a37-82fc-baad4945c400 None None] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:55:43.117040 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd4147b-713f-49d0-bfd8-d11f63f233ce service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d9bd8d649154c979be3744bfd016f95', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b992d6a65df4bf9b27c13192d57c5f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:55:43.121597 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd4147b-713f-49d0-bfd8-d11f63f233ce service neutron] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:55:43.121597 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd4147b-713f-49d0-bfd8-d11f63f233ce service neutron] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:55:43.121597 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd4147b-713f-49d0-bfd8-d11f63f233ce service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:55:43.128927 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 972/1927] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 1 16:55:43 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 01 16:55:43.136322 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bd4147b-713f-49d0-bfd8-d11f63f233ce service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:55:43.137821 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bd4147b-713f-49d0-bfd8-d11f63f233ce service neutron] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:55:43.148014 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 956/1928] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 1 16:55:43 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 01 16:55:43.203317 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG dbcounter [-] [87058] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=87058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:55:43.259589 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG dbcounter [-] [87057] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87057) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:55:43.586076 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cb65baa-880f-4848-8fdb-c151460f4039 service cinder] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:55:43.587532 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.fernet_utils [None req-6cb65baa-880f-4848-8fdb-c151460f4039 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:55:43.601658 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cb65baa-880f-4848-8fdb-c151460f4039 service cinder] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:55:43.607303 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa64b245-31d0-47c1-af76-204059076f0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:55:43.609377 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa64b245-31d0-47c1-af76-204059076f0e admin admin] REQUEST_METHOD: `DELETE` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:55:43.609377 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa64b245-31d0-47c1-af76-204059076f0e admin admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:55:43.609658 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa64b245-31d0-47c1-af76-204059076f0e admin admin] PATH_INFO: `/v3/projects/3a97c42aae924d89b5a59d95daf88738` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:55:43.611032 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa64b245-31d0-47c1-af76-204059076f0e admin admin] RBAC: Authorizing `identity:delete_project(project_id=3a97c42aae924d89b5a59d95daf88738)` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:55:43.613317 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa64b245-31d0-47c1-af76-204059076f0e admin admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:55:43.693667 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG dbcounter [-] [87057] Writing DB stats keystone:SELECT=87,keystone:INSERT=3,keystone:DELETE=10 {{(pid=87057) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:55:43.711144 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.notifications [None req-fa64b245-31d0-47c1-af76-204059076f0e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3a97c42aae924d89b5a59d95daf88738', 'request_id': 'req-fa64b245-31d0-47c1-af76-204059076f0e'} {{(pid=87058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:55:43.717890 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.notifications [None req-fa64b245-31d0-47c1-af76-204059076f0e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3a97c42aae924d89b5a59d95daf88738', 'request_id': 'req-fa64b245-31d0-47c1-af76-204059076f0e'} {{(pid=87058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:55:43.723550 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.notifications [None req-fa64b245-31d0-47c1-af76-204059076f0e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3a97c42aae924d89b5a59d95daf88738', 'request_id': 'req-fa64b245-31d0-47c1-af76-204059076f0e'} {{(pid=87058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:55:43.801450 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.notifications [None req-fa64b245-31d0-47c1-af76-204059076f0e admin admin] The token cache is being invalidate because project 3a97c42aae924d89b5a59d95daf88738 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87058) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 16:55:43.801743 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.notifications [None req-fa64b245-31d0-47c1-af76-204059076f0e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:55:43.806201 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.notifications [None req-fa64b245-31d0-47c1-af76-204059076f0e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:55:43.806201 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 973/1929] 10.222.0.20 () {58 vars in 1233 bytes} [Mon Aug 1 16:55:43 2022] DELETE /identity/v3/projects/3a97c42aae924d89b5a59d95daf88738 => generated 0 bytes in 218 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 16:55:43.807987 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd4147b-713f-49d0-bfd8-d11f63f233ce service neutron] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:55:43.817446 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd4147b-713f-49d0-bfd8-d11f63f233ce service neutron] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:55:43.882771 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d19acb-183e-44b5-bdcc-607943728944 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:55:43.886621 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d19acb-183e-44b5-bdcc-607943728944 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:55:43.886621 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d19acb-183e-44b5-bdcc-607943728944 admin admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:55:43.886621 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d19acb-183e-44b5-bdcc-607943728944 admin admin] PATH_INFO: `/v3/users/4e5312336075410e8ad0e1e93461628e` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:55:43.886621 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82d19acb-183e-44b5-bdcc-607943728944 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4e5312336075410e8ad0e1e93461628e)` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:55:43.888951 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82d19acb-183e-44b5-bdcc-607943728944 admin admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:55:44.139487 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa64b245-31d0-47c1-af76-204059076f0e admin admin] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:55:44.142273 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.fernet_utils [None req-fa64b245-31d0-47c1-af76-204059076f0e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:55:44.211951 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.notifications [None req-82d19acb-183e-44b5-bdcc-607943728944 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4e5312336075410e8ad0e1e93461628e'} {{(pid=87057) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:55:44.213242 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.notifications [None req-82d19acb-183e-44b5-bdcc-607943728944 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4e5312336075410e8ad0e1e93461628e'} {{(pid=87057) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:55:44.216104 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa64b245-31d0-47c1-af76-204059076f0e admin admin] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:55:44.221048 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2774c45a-1293-4d9f-8ade-a068f409b48c tempest-VolumeQuotaClassesTest-580713274 tempest-VolumeQuotaClassesTest-580713274-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2fbbb9ad49c4133bac2d2f02feefa25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27be61b212e24589ad5a6034f0d205e0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:55:44.221674 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2774c45a-1293-4d9f-8ade-a068f409b48c tempest-VolumeQuotaClassesTest-580713274 tempest-VolumeQuotaClassesTest-580713274-project-admin] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:55:44.221961 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2774c45a-1293-4d9f-8ade-a068f409b48c tempest-VolumeQuotaClassesTest-580713274 tempest-VolumeQuotaClassesTest-580713274-project-admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:55:44.222375 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2774c45a-1293-4d9f-8ade-a068f409b48c tempest-VolumeQuotaClassesTest-580713274 tempest-VolumeQuotaClassesTest-580713274-project-admin] PATH_INFO: `/v3/domains` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:55:44.223062 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2774c45a-1293-4d9f-8ade-a068f409b48c tempest-VolumeQuotaClassesTest-580713274 tempest-VolumeQuotaClassesTest-580713274-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=87058) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:55:44.223434 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2774c45a-1293-4d9f-8ade-a068f409b48c tempest-VolumeQuotaClassesTest-580713274 tempest-VolumeQuotaClassesTest-580713274-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:55:44.225076 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2774c45a-1293-4d9f-8ade-a068f409b48c tempest-VolumeQuotaClassesTest-580713274 tempest-VolumeQuotaClassesTest-580713274-project-admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:55:44.235776 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG dbcounter [-] [87058] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=87058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:55:44.239240 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 974/1930] 10.222.0.20 () {58 vars in 1153 bytes} [Mon Aug 1 16:55:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 16:55:44.243982 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2774c45a-1293-4d9f-8ade-a068f409b48c tempest-VolumeQuotaClassesTest-580713274 tempest-VolumeQuotaClassesTest-580713274-project-admin] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:55:44.254259 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2774c45a-1293-4d9f-8ade-a068f409b48c tempest-VolumeQuotaClassesTest-580713274 tempest-VolumeQuotaClassesTest-580713274-project-admin] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:55:44.260356 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-120354de-61fa-410d-ae2d-422b702b9b13 tempest-VolumeQuotaClassesTest-580713274 tempest-VolumeQuotaClassesTest-580713274-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2fbbb9ad49c4133bac2d2f02feefa25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27be61b212e24589ad5a6034f0d205e0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:55:44.260865 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120354de-61fa-410d-ae2d-422b702b9b13 tempest-VolumeQuotaClassesTest-580713274 tempest-VolumeQuotaClassesTest-580713274-project-admin] REQUEST_METHOD: `POST` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:55:44.261150 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120354de-61fa-410d-ae2d-422b702b9b13 tempest-VolumeQuotaClassesTest-580713274 tempest-VolumeQuotaClassesTest-580713274-project-admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:55:44.261431 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120354de-61fa-410d-ae2d-422b702b9b13 tempest-VolumeQuotaClassesTest-580713274 tempest-VolumeQuotaClassesTest-580713274-project-admin] PATH_INFO: `/v3/projects` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:55:44.262137 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-120354de-61fa-410d-ae2d-422b702b9b13 tempest-VolumeQuotaClassesTest-580713274 tempest-VolumeQuotaClassesTest-580713274-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:55:44.264104 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-120354de-61fa-410d-ae2d-422b702b9b13 tempest-VolumeQuotaClassesTest-580713274 tempest-VolumeQuotaClassesTest-580713274-project-admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:55:44.367905 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.notifications [None req-82d19acb-183e-44b5-bdcc-607943728944 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4e5312336075410e8ad0e1e93461628e'} {{(pid=87057) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:55:44.371720 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.notifications [None req-82d19acb-183e-44b5-bdcc-607943728944 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4e5312336075410e8ad0e1e93461628e'} {{(pid=87057) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:55:44.378989 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.notifications [None req-82d19acb-183e-44b5-bdcc-607943728944 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4e5312336075410e8ad0e1e93461628e'} {{(pid=87057) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:55:44.397476 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 957/1931] 10.222.0.20 () {58 vars in 1224 bytes} [Mon Aug 1 16:55:43 2022] DELETE /identity/v3/users/4e5312336075410e8ad0e1e93461628e => generated 0 bytes in 590 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 16:55:44.434058 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 975/1932] 10.222.0.20 () {60 vars in 1153 bytes} [Mon Aug 1 16:55:44 2022] POST /identity/v3/projects => generated 354 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 16:55:44.434812 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d19acb-183e-44b5-bdcc-607943728944 admin admin] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:55:44.436895 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.fernet_utils [None req-82d19acb-183e-44b5-bdcc-607943728944 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87057) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:55:44.549816 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d19acb-183e-44b5-bdcc-607943728944 admin admin] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:55:44.626707 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff4fd219-7551-4407-8ad0-fbdf617a44ac tempest-VolumeQuotaClassesTest-580713274 tempest-VolumeQuotaClassesTest-580713274-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2fbbb9ad49c4133bac2d2f02feefa25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27be61b212e24589ad5a6034f0d205e0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:55:44.627334 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4fd219-7551-4407-8ad0-fbdf617a44ac tempest-VolumeQuotaClassesTest-580713274 tempest-VolumeQuotaClassesTest-580713274-project-admin] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:55:44.627619 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4fd219-7551-4407-8ad0-fbdf617a44ac tempest-VolumeQuotaClassesTest-580713274 tempest-VolumeQuotaClassesTest-580713274-project-admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:55:44.627904 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4fd219-7551-4407-8ad0-fbdf617a44ac tempest-VolumeQuotaClassesTest-580713274 tempest-VolumeQuotaClassesTest-580713274-project-admin] PATH_INFO: `/v3/domains` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:55:44.628703 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff4fd219-7551-4407-8ad0-fbdf617a44ac tempest-VolumeQuotaClassesTest-580713274 tempest-VolumeQuotaClassesTest-580713274-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=87057) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:55:44.629008 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff4fd219-7551-4407-8ad0-fbdf617a44ac tempest-VolumeQuotaClassesTest-580713274 tempest-VolumeQuotaClassesTest-580713274-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:55:44.630637 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff4fd219-7551-4407-8ad0-fbdf617a44ac tempest-VolumeQuotaClassesTest-580713274 tempest-VolumeQuotaClassesTest-580713274-project-admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:55:44.642755 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 958/1933] 10.222.0.20 () {58 vars in 1153 bytes} [Mon Aug 1 16:55:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 16:55:44.664943 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG dbcounter [-] [87057] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=87057) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:55:44.680826 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-120354de-61fa-410d-ae2d-422b702b9b13 tempest-VolumeQuotaClassesTest-580713274 tempest-VolumeQuotaClassesTest-580713274-project-admin] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:55:44.693079 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-120354de-61fa-410d-ae2d-422b702b9b13 tempest-VolumeQuotaClassesTest-580713274 tempest-VolumeQuotaClassesTest-580713274-project-admin] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:55:44.699641 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0bfab45-cbd5-485a-96f4-97a88ebfd77a tempest-VolumeQuotaClassesTest-580713274 tempest-VolumeQuotaClassesTest-580713274-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2fbbb9ad49c4133bac2d2f02feefa25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27be61b212e24589ad5a6034f0d205e0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:55:44.700305 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0bfab45-cbd5-485a-96f4-97a88ebfd77a tempest-VolumeQuotaClassesTest-580713274 tempest-VolumeQuotaClassesTest-580713274-project-admin] REQUEST_METHOD: `DELETE` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:55:44.700591 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0bfab45-cbd5-485a-96f4-97a88ebfd77a tempest-VolumeQuotaClassesTest-580713274 tempest-VolumeQuotaClassesTest-580713274-project-admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:55:44.700868 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0bfab45-cbd5-485a-96f4-97a88ebfd77a tempest-VolumeQuotaClassesTest-580713274 tempest-VolumeQuotaClassesTest-580713274-project-admin] PATH_INFO: `/v3/projects/5bb386c4be9442be9724f35137b4d3b4` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:55:44.702057 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0bfab45-cbd5-485a-96f4-97a88ebfd77a tempest-VolumeQuotaClassesTest-580713274 tempest-VolumeQuotaClassesTest-580713274-project-admin] RBAC: Authorizing `identity:delete_project(project_id=5bb386c4be9442be9724f35137b4d3b4)` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:55:44.703971 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0bfab45-cbd5-485a-96f4-97a88ebfd77a tempest-VolumeQuotaClassesTest-580713274 tempest-VolumeQuotaClassesTest-580713274-project-admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:55:44.825537 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.notifications [None req-f0bfab45-cbd5-485a-96f4-97a88ebfd77a tempest-VolumeQuotaClassesTest-580713274 tempest-VolumeQuotaClassesTest-580713274-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5bb386c4be9442be9724f35137b4d3b4', 'request_id': 'req-f0bfab45-cbd5-485a-96f4-97a88ebfd77a'} {{(pid=87058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:55:44.832780 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.notifications [None req-f0bfab45-cbd5-485a-96f4-97a88ebfd77a tempest-VolumeQuotaClassesTest-580713274 tempest-VolumeQuotaClassesTest-580713274-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5bb386c4be9442be9724f35137b4d3b4', 'request_id': 'req-f0bfab45-cbd5-485a-96f4-97a88ebfd77a'} {{(pid=87058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:55:44.838560 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.notifications [None req-f0bfab45-cbd5-485a-96f4-97a88ebfd77a tempest-VolumeQuotaClassesTest-580713274 tempest-VolumeQuotaClassesTest-580713274-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5bb386c4be9442be9724f35137b4d3b4', 'request_id': 'req-f0bfab45-cbd5-485a-96f4-97a88ebfd77a'} {{(pid=87058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:55:44.894411 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff4fd219-7551-4407-8ad0-fbdf617a44ac tempest-VolumeQuotaClassesTest-580713274 tempest-VolumeQuotaClassesTest-580713274-project-admin] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:55:44.896167 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.fernet_utils [None req-ff4fd219-7551-4407-8ad0-fbdf617a44ac tempest-VolumeQuotaClassesTest-580713274 tempest-VolumeQuotaClassesTest-580713274-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87057) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:55:44.923475 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.notifications [None req-f0bfab45-cbd5-485a-96f4-97a88ebfd77a tempest-VolumeQuotaClassesTest-580713274 tempest-VolumeQuotaClassesTest-580713274-project-admin] The token cache is being invalidate because project 5bb386c4be9442be9724f35137b4d3b4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87058) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 16:55:44.923906 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.notifications [None req-f0bfab45-cbd5-485a-96f4-97a88ebfd77a tempest-VolumeQuotaClassesTest-580713274 tempest-VolumeQuotaClassesTest-580713274-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:55:44.935600 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.notifications [None req-f0bfab45-cbd5-485a-96f4-97a88ebfd77a tempest-VolumeQuotaClassesTest-580713274 tempest-VolumeQuotaClassesTest-580713274-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:55:44.935600 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 976/1934] 10.222.0.20 () {58 vars in 1233 bytes} [Mon Aug 1 16:55:44 2022] DELETE /identity/v3/projects/5bb386c4be9442be9724f35137b4d3b4 => generated 0 bytes in 247 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 16:55:44.972251 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff4fd219-7551-4407-8ad0-fbdf617a44ac tempest-VolumeQuotaClassesTest-580713274 tempest-VolumeQuotaClassesTest-580713274-project-admin] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:55:45.032040 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-429419b4-9eb8-458a-b3f9-a8eda9820127 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:55:45.032975 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429419b4-9eb8-458a-b3f9-a8eda9820127 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:55:45.033437 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429419b4-9eb8-458a-b3f9-a8eda9820127 admin admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:55:45.033859 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429419b4-9eb8-458a-b3f9-a8eda9820127 admin admin] PATH_INFO: `/v3/projects/c5848bd3203548e59309a80c31a72350` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:55:45.035568 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-429419b4-9eb8-458a-b3f9-a8eda9820127 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c5848bd3203548e59309a80c31a72350)` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:55:45.037787 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-429419b4-9eb8-458a-b3f9-a8eda9820127 admin admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:55:45.072351 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG dbcounter [-] [87057] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=87057) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:55:45.208593 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.notifications [None req-429419b4-9eb8-458a-b3f9-a8eda9820127 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c5848bd3203548e59309a80c31a72350', 'request_id': 'req-429419b4-9eb8-458a-b3f9-a8eda9820127'} {{(pid=87057) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:55:45.215621 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.notifications [None req-429419b4-9eb8-458a-b3f9-a8eda9820127 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c5848bd3203548e59309a80c31a72350', 'request_id': 'req-429419b4-9eb8-458a-b3f9-a8eda9820127'} {{(pid=87057) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:55:45.221242 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.notifications [None req-429419b4-9eb8-458a-b3f9-a8eda9820127 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c5848bd3203548e59309a80c31a72350', 'request_id': 'req-429419b4-9eb8-458a-b3f9-a8eda9820127'} {{(pid=87057) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:55:45.298241 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.notifications [None req-429419b4-9eb8-458a-b3f9-a8eda9820127 admin admin] The token cache is being invalidate because project c5848bd3203548e59309a80c31a72350 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87057) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 16:55:45.298558 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.notifications [None req-429419b4-9eb8-458a-b3f9-a8eda9820127 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87057) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:55:45.299189 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.notifications [None req-429419b4-9eb8-458a-b3f9-a8eda9820127 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87057) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:55:45.300742 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 959/1935] 10.222.0.20 () {58 vars in 1233 bytes} [Mon Aug 1 16:55:44 2022] DELETE /identity/v3/projects/c5848bd3203548e59309a80c31a72350 => generated 0 bytes in 407 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 16:55:45.305764 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883aad7f-9d4a-4792-959b-68e5c877d2d2 None None] REQUEST_METHOD: `POST` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:55:45.306271 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883aad7f-9d4a-4792-959b-68e5c877d2d2 None None] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:55:45.306380 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883aad7f-9d4a-4792-959b-68e5c877d2d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:55:45.328170 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.auth.core [None req-883aad7f-9d4a-4792-959b-68e5c877d2d2 None None] MFA Rules not processed for user `1007f8e379b74e169b98fb7c3c0ba5b4`. Rule list: `[]` (Enabled: `True`). {{(pid=87058) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:55:45.332145 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.fernet_utils [None req-883aad7f-9d4a-4792-959b-68e5c877d2d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:55:45.429344 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 977/1936] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 1 16:55:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:55:45.430025 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d21c43a-1162-4371-82e4-54860d3b31f5 None None] REQUEST_METHOD: `POST` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:55:45.430517 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d21c43a-1162-4371-82e4-54860d3b31f5 None None] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:55:45.431020 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d21c43a-1162-4371-82e4-54860d3b31f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:55:45.452100 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.auth.core [None req-1d21c43a-1162-4371-82e4-54860d3b31f5 None None] MFA Rules not processed for user `1007f8e379b74e169b98fb7c3c0ba5b4`. Rule list: `[]` (Enabled: `True`). {{(pid=87057) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:55:45.453348 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.fernet_utils [None req-1d21c43a-1162-4371-82e4-54860d3b31f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87057) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:55:45.461956 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 960/1937] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 1 16:55:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:55:45.466935 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-883aad7f-9d4a-4792-959b-68e5c877d2d2 None None] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:55:45.477220 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-883aad7f-9d4a-4792-959b-68e5c877d2d2 None None] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:55:45.485101 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d4442d6-d341-421a-9772-e04b8e1953df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:55:45.485771 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4442d6-d341-421a-9772-e04b8e1953df admin admin] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:55:45.486380 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4442d6-d341-421a-9772-e04b8e1953df admin admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:55:45.486614 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4442d6-d341-421a-9772-e04b8e1953df admin admin] PATH_INFO: `/v3/domains` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:55:45.487341 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d4442d6-d341-421a-9772-e04b8e1953df admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87058) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:55:45.487685 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d4442d6-d341-421a-9772-e04b8e1953df admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:55:45.489297 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d4442d6-d341-421a-9772-e04b8e1953df admin admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:55:45.502955 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 978/1938] 10.222.0.20 () {58 vars in 1153 bytes} [Mon Aug 1 16:55:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:55:45.509011 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d21c43a-1162-4371-82e4-54860d3b31f5 None None] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:55:45.519574 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d21c43a-1162-4371-82e4-54860d3b31f5 None None] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:55:45.525515 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a670e22-ce4c-4bb1-92b1-142ec8583c73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:55:45.526291 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a670e22-ce4c-4bb1-92b1-142ec8583c73 admin admin] REQUEST_METHOD: `POST` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:55:45.526575 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a670e22-ce4c-4bb1-92b1-142ec8583c73 admin admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:55:45.527210 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a670e22-ce4c-4bb1-92b1-142ec8583c73 admin admin] PATH_INFO: `/v3/projects` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:55:45.527534 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a670e22-ce4c-4bb1-92b1-142ec8583c73 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:55:45.529493 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a670e22-ce4c-4bb1-92b1-142ec8583c73 admin admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:55:45.536916 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d4442d6-d341-421a-9772-e04b8e1953df admin admin] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:55:45.538745 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.fernet_utils [None req-0d4442d6-d341-421a-9772-e04b8e1953df admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:55:45.552896 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d4442d6-d341-421a-9772-e04b8e1953df admin admin] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:55:45.558131 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c0dcee-be54-462a-a663-2213bf193e08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:55:45.558795 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c0dcee-be54-462a-a663-2213bf193e08 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:55:45.559072 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c0dcee-be54-462a-a663-2213bf193e08 admin admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:55:45.559343 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c0dcee-be54-462a-a663-2213bf193e08 admin admin] PATH_INFO: `/v3/users/d115e68b759b452f9482d1b759f75aac` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:55:45.560573 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7c0dcee-be54-462a-a663-2213bf193e08 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d115e68b759b452f9482d1b759f75aac)` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:55:45.562559 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7c0dcee-be54-462a-a663-2213bf193e08 admin admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:55:45.598341 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 961/1939] 10.222.0.20 () {60 vars in 1153 bytes} [Mon Aug 1 16:55:45 2022] POST /identity/v3/projects => generated 383 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 16:55:45.601563 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a670e22-ce4c-4bb1-92b1-142ec8583c73 admin admin] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:55:45.672742 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a670e22-ce4c-4bb1-92b1-142ec8583c73 admin admin] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:55:45.673452 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60894f55-0e72-49d0-ba6c-612715aca980 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:55:45.674098 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60894f55-0e72-49d0-ba6c-612715aca980 admin admin] REQUEST_METHOD: `POST` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:55:45.674383 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60894f55-0e72-49d0-ba6c-612715aca980 admin admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:55:45.674661 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60894f55-0e72-49d0-ba6c-612715aca980 admin admin] PATH_INFO: `/v3/users` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:55:45.675328 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60894f55-0e72-49d0-ba6c-612715aca980 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:55:45.677337 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60894f55-0e72-49d0-ba6c-612715aca980 admin admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:55:45.727925 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG dbcounter [-] [87058] Writing DB stats keystone:SELECT=85,keystone:INSERT=2,keystone:DELETE=13 {{(pid=87058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:55:45.730955 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.notifications [None req-b7c0dcee-be54-462a-a663-2213bf193e08 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd115e68b759b452f9482d1b759f75aac'} {{(pid=87058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:55:45.731633 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.notifications [None req-b7c0dcee-be54-462a-a663-2213bf193e08 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd115e68b759b452f9482d1b759f75aac'} {{(pid=87058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:55:45.732259 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.notifications [None req-b7c0dcee-be54-462a-a663-2213bf193e08 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd115e68b759b452f9482d1b759f75aac'} {{(pid=87058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:55:45.744958 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.notifications [None req-b7c0dcee-be54-462a-a663-2213bf193e08 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd115e68b759b452f9482d1b759f75aac'} {{(pid=87058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:55:45.745825 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 962/1940] 10.222.0.20 () {60 vars in 1144 bytes} [Mon Aug 1 16:55:45 2022] POST /identity/v3/users => generated 292 bytes in 145 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 16:55:45.750348 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60894f55-0e72-49d0-ba6c-612715aca980 admin admin] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:55:45.751946 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.fernet_utils [None req-60894f55-0e72-49d0-ba6c-612715aca980 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87057) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:55:45.752058 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.notifications [None req-b7c0dcee-be54-462a-a663-2213bf193e08 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd115e68b759b452f9482d1b759f75aac'} {{(pid=87058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:55:45.764037 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60894f55-0e72-49d0-ba6c-612715aca980 admin admin] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:55:45.768751 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ed1987-a278-496e-acd2-fd981b2f41b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:55:45.769354 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ed1987-a278-496e-acd2-fd981b2f41b1 admin admin] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:55:45.769632 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ed1987-a278-496e-acd2-fd981b2f41b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:55:45.769904 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ed1987-a278-496e-acd2-fd981b2f41b1 admin admin] PATH_INFO: `/v3/roles` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:55:45.770633 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8ed1987-a278-496e-acd2-fd981b2f41b1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:55:45.772224 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8ed1987-a278-496e-acd2-fd981b2f41b1 admin admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:55:45.782112 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 963/1941] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Aug 1 16:55:45 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 01 16:55:45.789947 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ed1987-a278-496e-acd2-fd981b2f41b1 admin admin] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:55:45.796493 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG dbcounter [-] [87057] Writing DB stats keystone:SELECT=88,keystone:DELETE=7,keystone:INSERT=5 {{(pid=87057) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:55:45.800209 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ed1987-a278-496e-acd2-fd981b2f41b1 admin admin] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:55:45.804814 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd0530a8-283f-4479-ae0a-05d9fbc971ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:55:45.805398 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0530a8-283f-4479-ae0a-05d9fbc971ec admin admin] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:55:45.805679 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0530a8-283f-4479-ae0a-05d9fbc971ec admin admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:55:45.806703 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0530a8-283f-4479-ae0a-05d9fbc971ec admin admin] PATH_INFO: `/v3/roles` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:55:45.807394 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd0530a8-283f-4479-ae0a-05d9fbc971ec admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:55:45.808949 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd0530a8-283f-4479-ae0a-05d9fbc971ec admin admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:55:45.819342 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 964/1942] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Aug 1 16:55:45 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 01 16:55:45.823286 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd0530a8-283f-4479-ae0a-05d9fbc971ec admin admin] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:55:45.832567 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd0530a8-283f-4479-ae0a-05d9fbc971ec admin admin] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:55:45.837447 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df036162-3ecd-4c8b-8dcc-a9db5dc99d48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:55:45.838118 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df036162-3ecd-4c8b-8dcc-a9db5dc99d48 admin admin] REQUEST_METHOD: `PUT` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:55:45.838400 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df036162-3ecd-4c8b-8dcc-a9db5dc99d48 admin admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:55:45.838680 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df036162-3ecd-4c8b-8dcc-a9db5dc99d48 admin admin] PATH_INFO: `/v3/projects/3934f1fe56ee4c08b86dbecfd42639b1/users/ee6f46b57c0d4350babc45b5aa3e78d4/roles/0999d3da63a545f18ff3252afdc90518` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:55:45.841658 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 979/1943] 10.222.0.20 () {58 vars in 1224 bytes} [Mon Aug 1 16:55:45 2022] DELETE /identity/v3/users/d115e68b759b452f9482d1b759f75aac => generated 0 bytes in 305 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 16:55:45.854381 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df036162-3ecd-4c8b-8dcc-a9db5dc99d48 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3934f1fe56ee4c08b86dbecfd42639b1, user_id=ee6f46b57c0d4350babc45b5aa3e78d4, role_id=0999d3da63a545f18ff3252afdc90518)` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:55:45.855020 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53c5a26-170f-47c4-9f38-1f333848572c None None] REQUEST_METHOD: `POST` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:55:45.855362 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53c5a26-170f-47c4-9f38-1f333848572c None None] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:55:45.855640 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53c5a26-170f-47c4-9f38-1f333848572c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:55:45.857177 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df036162-3ecd-4c8b-8dcc-a9db5dc99d48 admin admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:55:45.874745 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.auth.core [None req-e53c5a26-170f-47c4-9f38-1f333848572c None None] MFA Rules not processed for user `1007f8e379b74e169b98fb7c3c0ba5b4`. Rule list: `[]` (Enabled: `True`). {{(pid=87058) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:55:45.875949 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.fernet_utils [None req-e53c5a26-170f-47c4-9f38-1f333848572c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:55:45.933904 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 965/1944] 10.222.0.20 () {60 vars in 1483 bytes} [Mon Aug 1 16:55:45 2022] PUT /identity/v3/projects/3934f1fe56ee4c08b86dbecfd42639b1/users/ee6f46b57c0d4350babc45b5aa3e78d4/roles/0999d3da63a545f18ff3252afdc90518 => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 16:55:45.935913 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 980/1945] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 1 16:55:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 16:55:45.945273 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df036162-3ecd-4c8b-8dcc-a9db5dc99d48 admin admin] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:55:45.947032 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.fernet_utils [None req-df036162-3ecd-4c8b-8dcc-a9db5dc99d48 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87057) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:55:45.947951 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba10b3a-0d95-409e-8cae-c30f6005fed1 None None] REQUEST_METHOD: `POST` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:55:45.948240 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba10b3a-0d95-409e-8cae-c30f6005fed1 None None] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:55:45.948723 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba10b3a-0d95-409e-8cae-c30f6005fed1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:55:45.975681 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.auth.core [None req-fba10b3a-0d95-409e-8cae-c30f6005fed1 None None] MFA Rules not processed for user `ee6f46b57c0d4350babc45b5aa3e78d4`. Rule list: `[]` (Enabled: `True`). {{(pid=87058) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:55:45.976903 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.fernet_utils [None req-fba10b3a-0d95-409e-8cae-c30f6005fed1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:55:46.007428 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df036162-3ecd-4c8b-8dcc-a9db5dc99d48 admin admin] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:55:46.011335 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae70f2f-0c44-48cb-a390-9158d22c8a59 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d9bd8d649154c979be3744bfd016f95', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b992d6a65df4bf9b27c13192d57c5f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:55:46.011961 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae70f2f-0c44-48cb-a390-9158d22c8a59 service neutron] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:55:46.012243 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae70f2f-0c44-48cb-a390-9158d22c8a59 service neutron] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:55:46.012516 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae70f2f-0c44-48cb-a390-9158d22c8a59 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:55:46.043321 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 981/1946] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 1 16:55:45 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 16:55:46.047734 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba10b3a-0d95-409e-8cae-c30f6005fed1 None None] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:55:46.075840 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ae70f2f-0c44-48cb-a390-9158d22c8a59 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:55:46.077284 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ae70f2f-0c44-48cb-a390-9158d22c8a59 service neutron] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:55:46.088895 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 966/1947] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 1 16:55:45 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 144 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 16:55:46.107089 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG dbcounter [-] [87058] Writing DB stats keystone:DELETE=4,keystone:SELECT=95,keystone:INSERT=1 {{(pid=87058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:55:46.119746 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba10b3a-0d95-409e-8cae-c30f6005fed1 None None] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:55:46.124662 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6efdee62-64ea-4172-8255-a093e41fc01c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:55:46.125331 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6efdee62-64ea-4172-8255-a093e41fc01c admin admin] REQUEST_METHOD: `POST` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:55:46.125614 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6efdee62-64ea-4172-8255-a093e41fc01c admin admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:55:46.126088 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6efdee62-64ea-4172-8255-a093e41fc01c admin admin] PATH_INFO: `/v3/projects` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:55:46.126688 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6efdee62-64ea-4172-8255-a093e41fc01c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:55:46.128638 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6efdee62-64ea-4172-8255-a093e41fc01c admin admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:55:46.268378 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 982/1948] 10.222.0.20 () {60 vars in 1153 bytes} [Mon Aug 1 16:55:46 2022] POST /identity/v3/projects => generated 383 bytes in 221 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 16:55:46.272809 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae70f2f-0c44-48cb-a390-9158d22c8a59 service neutron] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:55:46.284143 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae70f2f-0c44-48cb-a390-9158d22c8a59 service neutron] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:55:46.366168 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f6fd0d-2067-4443-bedd-687673aa4535 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:55:46.366801 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f6fd0d-2067-4443-bedd-687673aa4535 admin admin] REQUEST_METHOD: `POST` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:55:46.367082 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f6fd0d-2067-4443-bedd-687673aa4535 admin admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:55:46.367354 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f6fd0d-2067-4443-bedd-687673aa4535 admin admin] PATH_INFO: `/v3/users` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:55:46.368032 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18f6fd0d-2067-4443-bedd-687673aa4535 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:55:46.370435 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18f6fd0d-2067-4443-bedd-687673aa4535 admin admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:55:46.448246 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 967/1949] 10.222.0.20 () {60 vars in 1144 bytes} [Mon Aug 1 16:55:46 2022] POST /identity/v3/users => generated 298 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 16:55:46.455725 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6efdee62-64ea-4172-8255-a093e41fc01c admin admin] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:55:46.467448 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6efdee62-64ea-4172-8255-a093e41fc01c admin admin] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:55:46.474731 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-190d9191-6a90-4597-b3f2-227971da895a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:55:46.475343 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190d9191-6a90-4597-b3f2-227971da895a admin admin] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:55:46.475620 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190d9191-6a90-4597-b3f2-227971da895a admin admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:55:46.475894 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190d9191-6a90-4597-b3f2-227971da895a admin admin] PATH_INFO: `/v3/roles` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:55:46.476587 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-190d9191-6a90-4597-b3f2-227971da895a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:55:46.478997 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-190d9191-6a90-4597-b3f2-227971da895a admin admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:55:46.489416 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 983/1950] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Aug 1 16:55:46 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 16:55:46.494220 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f6fd0d-2067-4443-bedd-687673aa4535 admin admin] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:55:46.504285 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f6fd0d-2067-4443-bedd-687673aa4535 admin admin] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:55:46.508895 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf33ac0-129b-4d4b-903a-5411edde02e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:55:46.510134 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf33ac0-129b-4d4b-903a-5411edde02e2 admin admin] REQUEST_METHOD: `PUT` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:55:46.510456 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf33ac0-129b-4d4b-903a-5411edde02e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:55:46.510738 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf33ac0-129b-4d4b-903a-5411edde02e2 admin admin] PATH_INFO: `/v3/projects/655b8720c7704ad4898b6c6079adbc35/users/401067c5615749f8bcf0a851330a34f6/roles/440bb943dd5b46fea038b903b6eaaf81` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:55:46.514224 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG dbcounter [-] [87057] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=87057) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:55:46.531377 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faf33ac0-129b-4d4b-903a-5411edde02e2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=655b8720c7704ad4898b6c6079adbc35, user_id=401067c5615749f8bcf0a851330a34f6, role_id=440bb943dd5b46fea038b903b6eaaf81)` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:55:46.534982 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faf33ac0-129b-4d4b-903a-5411edde02e2 admin admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:55:46.552632 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-190d9191-6a90-4597-b3f2-227971da895a admin admin] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:55:46.554423 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.fernet_utils [None req-190d9191-6a90-4597-b3f2-227971da895a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:55:46.576676 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-190d9191-6a90-4597-b3f2-227971da895a admin admin] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:55:46.586473 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f1075dc-8539-4c04-a740-d0309e1aa209 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:55:46.587270 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1075dc-8539-4c04-a740-d0309e1aa209 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:55:46.588199 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1075dc-8539-4c04-a740-d0309e1aa209 admin admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:55:46.588514 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1075dc-8539-4c04-a740-d0309e1aa209 admin admin] PATH_INFO: `/v3/projects/6b4d40c3c1c8441ca60aa8df7bc1a5c4` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:55:46.590178 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f1075dc-8539-4c04-a740-d0309e1aa209 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6b4d40c3c1c8441ca60aa8df7bc1a5c4)` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:55:46.592111 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f1075dc-8539-4c04-a740-d0309e1aa209 admin admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:55:46.596820 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 968/1951] 10.222.0.20 () {60 vars in 1483 bytes} [Mon Aug 1 16:55:46 2022] PUT /identity/v3/projects/655b8720c7704ad4898b6c6079adbc35/users/401067c5615749f8bcf0a851330a34f6/roles/440bb943dd5b46fea038b903b6eaaf81 => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 16:55:46.602776 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf33ac0-129b-4d4b-903a-5411edde02e2 admin admin] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:55:46.669945 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf33ac0-129b-4d4b-903a-5411edde02e2 admin admin] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:55:46.677986 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab1d8e66-bd72-4e77-a3ea-6752dd9007cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:55:46.677986 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1d8e66-bd72-4e77-a3ea-6752dd9007cf admin admin] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:55:46.677986 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1d8e66-bd72-4e77-a3ea-6752dd9007cf admin admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:55:46.677986 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1d8e66-bd72-4e77-a3ea-6752dd9007cf admin admin] PATH_INFO: `/v3/roles` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:55:46.678427 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab1d8e66-bd72-4e77-a3ea-6752dd9007cf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:55:46.678619 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab1d8e66-bd72-4e77-a3ea-6752dd9007cf admin admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:55:46.688730 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.notifications [None req-4f1075dc-8539-4c04-a740-d0309e1aa209 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6b4d40c3c1c8441ca60aa8df7bc1a5c4', 'request_id': 'req-4f1075dc-8539-4c04-a740-d0309e1aa209'} {{(pid=87058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:55:46.689922 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 969/1952] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Aug 1 16:55:46 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 16:55:46.693877 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab1d8e66-bd72-4e77-a3ea-6752dd9007cf admin admin] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:55:46.696348 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.notifications [None req-4f1075dc-8539-4c04-a740-d0309e1aa209 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6b4d40c3c1c8441ca60aa8df7bc1a5c4', 'request_id': 'req-4f1075dc-8539-4c04-a740-d0309e1aa209'} {{(pid=87058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:55:46.703585 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.notifications [None req-4f1075dc-8539-4c04-a740-d0309e1aa209 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6b4d40c3c1c8441ca60aa8df7bc1a5c4', 'request_id': 'req-4f1075dc-8539-4c04-a740-d0309e1aa209'} {{(pid=87058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:55:46.704916 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab1d8e66-bd72-4e77-a3ea-6752dd9007cf admin admin] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:55:46.764132 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66b15637-5074-4378-b0a2-6f4435b3e587 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:55:46.764816 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b15637-5074-4378-b0a2-6f4435b3e587 admin admin] REQUEST_METHOD: `PUT` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:55:46.765245 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b15637-5074-4378-b0a2-6f4435b3e587 admin admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:55:46.765532 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b15637-5074-4378-b0a2-6f4435b3e587 admin admin] PATH_INFO: `/v3/projects/655b8720c7704ad4898b6c6079adbc35/users/401067c5615749f8bcf0a851330a34f6/roles/440bb943dd5b46fea038b903b6eaaf81` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:55:46.767593 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.notifications [None req-4f1075dc-8539-4c04-a740-d0309e1aa209 admin admin] The token cache is being invalidate because project 6b4d40c3c1c8441ca60aa8df7bc1a5c4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87058) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 16:55:46.767720 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66b15637-5074-4378-b0a2-6f4435b3e587 admin admin] RBAC: Authorizing `identity:create_grant(project_id=655b8720c7704ad4898b6c6079adbc35, user_id=401067c5615749f8bcf0a851330a34f6, role_id=440bb943dd5b46fea038b903b6eaaf81)` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:55:46.767902 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.notifications [None req-4f1075dc-8539-4c04-a740-d0309e1aa209 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:55:46.768536 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.notifications [None req-4f1075dc-8539-4c04-a740-d0309e1aa209 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:55:46.770257 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 984/1953] 10.222.0.20 () {58 vars in 1233 bytes} [Mon Aug 1 16:55:46 2022] DELETE /identity/v3/projects/6b4d40c3c1c8441ca60aa8df7bc1a5c4 => generated 0 bytes in 218 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 16:55:46.770364 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66b15637-5074-4378-b0a2-6f4435b3e587 admin admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:55:46.773919 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f1075dc-8539-4c04-a740-d0309e1aa209 admin admin] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:55:46.775649 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.fernet_utils [None req-4f1075dc-8539-4c04-a740-d0309e1aa209 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:55:46.781201 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 970/1954] 10.222.0.20 () {60 vars in 1483 bytes} [Mon Aug 1 16:55:46 2022] PUT /identity/v3/projects/655b8720c7704ad4898b6c6079adbc35/users/401067c5615749f8bcf0a851330a34f6/roles/440bb943dd5b46fea038b903b6eaaf81 => generated 0 bytes in 88 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 16:55:46.789938 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f1075dc-8539-4c04-a740-d0309e1aa209 admin admin] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:55:46.791991 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c98530e-411c-449e-bb1b-3781f977f970 None None] REQUEST_METHOD: `POST` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:55:46.792276 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c98530e-411c-449e-bb1b-3781f977f970 None None] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:55:46.792549 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c98530e-411c-449e-bb1b-3781f977f970 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:55:46.821887 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.auth.core [None req-0c98530e-411c-449e-bb1b-3781f977f970 None None] MFA Rules not processed for user `401067c5615749f8bcf0a851330a34f6`. Rule list: `[]` (Enabled: `True`). {{(pid=87057) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:55:46.823142 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.fernet_utils [None req-0c98530e-411c-449e-bb1b-3781f977f970 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87057) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:55:46.845873 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a6d0b9-2691-43d7-ada8-aba2d971317d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:55:46.846562 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a6d0b9-2691-43d7-ada8-aba2d971317d admin admin] REQUEST_METHOD: `DELETE` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:55:46.846843 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a6d0b9-2691-43d7-ada8-aba2d971317d admin admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:55:46.847118 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a6d0b9-2691-43d7-ada8-aba2d971317d admin admin] PATH_INFO: `/v3/users/d2fbbb9ad49c4133bac2d2f02feefa25` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:55:46.848313 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80a6d0b9-2691-43d7-ada8-aba2d971317d admin admin] RBAC: Authorizing `identity:delete_user(user_id=d2fbbb9ad49c4133bac2d2f02feefa25)` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:55:46.850468 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80a6d0b9-2691-43d7-ada8-aba2d971317d admin admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:55:46.889602 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG dbcounter [-] [87057] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=87057) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:55:46.893247 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 971/1955] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 1 16:55:46 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 16:55:46.910234 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c98530e-411c-449e-bb1b-3781f977f970 None None] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:55:46.911630 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.fernet_utils [None req-0c98530e-411c-449e-bb1b-3781f977f970 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87057) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:55:46.970977 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c98530e-411c-449e-bb1b-3781f977f970 None None] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:55:46.974636 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41aee682-8a1b-4768-b3cd-d21ff46ac683 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25970b52ca15448fa1abb1f974d64038', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b992d6a65df4bf9b27c13192d57c5f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:55:46.982190 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41aee682-8a1b-4768-b3cd-d21ff46ac683 service cinder] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:55:46.982190 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41aee682-8a1b-4768-b3cd-d21ff46ac683 service cinder] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:55:46.982190 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41aee682-8a1b-4768-b3cd-d21ff46ac683 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:55:46.982190 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.fernet_utils [None req-41aee682-8a1b-4768-b3cd-d21ff46ac683 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87057) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:55:47.037593 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41aee682-8a1b-4768-b3cd-d21ff46ac683 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:55:47.039065 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41aee682-8a1b-4768-b3cd-d21ff46ac683 service cinder] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:55:47.044911 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.notifications [None req-80a6d0b9-2691-43d7-ada8-aba2d971317d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd2fbbb9ad49c4133bac2d2f02feefa25'} {{(pid=87058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:55:47.045591 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.notifications [None req-80a6d0b9-2691-43d7-ada8-aba2d971317d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd2fbbb9ad49c4133bac2d2f02feefa25'} {{(pid=87058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:55:47.046258 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.notifications [None req-80a6d0b9-2691-43d7-ada8-aba2d971317d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd2fbbb9ad49c4133bac2d2f02feefa25'} {{(pid=87058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:55:47.048180 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG dbcounter [-] [87058] Writing DB stats keystone:SELECT=84,keystone:INSERT=2,keystone:DELETE=14 {{(pid=87058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:55:47.050426 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 972/1956] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 1 16:55:46 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 01 16:55:47.063176 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.notifications [None req-80a6d0b9-2691-43d7-ada8-aba2d971317d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd2fbbb9ad49c4133bac2d2f02feefa25'} {{(pid=87058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:55:47.067326 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.notifications [None req-80a6d0b9-2691-43d7-ada8-aba2d971317d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd2fbbb9ad49c4133bac2d2f02feefa25'} {{(pid=87058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:55:47.161377 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 985/1957] 10.222.0.20 () {58 vars in 1224 bytes} [Mon Aug 1 16:55:46 2022] DELETE /identity/v3/users/d2fbbb9ad49c4133bac2d2f02feefa25 => generated 0 bytes in 388 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 16:55:47.639385 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41aee682-8a1b-4768-b3cd-d21ff46ac683 service cinder] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:55:47.650968 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41aee682-8a1b-4768-b3cd-d21ff46ac683 service cinder] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:55:47.742599 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f68ea01d-95f1-4b1b-89b5-3db5cdd00987 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:55:47.743271 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68ea01d-95f1-4b1b-89b5-3db5cdd00987 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:55:47.744138 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68ea01d-95f1-4b1b-89b5-3db5cdd00987 admin admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:55:47.750204 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68ea01d-95f1-4b1b-89b5-3db5cdd00987 admin admin] PATH_INFO: `/v3/projects/27be61b212e24589ad5a6034f0d205e0` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:55:47.750204 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f68ea01d-95f1-4b1b-89b5-3db5cdd00987 admin admin] RBAC: Authorizing `identity:delete_project(project_id=27be61b212e24589ad5a6034f0d205e0)` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:55:47.750204 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f68ea01d-95f1-4b1b-89b5-3db5cdd00987 admin admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:55:47.849952 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.notifications [None req-f68ea01d-95f1-4b1b-89b5-3db5cdd00987 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '27be61b212e24589ad5a6034f0d205e0', 'request_id': 'req-f68ea01d-95f1-4b1b-89b5-3db5cdd00987'} {{(pid=87057) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:55:47.857138 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.notifications [None req-f68ea01d-95f1-4b1b-89b5-3db5cdd00987 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '27be61b212e24589ad5a6034f0d205e0', 'request_id': 'req-f68ea01d-95f1-4b1b-89b5-3db5cdd00987'} {{(pid=87057) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:55:47.859477 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG dbcounter [-] [87057] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=87057) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:55:47.863621 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.notifications [None req-f68ea01d-95f1-4b1b-89b5-3db5cdd00987 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '27be61b212e24589ad5a6034f0d205e0', 'request_id': 'req-f68ea01d-95f1-4b1b-89b5-3db5cdd00987'} {{(pid=87057) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:55:47.956369 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.notifications [None req-f68ea01d-95f1-4b1b-89b5-3db5cdd00987 admin admin] The token cache is being invalidate because project 27be61b212e24589ad5a6034f0d205e0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87057) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 16:55:47.956698 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.notifications [None req-f68ea01d-95f1-4b1b-89b5-3db5cdd00987 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87057) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:55:47.957445 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.notifications [None req-f68ea01d-95f1-4b1b-89b5-3db5cdd00987 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87057) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:55:47.959099 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 973/1958] 10.222.0.20 () {58 vars in 1233 bytes} [Mon Aug 1 16:55:47 2022] DELETE /identity/v3/projects/27be61b212e24589ad5a6034f0d205e0 => generated 0 bytes in 320 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 16:55:47.964219 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45df9c3-1c85-4ecf-bc41-8050d27b4a1f None None] REQUEST_METHOD: `POST` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:55:47.964670 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45df9c3-1c85-4ecf-bc41-8050d27b4a1f None None] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:55:47.964771 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45df9c3-1c85-4ecf-bc41-8050d27b4a1f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:55:47.985778 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.auth.core [None req-d45df9c3-1c85-4ecf-bc41-8050d27b4a1f None None] MFA Rules not processed for user `1007f8e379b74e169b98fb7c3c0ba5b4`. Rule list: `[]` (Enabled: `True`). {{(pid=87058) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:55:47.987099 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.fernet_utils [None req-d45df9c3-1c85-4ecf-bc41-8050d27b4a1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:55:48.052846 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 986/1959] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 1 16:55:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:55:48.371780 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f2af3d-5707-487f-abc0-6102206ab6a9 None None] REQUEST_METHOD: `POST` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:55:48.371780 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f2af3d-5707-487f-abc0-6102206ab6a9 None None] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:55:48.371780 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f2af3d-5707-487f-abc0-6102206ab6a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:55:48.371780 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.auth.core [None req-71f2af3d-5707-487f-abc0-6102206ab6a9 None None] MFA Rules not processed for user `1007f8e379b74e169b98fb7c3c0ba5b4`. Rule list: `[]` (Enabled: `True`). {{(pid=87057) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:55:48.371780 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.fernet_utils [None req-71f2af3d-5707-487f-abc0-6102206ab6a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87057) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:55:48.371780 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 974/1960] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 1 16:55:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:55:48.372383 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d45df9c3-1c85-4ecf-bc41-8050d27b4a1f None None] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:55:48.372383 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d45df9c3-1c85-4ecf-bc41-8050d27b4a1f None None] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:55:48.372383 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de2baab-561a-46e9-ae94-248e1a938e31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:55:48.372383 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de2baab-561a-46e9-ae94-248e1a938e31 admin admin] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:55:48.372722 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de2baab-561a-46e9-ae94-248e1a938e31 admin admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:55:48.372722 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de2baab-561a-46e9-ae94-248e1a938e31 admin admin] PATH_INFO: `/v3/domains` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:55:48.372722 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4de2baab-561a-46e9-ae94-248e1a938e31 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87058) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:55:48.372722 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4de2baab-561a-46e9-ae94-248e1a938e31 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:55:48.372722 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4de2baab-561a-46e9-ae94-248e1a938e31 admin admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:55:48.372722 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 987/1961] 10.222.0.20 () {58 vars in 1153 bytes} [Mon Aug 1 16:55:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 16:55:48.373179 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f2af3d-5707-487f-abc0-6102206ab6a9 None None] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:55:48.373179 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f2af3d-5707-487f-abc0-6102206ab6a9 None None] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:55:48.373179 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f386949f-3e90-45d9-b07b-5a1d64b9fee3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:55:48.373179 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f386949f-3e90-45d9-b07b-5a1d64b9fee3 admin admin] REQUEST_METHOD: `POST` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:55:48.373482 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f386949f-3e90-45d9-b07b-5a1d64b9fee3 admin admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:55:48.373482 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f386949f-3e90-45d9-b07b-5a1d64b9fee3 admin admin] PATH_INFO: `/v3/projects` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:55:48.373482 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f386949f-3e90-45d9-b07b-5a1d64b9fee3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:55:48.373482 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f386949f-3e90-45d9-b07b-5a1d64b9fee3 admin admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:55:48.380571 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 975/1962] 10.222.0.20 () {60 vars in 1153 bytes} [Mon Aug 1 16:55:48 2022] POST /identity/v3/projects => generated 391 bytes in 141 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 16:55:48.384745 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de2baab-561a-46e9-ae94-248e1a938e31 admin admin] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:55:48.395497 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de2baab-561a-46e9-ae94-248e1a938e31 admin admin] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:55:48.452381 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4690a628-77c2-4a35-b3b6-a98b03cf2bfe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:55:48.453004 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4690a628-77c2-4a35-b3b6-a98b03cf2bfe admin admin] REQUEST_METHOD: `POST` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:55:48.453284 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4690a628-77c2-4a35-b3b6-a98b03cf2bfe admin admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:55:48.453557 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4690a628-77c2-4a35-b3b6-a98b03cf2bfe admin admin] PATH_INFO: `/v3/users` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:55:48.454255 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4690a628-77c2-4a35-b3b6-a98b03cf2bfe admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:55:48.456213 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4690a628-77c2-4a35-b3b6-a98b03cf2bfe admin admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:55:48.521835 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 988/1963] 10.222.0.20 () {60 vars in 1144 bytes} [Mon Aug 1 16:55:48 2022] POST /identity/v3/users => generated 296 bytes in 137 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 16:55:48.526657 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f386949f-3e90-45d9-b07b-5a1d64b9fee3 admin admin] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:55:48.537471 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f386949f-3e90-45d9-b07b-5a1d64b9fee3 admin admin] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:55:48.542839 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d1fdd4b-c66f-464f-9062-c478e332a355 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:55:48.543458 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1fdd4b-c66f-464f-9062-c478e332a355 admin admin] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:55:48.543733 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1fdd4b-c66f-464f-9062-c478e332a355 admin admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:55:48.544005 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1fdd4b-c66f-464f-9062-c478e332a355 admin admin] PATH_INFO: `/v3/roles` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:55:48.544707 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d1fdd4b-c66f-464f-9062-c478e332a355 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:55:48.546327 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d1fdd4b-c66f-464f-9062-c478e332a355 admin admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:55:48.558012 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 976/1964] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Aug 1 16:55:48 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 16:55:48.562246 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4690a628-77c2-4a35-b3b6-a98b03cf2bfe admin admin] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:55:48.572239 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4690a628-77c2-4a35-b3b6-a98b03cf2bfe admin admin] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:55:48.577032 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e26ba82-56c2-47f4-8453-ec91cfb3ae8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:55:48.577649 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e26ba82-56c2-47f4-8453-ec91cfb3ae8c admin admin] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:55:48.578262 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e26ba82-56c2-47f4-8453-ec91cfb3ae8c admin admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:55:48.578262 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e26ba82-56c2-47f4-8453-ec91cfb3ae8c admin admin] PATH_INFO: `/v3/roles` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:55:48.578909 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e26ba82-56c2-47f4-8453-ec91cfb3ae8c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:55:48.580535 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e26ba82-56c2-47f4-8453-ec91cfb3ae8c admin admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:55:48.595398 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 989/1965] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Aug 1 16:55:48 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 16:55:48.598011 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d1fdd4b-c66f-464f-9062-c478e332a355 admin admin] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:55:48.610922 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d1fdd4b-c66f-464f-9062-c478e332a355 admin admin] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:55:48.615598 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c5f805c-54c3-4fae-a856-052d46c9b29b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:55:48.616265 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5f805c-54c3-4fae-a856-052d46c9b29b admin admin] REQUEST_METHOD: `PUT` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:55:48.616543 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5f805c-54c3-4fae-a856-052d46c9b29b admin admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:55:48.616814 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5f805c-54c3-4fae-a856-052d46c9b29b admin admin] PATH_INFO: `/v3/projects/36cafe4005194967a3c308ed93d654b3/users/874f243c521c4cc981f0497dd0fb7160/roles/0999d3da63a545f18ff3252afdc90518` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:55:48.636696 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c5f805c-54c3-4fae-a856-052d46c9b29b admin admin] RBAC: Authorizing `identity:create_grant(project_id=36cafe4005194967a3c308ed93d654b3, user_id=874f243c521c4cc981f0497dd0fb7160, role_id=0999d3da63a545f18ff3252afdc90518)` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:55:48.637907 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c5f805c-54c3-4fae-a856-052d46c9b29b admin admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:55:48.700926 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 977/1966] 10.222.0.20 () {60 vars in 1483 bytes} [Mon Aug 1 16:55:48 2022] PUT /identity/v3/projects/36cafe4005194967a3c308ed93d654b3/users/874f243c521c4cc981f0497dd0fb7160/roles/0999d3da63a545f18ff3252afdc90518 => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 16:55:48.713218 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583de9ff-437e-40ea-a8bf-b9a691aa9838 None None] REQUEST_METHOD: `POST` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:55:48.713568 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583de9ff-437e-40ea-a8bf-b9a691aa9838 None None] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:55:48.713920 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583de9ff-437e-40ea-a8bf-b9a691aa9838 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:55:48.762486 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.auth.core [None req-583de9ff-437e-40ea-a8bf-b9a691aa9838 None None] MFA Rules not processed for user `874f243c521c4cc981f0497dd0fb7160`. Rule list: `[]` (Enabled: `True`). {{(pid=87058) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:55:48.763744 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.fernet_utils [None req-583de9ff-437e-40ea-a8bf-b9a691aa9838 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:55:48.786483 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG dbcounter [-] [87058] Writing DB stats keystone:SELECT=93,keystone:DELETE=3,keystone:INSERT=4 {{(pid=87058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:55:48.831299 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 990/1967] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 1 16:55:48 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:55:48.835935 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c5f805c-54c3-4fae-a856-052d46c9b29b admin admin] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:55:48.903988 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c5f805c-54c3-4fae-a856-052d46c9b29b admin admin] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:55:48.909711 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a39dfa32-8120-495c-864f-61fb4c84fe75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:55:48.910334 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39dfa32-8120-495c-864f-61fb4c84fe75 admin admin] REQUEST_METHOD: `POST` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:55:48.910616 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39dfa32-8120-495c-864f-61fb4c84fe75 admin admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:55:48.911035 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39dfa32-8120-495c-864f-61fb4c84fe75 admin admin] PATH_INFO: `/v3/projects` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:55:48.911782 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a39dfa32-8120-495c-864f-61fb4c84fe75 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:55:48.914203 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a39dfa32-8120-495c-864f-61fb4c84fe75 admin admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:55:49.032774 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 978/1968] 10.222.0.20 () {60 vars in 1153 bytes} [Mon Aug 1 16:55:48 2022] POST /identity/v3/projects => generated 391 bytes in 197 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 16:55:49.036979 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-583de9ff-437e-40ea-a8bf-b9a691aa9838 None None] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:55:49.048841 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-583de9ff-437e-40ea-a8bf-b9a691aa9838 None None] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:55:49.105501 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7fa61cf-0202-449d-b7c0-490805f2e446 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:55:49.106144 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7fa61cf-0202-449d-b7c0-490805f2e446 admin admin] REQUEST_METHOD: `POST` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:55:49.106427 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7fa61cf-0202-449d-b7c0-490805f2e446 admin admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:55:49.106702 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7fa61cf-0202-449d-b7c0-490805f2e446 admin admin] PATH_INFO: `/v3/users` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:55:49.107363 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7fa61cf-0202-449d-b7c0-490805f2e446 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:55:49.109384 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7fa61cf-0202-449d-b7c0-490805f2e446 admin admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:55:49.253403 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 991/1969] 10.222.0.20 () {60 vars in 1144 bytes} [Mon Aug 1 16:55:49 2022] POST /identity/v3/users => generated 302 bytes in 217 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 16:55:49.257202 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a39dfa32-8120-495c-864f-61fb4c84fe75 admin admin] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:55:49.267578 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a39dfa32-8120-495c-864f-61fb4c84fe75 admin admin] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:55:49.272077 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b04227-2b31-4e51-8f23-ec0277bf55f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:55:49.272680 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b04227-2b31-4e51-8f23-ec0277bf55f0 admin admin] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:55:49.272958 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b04227-2b31-4e51-8f23-ec0277bf55f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:55:49.273230 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b04227-2b31-4e51-8f23-ec0277bf55f0 admin admin] PATH_INFO: `/v3/roles` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:55:49.273923 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90b04227-2b31-4e51-8f23-ec0277bf55f0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:55:49.276296 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90b04227-2b31-4e51-8f23-ec0277bf55f0 admin admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:55:49.286675 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 979/1970] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Aug 1 16:55:49 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 16:55:49.291480 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7fa61cf-0202-449d-b7c0-490805f2e446 admin admin] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:55:49.301353 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7fa61cf-0202-449d-b7c0-490805f2e446 admin admin] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:55:49.305879 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf90cbe-4ab9-434d-8f68-131d644af08a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:55:49.306553 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf90cbe-4ab9-434d-8f68-131d644af08a admin admin] REQUEST_METHOD: `PUT` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:55:49.306835 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf90cbe-4ab9-434d-8f68-131d644af08a admin admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:55:49.307110 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf90cbe-4ab9-434d-8f68-131d644af08a admin admin] PATH_INFO: `/v3/projects/cd45893e9425436d8cbeeb8de6da52fe/users/aab36b624bcd4a97bb690a0bb163f496/roles/440bb943dd5b46fea038b903b6eaaf81` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:55:49.321886 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acf90cbe-4ab9-434d-8f68-131d644af08a admin admin] RBAC: Authorizing `identity:create_grant(project_id=cd45893e9425436d8cbeeb8de6da52fe, user_id=aab36b624bcd4a97bb690a0bb163f496, role_id=440bb943dd5b46fea038b903b6eaaf81)` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:55:49.324639 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acf90cbe-4ab9-434d-8f68-131d644af08a admin admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:55:49.403578 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 992/1971] 10.222.0.20 () {60 vars in 1483 bytes} [Mon Aug 1 16:55:49 2022] PUT /identity/v3/projects/cd45893e9425436d8cbeeb8de6da52fe/users/aab36b624bcd4a97bb690a0bb163f496/roles/440bb943dd5b46fea038b903b6eaaf81 => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 16:55:49.407709 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b04227-2b31-4e51-8f23-ec0277bf55f0 admin admin] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:55:49.418307 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b04227-2b31-4e51-8f23-ec0277bf55f0 admin admin] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:55:49.467312 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG dbcounter [-] [87057] Writing DB stats keystone:SELECT=94,keystone:DELETE=2,keystone:INSERT=4 {{(pid=87057) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:55:49.473967 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aa7cbe7-1715-48c6-97a5-73d10494aeaf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:55:49.474653 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa7cbe7-1715-48c6-97a5-73d10494aeaf admin admin] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:55:49.474937 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa7cbe7-1715-48c6-97a5-73d10494aeaf admin admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:55:49.475214 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa7cbe7-1715-48c6-97a5-73d10494aeaf admin admin] PATH_INFO: `/v3/roles` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:55:49.475922 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aa7cbe7-1715-48c6-97a5-73d10494aeaf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:55:49.477526 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aa7cbe7-1715-48c6-97a5-73d10494aeaf admin admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:55:49.487974 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 980/1972] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Aug 1 16:55:49 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 16:55:49.492324 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf90cbe-4ab9-434d-8f68-131d644af08a admin admin] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:55:49.503532 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf90cbe-4ab9-434d-8f68-131d644af08a admin admin] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:55:49.508112 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6af4d0ba-d019-4c27-8429-4c7e67790c34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:55:49.513815 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af4d0ba-d019-4c27-8429-4c7e67790c34 admin admin] REQUEST_METHOD: `PUT` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:55:49.513815 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af4d0ba-d019-4c27-8429-4c7e67790c34 admin admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:55:49.513815 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af4d0ba-d019-4c27-8429-4c7e67790c34 admin admin] PATH_INFO: `/v3/projects/cd45893e9425436d8cbeeb8de6da52fe/users/aab36b624bcd4a97bb690a0bb163f496/roles/440bb943dd5b46fea038b903b6eaaf81` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:55:49.513815 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6af4d0ba-d019-4c27-8429-4c7e67790c34 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cd45893e9425436d8cbeeb8de6da52fe, user_id=aab36b624bcd4a97bb690a0bb163f496, role_id=440bb943dd5b46fea038b903b6eaaf81)` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:55:49.514729 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6af4d0ba-d019-4c27-8429-4c7e67790c34 admin admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:55:49.525787 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 993/1973] 10.222.0.20 () {60 vars in 1483 bytes} [Mon Aug 1 16:55:49 2022] PUT /identity/v3/projects/cd45893e9425436d8cbeeb8de6da52fe/users/aab36b624bcd4a97bb690a0bb163f496/roles/440bb943dd5b46fea038b903b6eaaf81 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 16:55:49.535723 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec40a98-54e6-4605-bb5e-7ec08e8fd58f None None] REQUEST_METHOD: `POST` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:55:49.536014 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec40a98-54e6-4605-bb5e-7ec08e8fd58f None None] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:55:49.536294 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec40a98-54e6-4605-bb5e-7ec08e8fd58f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:55:49.563283 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.auth.core [None req-eec40a98-54e6-4605-bb5e-7ec08e8fd58f None None] MFA Rules not processed for user `aab36b624bcd4a97bb690a0bb163f496`. Rule list: `[]` (Enabled: `True`). {{(pid=87057) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:55:49.564533 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.fernet_utils [None req-eec40a98-54e6-4605-bb5e-7ec08e8fd58f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87057) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:55:49.635139 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 981/1974] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 1 16:55:49 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:55:49.645857 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6af4d0ba-d019-4c27-8429-4c7e67790c34 admin admin] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:55:49.647555 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.fernet_utils [None req-6af4d0ba-d019-4c27-8429-4c7e67790c34 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:55:49.704219 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6af4d0ba-d019-4c27-8429-4c7e67790c34 admin admin] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:55:49.707832 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e49db93-92e9-45ee-a43f-282e0840e371 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25970b52ca15448fa1abb1f974d64038', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b992d6a65df4bf9b27c13192d57c5f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:55:49.708440 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e49db93-92e9-45ee-a43f-282e0840e371 service cinder] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:55:49.708720 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e49db93-92e9-45ee-a43f-282e0840e371 service cinder] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:55:49.708991 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e49db93-92e9-45ee-a43f-282e0840e371 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:55:49.718579 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e49db93-92e9-45ee-a43f-282e0840e371 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:55:49.719998 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e49db93-92e9-45ee-a43f-282e0840e371 service cinder] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:55:49.729544 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 994/1975] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 1 16:55:49 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 16:55:49.777434 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eec40a98-54e6-4605-bb5e-7ec08e8fd58f None None] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:55:49.778915 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.fernet_utils [None req-eec40a98-54e6-4605-bb5e-7ec08e8fd58f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87057) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:55:49.791287 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eec40a98-54e6-4605-bb5e-7ec08e8fd58f None None] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:55:49.794925 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e25c96-4a01-4091-98bf-d072cc9e7845 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25970b52ca15448fa1abb1f974d64038', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b992d6a65df4bf9b27c13192d57c5f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:55:49.795539 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e25c96-4a01-4091-98bf-d072cc9e7845 service cinder] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:55:49.795819 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e25c96-4a01-4091-98bf-d072cc9e7845 service cinder] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:55:49.796093 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e25c96-4a01-4091-98bf-d072cc9e7845 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:55:49.892611 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99e25c96-4a01-4091-98bf-d072cc9e7845 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:55:49.894106 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99e25c96-4a01-4091-98bf-d072cc9e7845 service cinder] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:55:49.905210 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 982/1976] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 1 16:55:49 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 16:55:51.866465 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3318867f-1bb6-43bb-9fc6-736d49c4b728 None None] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:55:51.866953 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3318867f-1bb6-43bb-9fc6-736d49c4b728 None None] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:55:51.867053 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3318867f-1bb6-43bb-9fc6-736d49c4b728 None None] PATH_INFO: `/` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:55:51.868752 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 995/1977] 10.222.0.20 () {54 vars in 921 bytes} [Mon Aug 1 16:55:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 16:55:51.878234 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3af5a6-bc50-4d2e-b71d-4eaa54a12d2a None None] REQUEST_METHOD: `POST` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:55:51.879074 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3af5a6-bc50-4d2e-b71d-4eaa54a12d2a None None] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:55:51.879334 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3af5a6-bc50-4d2e-b71d-4eaa54a12d2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:55:51.899027 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.auth.core [None req-1b3af5a6-bc50-4d2e-b71d-4eaa54a12d2a None None] MFA Rules not processed for user `6533106d03eb4ca185a8839336d2aff9`. Rule list: `[]` (Enabled: `True`). {{(pid=87057) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:55:51.904488 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.fernet_utils [None req-1b3af5a6-bc50-4d2e-b71d-4eaa54a12d2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87057) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:55:51.972595 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 983/1978] 10.222.0.20 () {58 vars in 1018 bytes} [Mon Aug 1 16:55:51 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:55:51.990380 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3318867f-1bb6-43bb-9fc6-736d49c4b728 None None] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:55:51.992069 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.fernet_utils [None req-3318867f-1bb6-43bb-9fc6-736d49c4b728 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:55:52.011319 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3318867f-1bb6-43bb-9fc6-736d49c4b728 None None] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:55:52.016948 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e719014c-eedd-4bfc-b12c-229898ce5811 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6533106d03eb4ca185a8839336d2aff9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b992d6a65df4bf9b27c13192d57c5f0', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:55:52.017685 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e719014c-eedd-4bfc-b12c-229898ce5811 service nova] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:55:52.017963 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e719014c-eedd-4bfc-b12c-229898ce5811 service nova] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:55:52.018270 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e719014c-eedd-4bfc-b12c-229898ce5811 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:55:52.027415 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e719014c-eedd-4bfc-b12c-229898ce5811 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:55:52.029078 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e719014c-eedd-4bfc-b12c-229898ce5811 service nova] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:55:52.039560 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 996/1979] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 1 16:55:51 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 16:56:01.960744 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG dbcounter [-] [87057] Writing DB stats keystone:SELECT=97 {{(pid=87057) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:56:02.035427 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG dbcounter [-] [87058] Writing DB stats keystone:SELECT=93,keystone:INSERT=5 {{(pid=87058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:56:03.725511 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b3af5a6-bc50-4d2e-b71d-4eaa54a12d2a None None] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:56:03.726059 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e719014c-eedd-4bfc-b12c-229898ce5811 service nova] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:56:03.736582 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e719014c-eedd-4bfc-b12c-229898ce5811 service nova] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:56:03.740417 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-533490ef-ddd4-436b-a03a-97d342583d64 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25970b52ca15448fa1abb1f974d64038', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b992d6a65df4bf9b27c13192d57c5f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:56:03.741100 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533490ef-ddd4-436b-a03a-97d342583d64 service cinder] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:56:03.741510 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533490ef-ddd4-436b-a03a-97d342583d64 service cinder] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:56:03.741855 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533490ef-ddd4-436b-a03a-97d342583d64 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:56:03.744223 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.fernet_utils [None req-533490ef-ddd4-436b-a03a-97d342583d64 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:56:03.799789 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b3af5a6-bc50-4d2e-b71d-4eaa54a12d2a None None] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:56:03.804636 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f2fb097-9f5c-4e8f-a4e1-2dc064d7c948 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:56:03.805267 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2fb097-9f5c-4e8f-a4e1-2dc064d7c948 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:56:03.805547 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2fb097-9f5c-4e8f-a4e1-2dc064d7c948 admin admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:56:03.805821 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2fb097-9f5c-4e8f-a4e1-2dc064d7c948 admin admin] PATH_INFO: `/v3/users/874f243c521c4cc981f0497dd0fb7160` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:56:03.807279 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f2fb097-9f5c-4e8f-a4e1-2dc064d7c948 admin admin] RBAC: Authorizing `identity:delete_user(user_id=874f243c521c4cc981f0497dd0fb7160)` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:56:03.811011 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f2fb097-9f5c-4e8f-a4e1-2dc064d7c948 admin admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:56:03.811277 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-533490ef-ddd4-436b-a03a-97d342583d64 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:56:03.812688 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-533490ef-ddd4-436b-a03a-97d342583d64 service cinder] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:56:03.823622 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 997/1980] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 1 16:56:03 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 01 16:56:04.013811 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.notifications [None req-5f2fb097-9f5c-4e8f-a4e1-2dc064d7c948 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '874f243c521c4cc981f0497dd0fb7160'} {{(pid=87057) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:56:04.015291 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.notifications [None req-5f2fb097-9f5c-4e8f-a4e1-2dc064d7c948 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '874f243c521c4cc981f0497dd0fb7160'} {{(pid=87057) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:56:04.122906 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.notifications [None req-5f2fb097-9f5c-4e8f-a4e1-2dc064d7c948 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '874f243c521c4cc981f0497dd0fb7160'} {{(pid=87057) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:56:04.123619 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.notifications [None req-5f2fb097-9f5c-4e8f-a4e1-2dc064d7c948 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '874f243c521c4cc981f0497dd0fb7160'} {{(pid=87057) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:56:04.132687 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.notifications [None req-5f2fb097-9f5c-4e8f-a4e1-2dc064d7c948 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '874f243c521c4cc981f0497dd0fb7160'} {{(pid=87057) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:56:04.154231 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 984/1981] 10.222.0.20 () {58 vars in 1224 bytes} [Mon Aug 1 16:56:03 2022] DELETE /identity/v3/users/874f243c521c4cc981f0497dd0fb7160 => generated 0 bytes in 429 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 16:56:04.158614 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2277e6-011e-4ce1-97df-ea67f21feea6 None None] REQUEST_METHOD: `POST` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:56:04.158902 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2277e6-011e-4ce1-97df-ea67f21feea6 None None] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:56:04.159175 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2277e6-011e-4ce1-97df-ea67f21feea6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:56:04.197778 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.auth.core [None req-1b2277e6-011e-4ce1-97df-ea67f21feea6 None None] MFA Rules not processed for user `1007f8e379b74e169b98fb7c3c0ba5b4`. Rule list: `[]` (Enabled: `True`). {{(pid=87058) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:56:04.199096 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.fernet_utils [None req-1b2277e6-011e-4ce1-97df-ea67f21feea6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:56:04.267434 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 998/1982] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 1 16:56:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 16:56:04.277621 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f2fb097-9f5c-4e8f-a4e1-2dc064d7c948 admin admin] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:56:04.279356 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.fernet_utils [None req-5f2fb097-9f5c-4e8f-a4e1-2dc064d7c948 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87057) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:56:04.344628 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f2fb097-9f5c-4e8f-a4e1-2dc064d7c948 admin admin] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:56:04.348393 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b4d04b6-c9d3-485d-a381-978b60210549 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d9bd8d649154c979be3744bfd016f95', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b992d6a65df4bf9b27c13192d57c5f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:56:04.349020 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4d04b6-c9d3-485d-a381-978b60210549 service neutron] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:56:04.349294 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4d04b6-c9d3-485d-a381-978b60210549 service neutron] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:56:04.352945 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4d04b6-c9d3-485d-a381-978b60210549 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:56:04.359700 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b4d04b6-c9d3-485d-a381-978b60210549 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:56:04.361144 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b4d04b6-c9d3-485d-a381-978b60210549 service neutron] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:56:04.373552 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 985/1983] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 1 16:56:04 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 16:56:04.888377 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b2277e6-011e-4ce1-97df-ea67f21feea6 None None] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:56:04.890317 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.fernet_utils [None req-1b2277e6-011e-4ce1-97df-ea67f21feea6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:56:04.903319 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b2277e6-011e-4ce1-97df-ea67f21feea6 None None] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:56:04.908212 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13349b2d-557a-4adb-9d3f-dcdd7c02645f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:56:04.908868 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13349b2d-557a-4adb-9d3f-dcdd7c02645f admin admin] REQUEST_METHOD: `DELETE` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:56:04.909148 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13349b2d-557a-4adb-9d3f-dcdd7c02645f admin admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:56:04.909674 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13349b2d-557a-4adb-9d3f-dcdd7c02645f admin admin] PATH_INFO: `/v3/projects/36cafe4005194967a3c308ed93d654b3` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:56:04.910966 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13349b2d-557a-4adb-9d3f-dcdd7c02645f admin admin] RBAC: Authorizing `identity:delete_project(project_id=36cafe4005194967a3c308ed93d654b3)` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:56:04.912876 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13349b2d-557a-4adb-9d3f-dcdd7c02645f admin admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:56:05.042184 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.notifications [None req-13349b2d-557a-4adb-9d3f-dcdd7c02645f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '36cafe4005194967a3c308ed93d654b3', 'request_id': 'req-13349b2d-557a-4adb-9d3f-dcdd7c02645f'} {{(pid=87058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:56:05.048749 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.notifications [None req-13349b2d-557a-4adb-9d3f-dcdd7c02645f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '36cafe4005194967a3c308ed93d654b3', 'request_id': 'req-13349b2d-557a-4adb-9d3f-dcdd7c02645f'} {{(pid=87058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:56:05.054920 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.notifications [None req-13349b2d-557a-4adb-9d3f-dcdd7c02645f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '36cafe4005194967a3c308ed93d654b3', 'request_id': 'req-13349b2d-557a-4adb-9d3f-dcdd7c02645f'} {{(pid=87058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:56:05.172306 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.notifications [None req-13349b2d-557a-4adb-9d3f-dcdd7c02645f admin admin] The token cache is being invalidate because project 36cafe4005194967a3c308ed93d654b3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87058) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 16:56:05.172601 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.notifications [None req-13349b2d-557a-4adb-9d3f-dcdd7c02645f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:56:05.173407 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.notifications [None req-13349b2d-557a-4adb-9d3f-dcdd7c02645f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:56:05.174999 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 999/1984] 10.222.0.20 () {58 vars in 1233 bytes} [Mon Aug 1 16:56:04 2022] DELETE /identity/v3/projects/36cafe4005194967a3c308ed93d654b3 => generated 0 bytes in 287 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 16:56:05.179179 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b4d04b6-c9d3-485d-a381-978b60210549 service neutron] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:56:05.190676 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b4d04b6-c9d3-485d-a381-978b60210549 service neutron] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:56:05.229217 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG dbcounter [-] [87057] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87057) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:56:05.252468 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff1fac78-192a-40fa-bc37-361d350b1c9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:56:05.253140 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1fac78-192a-40fa-bc37-361d350b1c9a admin admin] REQUEST_METHOD: `DELETE` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:56:05.253426 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1fac78-192a-40fa-bc37-361d350b1c9a admin admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:56:05.253703 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1fac78-192a-40fa-bc37-361d350b1c9a admin admin] PATH_INFO: `/v3/users/aab36b624bcd4a97bb690a0bb163f496` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:56:05.254946 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff1fac78-192a-40fa-bc37-361d350b1c9a admin admin] RBAC: Authorizing `identity:delete_user(user_id=aab36b624bcd4a97bb690a0bb163f496)` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:56:05.256852 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff1fac78-192a-40fa-bc37-361d350b1c9a admin admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:56:05.567118 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.notifications [None req-ff1fac78-192a-40fa-bc37-361d350b1c9a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'aab36b624bcd4a97bb690a0bb163f496'} {{(pid=87057) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:56:05.567816 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.notifications [None req-ff1fac78-192a-40fa-bc37-361d350b1c9a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'aab36b624bcd4a97bb690a0bb163f496'} {{(pid=87057) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:56:05.681470 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.notifications [None req-ff1fac78-192a-40fa-bc37-361d350b1c9a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'aab36b624bcd4a97bb690a0bb163f496'} {{(pid=87057) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:56:05.682417 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.notifications [None req-ff1fac78-192a-40fa-bc37-361d350b1c9a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'aab36b624bcd4a97bb690a0bb163f496'} {{(pid=87057) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:56:05.690589 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.notifications [None req-ff1fac78-192a-40fa-bc37-361d350b1c9a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'aab36b624bcd4a97bb690a0bb163f496'} {{(pid=87057) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:56:05.711005 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 986/1985] 10.222.0.20 () {58 vars in 1224 bytes} [Mon Aug 1 16:56:05 2022] DELETE /identity/v3/users/aab36b624bcd4a97bb690a0bb163f496 => generated 0 bytes in 532 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 16:56:06.197086 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13349b2d-557a-4adb-9d3f-dcdd7c02645f admin admin] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:56:06.198593 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.fernet_utils [None req-13349b2d-557a-4adb-9d3f-dcdd7c02645f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:56:06.211976 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13349b2d-557a-4adb-9d3f-dcdd7c02645f admin admin] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:56:06.231491 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG dbcounter [-] [87058] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=87058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:56:06.276780 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-986b58d9-a882-412e-8af4-4d7f6328e4b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:56:06.277433 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-986b58d9-a882-412e-8af4-4d7f6328e4b8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:56:06.277708 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-986b58d9-a882-412e-8af4-4d7f6328e4b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:56:06.277979 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-986b58d9-a882-412e-8af4-4d7f6328e4b8 admin admin] PATH_INFO: `/v3/projects/cd45893e9425436d8cbeeb8de6da52fe` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:56:06.279179 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-986b58d9-a882-412e-8af4-4d7f6328e4b8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cd45893e9425436d8cbeeb8de6da52fe)` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:56:06.281256 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-986b58d9-a882-412e-8af4-4d7f6328e4b8 admin admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:56:06.396335 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.notifications [None req-986b58d9-a882-412e-8af4-4d7f6328e4b8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cd45893e9425436d8cbeeb8de6da52fe', 'request_id': 'req-986b58d9-a882-412e-8af4-4d7f6328e4b8'} {{(pid=87058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:56:06.403413 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.notifications [None req-986b58d9-a882-412e-8af4-4d7f6328e4b8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cd45893e9425436d8cbeeb8de6da52fe', 'request_id': 'req-986b58d9-a882-412e-8af4-4d7f6328e4b8'} {{(pid=87058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:56:06.409149 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.notifications [None req-986b58d9-a882-412e-8af4-4d7f6328e4b8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cd45893e9425436d8cbeeb8de6da52fe', 'request_id': 'req-986b58d9-a882-412e-8af4-4d7f6328e4b8'} {{(pid=87058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:56:06.492083 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.notifications [None req-986b58d9-a882-412e-8af4-4d7f6328e4b8 admin admin] The token cache is being invalidate because project cd45893e9425436d8cbeeb8de6da52fe was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87058) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 16:56:06.492513 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.notifications [None req-986b58d9-a882-412e-8af4-4d7f6328e4b8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:56:06.493647 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.notifications [None req-986b58d9-a882-412e-8af4-4d7f6328e4b8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:56:06.495236 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 1000/1986] 10.222.0.20 () {58 vars in 1233 bytes} [Mon Aug 1 16:56:06 2022] DELETE /identity/v3/projects/cd45893e9425436d8cbeeb8de6da52fe => generated 0 bytes in 299 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 16:56:06.500286 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e2ab3e-0425-4bdb-8e40-29cf787f38d2 None None] REQUEST_METHOD: `POST` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:56:06.500785 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e2ab3e-0425-4bdb-8e40-29cf787f38d2 None None] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:56:06.500905 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e2ab3e-0425-4bdb-8e40-29cf787f38d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:56:06.520756 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.auth.core [None req-d3e2ab3e-0425-4bdb-8e40-29cf787f38d2 None None] MFA Rules not processed for user `1007f8e379b74e169b98fb7c3c0ba5b4`. Rule list: `[]` (Enabled: `True`). {{(pid=87057) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:56:06.521906 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.fernet_utils [None req-d3e2ab3e-0425-4bdb-8e40-29cf787f38d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87057) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:56:06.581694 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 987/1987] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 1 16:56:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:56:06.590091 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa0d954-00e7-4d14-98ae-8fe776625087 None None] REQUEST_METHOD: `POST` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:56:06.590414 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa0d954-00e7-4d14-98ae-8fe776625087 None None] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:56:06.590695 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa0d954-00e7-4d14-98ae-8fe776625087 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:56:06.612172 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.auth.core [None req-efa0d954-00e7-4d14-98ae-8fe776625087 None None] MFA Rules not processed for user `1007f8e379b74e169b98fb7c3c0ba5b4`. Rule list: `[]` (Enabled: `True`). {{(pid=87058) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:56:06.613396 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.fernet_utils [None req-efa0d954-00e7-4d14-98ae-8fe776625087 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:56:06.622465 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 1001/1988] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 1 16:56:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:56:06.627744 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e2ab3e-0425-4bdb-8e40-29cf787f38d2 None None] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:56:06.638575 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e2ab3e-0425-4bdb-8e40-29cf787f38d2 None None] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:56:06.643612 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d3fdf14-6afc-4988-948a-8f976f529bad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:56:06.644310 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3fdf14-6afc-4988-948a-8f976f529bad admin admin] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:56:06.644674 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3fdf14-6afc-4988-948a-8f976f529bad admin admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:56:06.645017 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3fdf14-6afc-4988-948a-8f976f529bad admin admin] PATH_INFO: `/v3/domains` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:56:06.645880 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d3fdf14-6afc-4988-948a-8f976f529bad admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87057) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:56:06.647170 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d3fdf14-6afc-4988-948a-8f976f529bad admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:56:06.649242 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d3fdf14-6afc-4988-948a-8f976f529bad admin admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:56:06.665915 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 988/1989] 10.222.0.20 () {58 vars in 1153 bytes} [Mon Aug 1 16:56:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:56:06.670339 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efa0d954-00e7-4d14-98ae-8fe776625087 None None] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:56:06.682076 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efa0d954-00e7-4d14-98ae-8fe776625087 None None] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:56:06.689265 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc5d913d-670a-46c3-8372-6dcd3f04106b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:56:06.690760 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5d913d-670a-46c3-8372-6dcd3f04106b admin admin] REQUEST_METHOD: `POST` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:56:06.691110 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5d913d-670a-46c3-8372-6dcd3f04106b admin admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:56:06.691453 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5d913d-670a-46c3-8372-6dcd3f04106b admin admin] PATH_INFO: `/v3/projects` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:56:06.692936 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc5d913d-670a-46c3-8372-6dcd3f04106b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:56:06.695928 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc5d913d-670a-46c3-8372-6dcd3f04106b admin admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:56:06.784500 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 1002/1990] 10.222.0.20 () {60 vars in 1153 bytes} [Mon Aug 1 16:56:06 2022] POST /identity/v3/projects => generated 407 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 16:56:06.789629 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d3fdf14-6afc-4988-948a-8f976f529bad admin admin] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:56:06.799610 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d3fdf14-6afc-4988-948a-8f976f529bad admin admin] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:56:06.827504 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG dbcounter [-] [87057] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87057) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:56:06.858564 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-103c1ae3-a525-40ff-8c46-a3cbcafbf4e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:56:06.859195 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103c1ae3-a525-40ff-8c46-a3cbcafbf4e5 admin admin] REQUEST_METHOD: `POST` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:56:06.859478 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103c1ae3-a525-40ff-8c46-a3cbcafbf4e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:56:06.859779 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103c1ae3-a525-40ff-8c46-a3cbcafbf4e5 admin admin] PATH_INFO: `/v3/users` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:56:06.860466 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-103c1ae3-a525-40ff-8c46-a3cbcafbf4e5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:56:06.862438 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-103c1ae3-a525-40ff-8c46-a3cbcafbf4e5 admin admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:56:06.933130 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 989/1991] 10.222.0.20 () {60 vars in 1144 bytes} [Mon Aug 1 16:56:06 2022] POST /identity/v3/users => generated 304 bytes in 144 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 16:56:06.937822 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc5d913d-670a-46c3-8372-6dcd3f04106b admin admin] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:56:06.948550 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc5d913d-670a-46c3-8372-6dcd3f04106b admin admin] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:56:06.953490 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3118283c-cdd3-45f9-ba64-dd3dd64b5d64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:56:06.954143 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3118283c-cdd3-45f9-ba64-dd3dd64b5d64 admin admin] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:56:06.954514 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3118283c-cdd3-45f9-ba64-dd3dd64b5d64 admin admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:56:06.963410 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3118283c-cdd3-45f9-ba64-dd3dd64b5d64 admin admin] PATH_INFO: `/v3/roles` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:56:06.963410 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3118283c-cdd3-45f9-ba64-dd3dd64b5d64 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:56:06.963410 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3118283c-cdd3-45f9-ba64-dd3dd64b5d64 admin admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:56:06.968306 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 1003/1992] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Aug 1 16:56:06 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 16:56:06.972334 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-103c1ae3-a525-40ff-8c46-a3cbcafbf4e5 admin admin] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:56:06.983243 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-103c1ae3-a525-40ff-8c46-a3cbcafbf4e5 admin admin] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:56:06.988122 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17cbf329-120a-4c76-b5ee-d4efe6230f0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:56:06.988720 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17cbf329-120a-4c76-b5ee-d4efe6230f0b admin admin] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:56:06.989295 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17cbf329-120a-4c76-b5ee-d4efe6230f0b admin admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:56:06.989295 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17cbf329-120a-4c76-b5ee-d4efe6230f0b admin admin] PATH_INFO: `/v3/roles` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:56:06.990099 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17cbf329-120a-4c76-b5ee-d4efe6230f0b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:56:06.991699 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17cbf329-120a-4c76-b5ee-d4efe6230f0b admin admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:56:07.002890 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 990/1993] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Aug 1 16:56:06 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 16:56:07.007135 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3118283c-cdd3-45f9-ba64-dd3dd64b5d64 admin admin] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:56:07.016959 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3118283c-cdd3-45f9-ba64-dd3dd64b5d64 admin admin] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:56:07.021616 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f7b0c5e-d96b-44a5-9b5b-5a11977e1bc7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:56:07.022304 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7b0c5e-d96b-44a5-9b5b-5a11977e1bc7 admin admin] REQUEST_METHOD: `PUT` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:56:07.022585 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7b0c5e-d96b-44a5-9b5b-5a11977e1bc7 admin admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:56:07.022859 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7b0c5e-d96b-44a5-9b5b-5a11977e1bc7 admin admin] PATH_INFO: `/v3/projects/6e818621422f4ce7835c03e96bd4afbf/users/1e64c30e6b704a40a57115f030356f21/roles/0999d3da63a545f18ff3252afdc90518` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:56:07.037198 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f7b0c5e-d96b-44a5-9b5b-5a11977e1bc7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6e818621422f4ce7835c03e96bd4afbf, user_id=1e64c30e6b704a40a57115f030356f21, role_id=0999d3da63a545f18ff3252afdc90518)` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:56:07.039892 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f7b0c5e-d96b-44a5-9b5b-5a11977e1bc7 admin admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:56:07.114717 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 1004/1994] 10.222.0.20 () {60 vars in 1483 bytes} [Mon Aug 1 16:56:07 2022] PUT /identity/v3/projects/6e818621422f4ce7835c03e96bd4afbf/users/1e64c30e6b704a40a57115f030356f21/roles/0999d3da63a545f18ff3252afdc90518 => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 16:56:07.126128 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d086ac9-3930-40b5-b9e3-a0917c9c542d None None] REQUEST_METHOD: `POST` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:56:07.126419 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d086ac9-3930-40b5-b9e3-a0917c9c542d None None] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:56:07.126698 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d086ac9-3930-40b5-b9e3-a0917c9c542d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:56:07.155201 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.auth.core [None req-0d086ac9-3930-40b5-b9e3-a0917c9c542d None None] MFA Rules not processed for user `1e64c30e6b704a40a57115f030356f21`. Rule list: `[]` (Enabled: `True`). {{(pid=87057) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:56:07.156416 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.fernet_utils [None req-0d086ac9-3930-40b5-b9e3-a0917c9c542d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87057) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:56:07.219355 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 991/1995] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 1 16:56:07 2022] POST /identity/v3/auth/tokens => generated 3274 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:56:07.223823 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f7b0c5e-d96b-44a5-9b5b-5a11977e1bc7 admin admin] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:56:07.269154 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG dbcounter [-] [87058] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=87058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:56:07.285475 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f7b0c5e-d96b-44a5-9b5b-5a11977e1bc7 admin admin] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:56:07.290136 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d572b85-c573-4779-9213-4b4c5757e1c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:56:07.291431 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d572b85-c573-4779-9213-4b4c5757e1c2 admin admin] REQUEST_METHOD: `POST` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:56:07.291715 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d572b85-c573-4779-9213-4b4c5757e1c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:56:07.291993 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d572b85-c573-4779-9213-4b4c5757e1c2 admin admin] PATH_INFO: `/v3/projects` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:56:07.292660 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d572b85-c573-4779-9213-4b4c5757e1c2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:56:07.294575 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d572b85-c573-4779-9213-4b4c5757e1c2 admin admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:56:07.367907 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 1005/1996] 10.222.0.20 () {60 vars in 1153 bytes} [Mon Aug 1 16:56:07 2022] POST /identity/v3/projects => generated 407 bytes in 144 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 16:56:07.372017 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d086ac9-3930-40b5-b9e3-a0917c9c542d None None] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:56:07.382794 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d086ac9-3930-40b5-b9e3-a0917c9c542d None None] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:56:07.438102 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a8d259b-4d23-411f-928e-573c0badd6a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:56:07.438718 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8d259b-4d23-411f-928e-573c0badd6a2 admin admin] REQUEST_METHOD: `POST` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:56:07.439001 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8d259b-4d23-411f-928e-573c0badd6a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:56:07.439282 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8d259b-4d23-411f-928e-573c0badd6a2 admin admin] PATH_INFO: `/v3/users` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:56:07.439946 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a8d259b-4d23-411f-928e-573c0badd6a2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:56:07.441875 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a8d259b-4d23-411f-928e-573c0badd6a2 admin admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:56:07.532638 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 992/1997] 10.222.0.20 () {60 vars in 1144 bytes} [Mon Aug 1 16:56:07 2022] POST /identity/v3/users => generated 310 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 16:56:07.536631 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d572b85-c573-4779-9213-4b4c5757e1c2 admin admin] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:56:07.547877 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d572b85-c573-4779-9213-4b4c5757e1c2 admin admin] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:56:07.552542 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a20b5d1-49f4-4da1-bdcf-8933b0cdf05f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:56:07.553156 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a20b5d1-49f4-4da1-bdcf-8933b0cdf05f admin admin] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:56:07.553489 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a20b5d1-49f4-4da1-bdcf-8933b0cdf05f admin admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:56:07.553807 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a20b5d1-49f4-4da1-bdcf-8933b0cdf05f admin admin] PATH_INFO: `/v3/roles` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:56:07.554517 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a20b5d1-49f4-4da1-bdcf-8933b0cdf05f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:56:07.556193 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a20b5d1-49f4-4da1-bdcf-8933b0cdf05f admin admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:56:07.566918 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 1006/1998] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Aug 1 16:56:07 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 16:56:07.570940 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a8d259b-4d23-411f-928e-573c0badd6a2 admin admin] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:56:07.581737 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a8d259b-4d23-411f-928e-573c0badd6a2 admin admin] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:56:07.586504 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7712216-1faf-461a-b235-aa73e026f95c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:56:07.587765 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7712216-1faf-461a-b235-aa73e026f95c admin admin] REQUEST_METHOD: `PUT` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:56:07.588223 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7712216-1faf-461a-b235-aa73e026f95c admin admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:56:07.588500 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7712216-1faf-461a-b235-aa73e026f95c admin admin] PATH_INFO: `/v3/projects/fa80280bf94d4df7a889c93ab3b2f84a/users/4a08b8b5945b4b438bd47b8ce5ed73f3/roles/440bb943dd5b46fea038b903b6eaaf81` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:56:07.603430 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7712216-1faf-461a-b235-aa73e026f95c admin admin] RBAC: Authorizing `identity:create_grant(project_id=fa80280bf94d4df7a889c93ab3b2f84a, user_id=4a08b8b5945b4b438bd47b8ce5ed73f3, role_id=440bb943dd5b46fea038b903b6eaaf81)` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:56:07.606308 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7712216-1faf-461a-b235-aa73e026f95c admin admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:56:07.687929 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 993/1999] 10.222.0.20 () {60 vars in 1483 bytes} [Mon Aug 1 16:56:07 2022] PUT /identity/v3/projects/fa80280bf94d4df7a889c93ab3b2f84a/users/4a08b8b5945b4b438bd47b8ce5ed73f3/roles/440bb943dd5b46fea038b903b6eaaf81 => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 16:56:07.692038 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a20b5d1-49f4-4da1-bdcf-8933b0cdf05f admin admin] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:56:07.703702 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a20b5d1-49f4-4da1-bdcf-8933b0cdf05f admin admin] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:56:07.780596 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e06bc17a-2b62-4fd3-aaa7-37b99eb7c25f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:56:07.781274 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06bc17a-2b62-4fd3-aaa7-37b99eb7c25f admin admin] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:56:07.781558 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06bc17a-2b62-4fd3-aaa7-37b99eb7c25f admin admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:56:07.781831 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06bc17a-2b62-4fd3-aaa7-37b99eb7c25f admin admin] PATH_INFO: `/v3/roles` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:56:07.782626 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e06bc17a-2b62-4fd3-aaa7-37b99eb7c25f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:56:07.788013 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e06bc17a-2b62-4fd3-aaa7-37b99eb7c25f admin admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:56:07.794549 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 1007/2000] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Aug 1 16:56:07 2022] GET /identity/v3/roles => generated 1420 bytes in 103 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 01 16:56:07.798755 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7712216-1faf-461a-b235-aa73e026f95c admin admin] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:56:07.804246 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG dbcounter [-] [87057] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=87057) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:56:07.810190 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7712216-1faf-461a-b235-aa73e026f95c admin admin] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:56:07.814901 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27843e33-bb87-4c66-8205-c6126fefb064 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:56:07.816695 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27843e33-bb87-4c66-8205-c6126fefb064 admin admin] REQUEST_METHOD: `PUT` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:56:07.816695 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27843e33-bb87-4c66-8205-c6126fefb064 admin admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:56:07.816695 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27843e33-bb87-4c66-8205-c6126fefb064 admin admin] PATH_INFO: `/v3/projects/fa80280bf94d4df7a889c93ab3b2f84a/users/4a08b8b5945b4b438bd47b8ce5ed73f3/roles/440bb943dd5b46fea038b903b6eaaf81` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:56:07.818251 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27843e33-bb87-4c66-8205-c6126fefb064 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fa80280bf94d4df7a889c93ab3b2f84a, user_id=4a08b8b5945b4b438bd47b8ce5ed73f3, role_id=440bb943dd5b46fea038b903b6eaaf81)` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:56:07.820972 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27843e33-bb87-4c66-8205-c6126fefb064 admin admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:56:07.832512 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 994/2001] 10.222.0.20 () {60 vars in 1483 bytes} [Mon Aug 1 16:56:07 2022] PUT /identity/v3/projects/fa80280bf94d4df7a889c93ab3b2f84a/users/4a08b8b5945b4b438bd47b8ce5ed73f3/roles/440bb943dd5b46fea038b903b6eaaf81 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 16:56:07.843240 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b303bc-908e-438b-bbb6-622e36da753b None None] REQUEST_METHOD: `POST` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:56:07.843537 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b303bc-908e-438b-bbb6-622e36da753b None None] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:56:07.843815 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b303bc-908e-438b-bbb6-622e36da753b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:56:07.873682 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.auth.core [None req-a7b303bc-908e-438b-bbb6-622e36da753b None None] MFA Rules not processed for user `4a08b8b5945b4b438bd47b8ce5ed73f3`. Rule list: `[]` (Enabled: `True`). {{(pid=87058) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:56:07.874975 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.fernet_utils [None req-a7b303bc-908e-438b-bbb6-622e36da753b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:56:07.949838 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 1008/2002] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 1 16:56:07 2022] POST /identity/v3/auth/tokens => generated 3341 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:56:07.960810 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27843e33-bb87-4c66-8205-c6126fefb064 admin admin] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:56:07.962616 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.fernet_utils [None req-27843e33-bb87-4c66-8205-c6126fefb064 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87057) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:56:08.023830 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27843e33-bb87-4c66-8205-c6126fefb064 admin admin] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:56:08.027656 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d54d95c3-7fee-4192-86fa-a52e88c6270f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25970b52ca15448fa1abb1f974d64038', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b992d6a65df4bf9b27c13192d57c5f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:56:08.028336 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54d95c3-7fee-4192-86fa-a52e88c6270f service cinder] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:56:08.028619 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54d95c3-7fee-4192-86fa-a52e88c6270f service cinder] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:56:08.028903 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54d95c3-7fee-4192-86fa-a52e88c6270f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:56:08.041190 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d54d95c3-7fee-4192-86fa-a52e88c6270f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:56:08.041190 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d54d95c3-7fee-4192-86fa-a52e88c6270f service cinder] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:56:08.050391 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 995/2003] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 1 16:56:07 2022] GET /identity/v3/auth/tokens => generated 3341 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 16:56:08.407289 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b303bc-908e-438b-bbb6-622e36da753b None None] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:56:08.418149 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.fernet_utils [None req-a7b303bc-908e-438b-bbb6-622e36da753b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:56:08.420436 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b303bc-908e-438b-bbb6-622e36da753b None None] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:56:08.423967 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a42761cf-24eb-4e84-aecc-b5f2be25f7d3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25970b52ca15448fa1abb1f974d64038', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b992d6a65df4bf9b27c13192d57c5f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:56:08.424573 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42761cf-24eb-4e84-aecc-b5f2be25f7d3 service cinder] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:56:08.424910 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42761cf-24eb-4e84-aecc-b5f2be25f7d3 service cinder] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:56:08.425186 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42761cf-24eb-4e84-aecc-b5f2be25f7d3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:56:08.463725 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG dbcounter [-] [87058] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:56:08.482252 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a42761cf-24eb-4e84-aecc-b5f2be25f7d3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:56:08.483697 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a42761cf-24eb-4e84-aecc-b5f2be25f7d3 service cinder] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:56:08.492956 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 1009/2004] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 1 16:56:08 2022] GET /identity/v3/auth/tokens => generated 3274 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 16:56:14.132299 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d54d95c3-7fee-4192-86fa-a52e88c6270f service cinder] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:56:14.133774 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.fernet_utils [None req-d54d95c3-7fee-4192-86fa-a52e88c6270f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87057) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:56:14.200003 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d54d95c3-7fee-4192-86fa-a52e88c6270f service cinder] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:56:14.204957 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82831aa1-abe1-4be9-9db0-326ae419998e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:56:14.205604 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82831aa1-abe1-4be9-9db0-326ae419998e admin admin] REQUEST_METHOD: `DELETE` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:56:14.205883 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82831aa1-abe1-4be9-9db0-326ae419998e admin admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:56:14.206215 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82831aa1-abe1-4be9-9db0-326ae419998e admin admin] PATH_INFO: `/v3/users/8377f3912caf4b6db3e939e4daeb80ff` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:56:14.207386 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82831aa1-abe1-4be9-9db0-326ae419998e admin admin] RBAC: Authorizing `identity:delete_user(user_id=8377f3912caf4b6db3e939e4daeb80ff)` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:56:14.209317 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82831aa1-abe1-4be9-9db0-326ae419998e admin admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:56:14.408083 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.notifications [None req-82831aa1-abe1-4be9-9db0-326ae419998e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8377f3912caf4b6db3e939e4daeb80ff'} {{(pid=87057) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:56:14.408729 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.notifications [None req-82831aa1-abe1-4be9-9db0-326ae419998e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8377f3912caf4b6db3e939e4daeb80ff'} {{(pid=87057) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:56:14.491758 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.notifications [None req-82831aa1-abe1-4be9-9db0-326ae419998e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8377f3912caf4b6db3e939e4daeb80ff'} {{(pid=87057) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:56:14.492350 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.notifications [None req-82831aa1-abe1-4be9-9db0-326ae419998e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8377f3912caf4b6db3e939e4daeb80ff'} {{(pid=87057) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:56:14.500496 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.notifications [None req-82831aa1-abe1-4be9-9db0-326ae419998e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8377f3912caf4b6db3e939e4daeb80ff'} {{(pid=87057) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:56:14.517561 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 996/2005] 10.222.0.20 () {58 vars in 1224 bytes} [Mon Aug 1 16:56:14 2022] DELETE /identity/v3/users/8377f3912caf4b6db3e939e4daeb80ff => generated 0 bytes in 386 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 16:56:14.522065 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ae43ea-33ce-4321-8ec3-6e65fac95716 None None] REQUEST_METHOD: `POST` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:56:14.522554 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ae43ea-33ce-4321-8ec3-6e65fac95716 None None] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:56:14.522646 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ae43ea-33ce-4321-8ec3-6e65fac95716 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:56:14.542009 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.auth.core [None req-c6ae43ea-33ce-4321-8ec3-6e65fac95716 None None] MFA Rules not processed for user `1007f8e379b74e169b98fb7c3c0ba5b4`. Rule list: `[]` (Enabled: `True`). {{(pid=87058) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:56:14.543249 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.fernet_utils [None req-c6ae43ea-33ce-4321-8ec3-6e65fac95716 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:56:14.603387 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 1010/2006] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 1 16:56:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:56:14.612968 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82831aa1-abe1-4be9-9db0-326ae419998e admin admin] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:56:14.614437 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.fernet_utils [None req-82831aa1-abe1-4be9-9db0-326ae419998e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87057) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:56:14.628119 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG dbcounter [-] [87057] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=87057) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:56:14.672149 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82831aa1-abe1-4be9-9db0-326ae419998e admin admin] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:56:14.675934 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b997c414-9799-40f8-96a4-572e73f4d1f8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d9bd8d649154c979be3744bfd016f95', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b992d6a65df4bf9b27c13192d57c5f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:56:14.676501 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b997c414-9799-40f8-96a4-572e73f4d1f8 service neutron] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:56:14.676783 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b997c414-9799-40f8-96a4-572e73f4d1f8 service neutron] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:56:14.677061 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b997c414-9799-40f8-96a4-572e73f4d1f8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:56:14.686217 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b997c414-9799-40f8-96a4-572e73f4d1f8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:56:14.687650 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b997c414-9799-40f8-96a4-572e73f4d1f8 service neutron] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:56:14.697331 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 997/2007] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 1 16:56:14 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 16:56:15.202023 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ae43ea-33ce-4321-8ec3-6e65fac95716 None None] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:56:15.206996 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.fernet_utils [None req-c6ae43ea-33ce-4321-8ec3-6e65fac95716 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:56:15.215989 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ae43ea-33ce-4321-8ec3-6e65fac95716 None None] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:56:15.220724 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9823679-d385-452c-ae56-78520621427d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:56:15.221378 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9823679-d385-452c-ae56-78520621427d admin admin] REQUEST_METHOD: `DELETE` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:56:15.221659 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9823679-d385-452c-ae56-78520621427d admin admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:56:15.221978 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9823679-d385-452c-ae56-78520621427d admin admin] PATH_INFO: `/v3/projects/7a6768f1a61941aa943ee21041280cb8` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:56:15.223531 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9823679-d385-452c-ae56-78520621427d admin admin] RBAC: Authorizing `identity:delete_project(project_id=7a6768f1a61941aa943ee21041280cb8)` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:56:15.225454 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9823679-d385-452c-ae56-78520621427d admin admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:56:15.333545 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.notifications [None req-a9823679-d385-452c-ae56-78520621427d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7a6768f1a61941aa943ee21041280cb8', 'request_id': 'req-a9823679-d385-452c-ae56-78520621427d'} {{(pid=87058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:56:15.340825 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.notifications [None req-a9823679-d385-452c-ae56-78520621427d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7a6768f1a61941aa943ee21041280cb8', 'request_id': 'req-a9823679-d385-452c-ae56-78520621427d'} {{(pid=87058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:56:15.346568 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.notifications [None req-a9823679-d385-452c-ae56-78520621427d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7a6768f1a61941aa943ee21041280cb8', 'request_id': 'req-a9823679-d385-452c-ae56-78520621427d'} {{(pid=87058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:56:15.428494 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.notifications [None req-a9823679-d385-452c-ae56-78520621427d admin admin] The token cache is being invalidate because project 7a6768f1a61941aa943ee21041280cb8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87058) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 16:56:15.428782 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.notifications [None req-a9823679-d385-452c-ae56-78520621427d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:56:15.429412 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.notifications [None req-a9823679-d385-452c-ae56-78520621427d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:56:15.431680 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 1011/2008] 10.222.0.20 () {58 vars in 1233 bytes} [Mon Aug 1 16:56:15 2022] DELETE /identity/v3/projects/7a6768f1a61941aa943ee21041280cb8 => generated 0 bytes in 230 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 16:56:15.435685 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b997c414-9799-40f8-96a4-572e73f4d1f8 service neutron] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:56:15.445274 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b997c414-9799-40f8-96a4-572e73f4d1f8 service neutron] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:56:15.500632 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c36c8821-05ee-44e8-9370-23edf1b8a47c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:56:15.501273 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36c8821-05ee-44e8-9370-23edf1b8a47c admin admin] REQUEST_METHOD: `DELETE` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:56:15.501549 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36c8821-05ee-44e8-9370-23edf1b8a47c admin admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:56:15.501829 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36c8821-05ee-44e8-9370-23edf1b8a47c admin admin] PATH_INFO: `/v3/users/d6faaa1f4f66409e813e677f4d4fa79a` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:56:15.503005 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c36c8821-05ee-44e8-9370-23edf1b8a47c admin admin] RBAC: Authorizing `identity:delete_user(user_id=d6faaa1f4f66409e813e677f4d4fa79a)` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:56:15.504997 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c36c8821-05ee-44e8-9370-23edf1b8a47c admin admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:56:15.653724 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.notifications [None req-c36c8821-05ee-44e8-9370-23edf1b8a47c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd6faaa1f4f66409e813e677f4d4fa79a'} {{(pid=87057) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:56:15.654316 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.notifications [None req-c36c8821-05ee-44e8-9370-23edf1b8a47c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd6faaa1f4f66409e813e677f4d4fa79a'} {{(pid=87057) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:56:15.737269 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.notifications [None req-c36c8821-05ee-44e8-9370-23edf1b8a47c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd6faaa1f4f66409e813e677f4d4fa79a'} {{(pid=87057) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:56:15.737965 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.notifications [None req-c36c8821-05ee-44e8-9370-23edf1b8a47c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd6faaa1f4f66409e813e677f4d4fa79a'} {{(pid=87057) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:56:15.745727 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.notifications [None req-c36c8821-05ee-44e8-9370-23edf1b8a47c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd6faaa1f4f66409e813e677f4d4fa79a'} {{(pid=87057) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:56:15.763682 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 998/2009] 10.222.0.20 () {58 vars in 1224 bytes} [Mon Aug 1 16:56:15 2022] DELETE /identity/v3/users/d6faaa1f4f66409e813e677f4d4fa79a => generated 0 bytes in 328 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 16:56:16.191111 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9823679-d385-452c-ae56-78520621427d admin admin] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:56:16.192565 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.fernet_utils [None req-a9823679-d385-452c-ae56-78520621427d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:56:16.225955 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9823679-d385-452c-ae56-78520621427d admin admin] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:56:16.322308 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a4045a-956a-4427-89e0-46e279f4e429 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:56:16.323185 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a4045a-956a-4427-89e0-46e279f4e429 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:56:16.323469 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a4045a-956a-4427-89e0-46e279f4e429 admin admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:56:16.323748 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a4045a-956a-4427-89e0-46e279f4e429 admin admin] PATH_INFO: `/v3/projects/a9d4ce10ed224fe2b77fb79e6338aef8` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:56:16.324966 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3a4045a-956a-4427-89e0-46e279f4e429 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a9d4ce10ed224fe2b77fb79e6338aef8)` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:56:16.326919 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3a4045a-956a-4427-89e0-46e279f4e429 admin admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:56:16.329572 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG dbcounter [-] [87058] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=87058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:56:16.419020 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.notifications [None req-e3a4045a-956a-4427-89e0-46e279f4e429 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a9d4ce10ed224fe2b77fb79e6338aef8', 'request_id': 'req-e3a4045a-956a-4427-89e0-46e279f4e429'} {{(pid=87058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:56:16.425096 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.notifications [None req-e3a4045a-956a-4427-89e0-46e279f4e429 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a9d4ce10ed224fe2b77fb79e6338aef8', 'request_id': 'req-e3a4045a-956a-4427-89e0-46e279f4e429'} {{(pid=87058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:56:16.431179 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.notifications [None req-e3a4045a-956a-4427-89e0-46e279f4e429 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a9d4ce10ed224fe2b77fb79e6338aef8', 'request_id': 'req-e3a4045a-956a-4427-89e0-46e279f4e429'} {{(pid=87058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:56:16.498971 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.notifications [None req-e3a4045a-956a-4427-89e0-46e279f4e429 admin admin] The token cache is being invalidate because project a9d4ce10ed224fe2b77fb79e6338aef8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87058) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 16:56:16.499274 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.notifications [None req-e3a4045a-956a-4427-89e0-46e279f4e429 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:56:16.499881 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.notifications [None req-e3a4045a-956a-4427-89e0-46e279f4e429 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:56:16.501448 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 1012/2010] 10.222.0.20 () {58 vars in 1233 bytes} [Mon Aug 1 16:56:16 2022] DELETE /identity/v3/projects/a9d4ce10ed224fe2b77fb79e6338aef8 => generated 0 bytes in 311 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 16:56:16.506538 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91cd373d-cf9e-4cf7-83c4-bb72b610a607 None None] REQUEST_METHOD: `POST` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:56:16.507266 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91cd373d-cf9e-4cf7-83c4-bb72b610a607 None None] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:56:16.507487 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91cd373d-cf9e-4cf7-83c4-bb72b610a607 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:56:16.527042 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.auth.core [None req-91cd373d-cf9e-4cf7-83c4-bb72b610a607 None None] MFA Rules not processed for user `1007f8e379b74e169b98fb7c3c0ba5b4`. Rule list: `[]` (Enabled: `True`). {{(pid=87057) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:56:16.528233 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.fernet_utils [None req-91cd373d-cf9e-4cf7-83c4-bb72b610a607 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87057) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:56:16.558941 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG dbcounter [-] [87057] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87057) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:56:16.588621 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 999/2011] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 1 16:56:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:56:16.597502 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408a0927-ff25-4e03-ba77-872ecc60b950 None None] REQUEST_METHOD: `POST` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:56:16.597789 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408a0927-ff25-4e03-ba77-872ecc60b950 None None] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:56:16.598103 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408a0927-ff25-4e03-ba77-872ecc60b950 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:56:16.618954 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.auth.core [None req-408a0927-ff25-4e03-ba77-872ecc60b950 None None] MFA Rules not processed for user `1007f8e379b74e169b98fb7c3c0ba5b4`. Rule list: `[]` (Enabled: `True`). {{(pid=87058) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:56:16.620174 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.fernet_utils [None req-408a0927-ff25-4e03-ba77-872ecc60b950 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:56:16.628167 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 1013/2012] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 1 16:56:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:56:16.632445 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91cd373d-cf9e-4cf7-83c4-bb72b610a607 None None] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:56:16.642556 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91cd373d-cf9e-4cf7-83c4-bb72b610a607 None None] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:56:16.647040 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd11542-d010-4c16-acde-314ae60bf579 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:56:16.648511 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd11542-d010-4c16-acde-314ae60bf579 admin admin] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:56:16.648511 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd11542-d010-4c16-acde-314ae60bf579 admin admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:56:16.648511 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd11542-d010-4c16-acde-314ae60bf579 admin admin] PATH_INFO: `/v3/domains` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:56:16.648827 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afd11542-d010-4c16-acde-314ae60bf579 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87057) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:56:16.649124 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afd11542-d010-4c16-acde-314ae60bf579 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:56:16.651619 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afd11542-d010-4c16-acde-314ae60bf579 admin admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:56:16.662755 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 1000/2013] 10.222.0.20 () {58 vars in 1153 bytes} [Mon Aug 1 16:56:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 16:56:16.667179 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-408a0927-ff25-4e03-ba77-872ecc60b950 None None] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:56:16.677194 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-408a0927-ff25-4e03-ba77-872ecc60b950 None None] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:56:16.681234 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f277bc-3c77-4a06-83c0-c5100fe99736 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:56:16.681831 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f277bc-3c77-4a06-83c0-c5100fe99736 admin admin] REQUEST_METHOD: `POST` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:56:16.682134 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f277bc-3c77-4a06-83c0-c5100fe99736 admin admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:56:16.682405 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f277bc-3c77-4a06-83c0-c5100fe99736 admin admin] PATH_INFO: `/v3/projects` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:56:16.683184 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85f277bc-3c77-4a06-83c0-c5100fe99736 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:56:16.685132 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85f277bc-3c77-4a06-83c0-c5100fe99736 admin admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:56:16.783949 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 1014/2014] 10.222.0.20 () {60 vars in 1153 bytes} [Mon Aug 1 16:56:16 2022] POST /identity/v3/projects => generated 387 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 16:56:16.788076 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd11542-d010-4c16-acde-314ae60bf579 admin admin] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:56:16.798159 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd11542-d010-4c16-acde-314ae60bf579 admin admin] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:56:16.854432 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77dbacc2-69a0-4851-8635-2bb0d6b8fd4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:56:16.855312 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77dbacc2-69a0-4851-8635-2bb0d6b8fd4a admin admin] REQUEST_METHOD: `POST` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:56:16.855312 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77dbacc2-69a0-4851-8635-2bb0d6b8fd4a admin admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:56:16.855575 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77dbacc2-69a0-4851-8635-2bb0d6b8fd4a admin admin] PATH_INFO: `/v3/users` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:56:16.856242 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77dbacc2-69a0-4851-8635-2bb0d6b8fd4a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:56:16.858211 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77dbacc2-69a0-4851-8635-2bb0d6b8fd4a admin admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:56:16.935508 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 1001/2015] 10.222.0.20 () {60 vars in 1144 bytes} [Mon Aug 1 16:56:16 2022] POST /identity/v3/users => generated 294 bytes in 145 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 16:56:16.936635 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f277bc-3c77-4a06-83c0-c5100fe99736 admin admin] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:56:16.947726 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f277bc-3c77-4a06-83c0-c5100fe99736 admin admin] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:56:16.952531 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77ce9fb0-300e-434e-b860-5ef6c33eebd0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:56:16.953140 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ce9fb0-300e-434e-b860-5ef6c33eebd0 admin admin] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:56:16.953413 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ce9fb0-300e-434e-b860-5ef6c33eebd0 admin admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:56:16.953681 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ce9fb0-300e-434e-b860-5ef6c33eebd0 admin admin] PATH_INFO: `/v3/roles` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:56:16.954407 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77ce9fb0-300e-434e-b860-5ef6c33eebd0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:56:16.956000 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77ce9fb0-300e-434e-b860-5ef6c33eebd0 admin admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:56:16.966540 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 1015/2016] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Aug 1 16:56:16 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 16:56:16.971097 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77dbacc2-69a0-4851-8635-2bb0d6b8fd4a admin admin] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:56:16.981802 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77dbacc2-69a0-4851-8635-2bb0d6b8fd4a admin admin] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:56:16.986767 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-735e2fcb-1083-40ec-afb3-c6477bca565f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:56:16.987384 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735e2fcb-1083-40ec-afb3-c6477bca565f admin admin] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:56:16.987662 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735e2fcb-1083-40ec-afb3-c6477bca565f admin admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:56:16.987937 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735e2fcb-1083-40ec-afb3-c6477bca565f admin admin] PATH_INFO: `/v3/roles` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:56:16.988673 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-735e2fcb-1083-40ec-afb3-c6477bca565f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:56:16.990361 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-735e2fcb-1083-40ec-afb3-c6477bca565f admin admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:56:17.002192 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 1002/2017] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Aug 1 16:56:16 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 16:56:17.006509 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77ce9fb0-300e-434e-b860-5ef6c33eebd0 admin admin] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:56:17.017404 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77ce9fb0-300e-434e-b860-5ef6c33eebd0 admin admin] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:56:17.021989 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-627f3bd3-faa7-405a-8bdc-1be384422ea2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:56:17.022683 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627f3bd3-faa7-405a-8bdc-1be384422ea2 admin admin] REQUEST_METHOD: `PUT` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:56:17.022957 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627f3bd3-faa7-405a-8bdc-1be384422ea2 admin admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:56:17.023225 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627f3bd3-faa7-405a-8bdc-1be384422ea2 admin admin] PATH_INFO: `/v3/projects/ffa37dfeafcf49b085bf68046f0bb2ba/users/ad15c781be3a4a67ab2d832948ddf94a/roles/0999d3da63a545f18ff3252afdc90518` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:56:17.038152 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-627f3bd3-faa7-405a-8bdc-1be384422ea2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ffa37dfeafcf49b085bf68046f0bb2ba, user_id=ad15c781be3a4a67ab2d832948ddf94a, role_id=0999d3da63a545f18ff3252afdc90518)` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:56:17.041184 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-627f3bd3-faa7-405a-8bdc-1be384422ea2 admin admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:56:17.114109 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 1016/2018] 10.222.0.20 () {60 vars in 1483 bytes} [Mon Aug 1 16:56:17 2022] PUT /identity/v3/projects/ffa37dfeafcf49b085bf68046f0bb2ba/users/ad15c781be3a4a67ab2d832948ddf94a/roles/0999d3da63a545f18ff3252afdc90518 => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 16:56:17.125279 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d323543-0917-468c-ae70-453575ff36e0 None None] REQUEST_METHOD: `POST` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:56:17.125565 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d323543-0917-468c-ae70-453575ff36e0 None None] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:56:17.125835 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d323543-0917-468c-ae70-453575ff36e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:56:17.154262 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.auth.core [None req-2d323543-0917-468c-ae70-453575ff36e0 None None] MFA Rules not processed for user `ad15c781be3a4a67ab2d832948ddf94a`. Rule list: `[]` (Enabled: `True`). {{(pid=87057) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:56:17.155469 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.fernet_utils [None req-2d323543-0917-468c-ae70-453575ff36e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87057) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:56:17.221964 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 1003/2019] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 1 16:56:17 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 16:56:17.226401 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-627f3bd3-faa7-405a-8bdc-1be384422ea2 admin admin] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:56:17.289679 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-627f3bd3-faa7-405a-8bdc-1be384422ea2 admin admin] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:56:17.294973 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69db5613-16fb-49c0-9615-3355e0dfacae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:56:17.295595 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69db5613-16fb-49c0-9615-3355e0dfacae admin admin] REQUEST_METHOD: `POST` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:56:17.295874 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69db5613-16fb-49c0-9615-3355e0dfacae admin admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:56:17.296136 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69db5613-16fb-49c0-9615-3355e0dfacae admin admin] PATH_INFO: `/v3/projects` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:56:17.296846 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69db5613-16fb-49c0-9615-3355e0dfacae admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:56:17.298777 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69db5613-16fb-49c0-9615-3355e0dfacae admin admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:56:17.397232 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 1017/2020] 10.222.0.20 () {60 vars in 1153 bytes} [Mon Aug 1 16:56:17 2022] POST /identity/v3/projects => generated 387 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 16:56:17.400861 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d323543-0917-468c-ae70-453575ff36e0 None None] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:56:17.414070 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d323543-0917-468c-ae70-453575ff36e0 None None] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:56:17.444308 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG dbcounter [-] [87057] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87057) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:56:17.479036 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0acb9f8a-febc-4cba-8d65-53c35ad329a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:56:17.479561 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0acb9f8a-febc-4cba-8d65-53c35ad329a9 admin admin] REQUEST_METHOD: `POST` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:56:17.479840 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0acb9f8a-febc-4cba-8d65-53c35ad329a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:56:17.480114 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0acb9f8a-febc-4cba-8d65-53c35ad329a9 admin admin] PATH_INFO: `/v3/users` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:56:17.480784 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0acb9f8a-febc-4cba-8d65-53c35ad329a9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:56:17.490208 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0acb9f8a-febc-4cba-8d65-53c35ad329a9 admin admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:56:17.600162 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 1004/2021] 10.222.0.20 () {60 vars in 1144 bytes} [Mon Aug 1 16:56:17 2022] POST /identity/v3/users => generated 300 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 16:56:17.604871 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69db5613-16fb-49c0-9615-3355e0dfacae admin admin] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:56:17.615204 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69db5613-16fb-49c0-9615-3355e0dfacae admin admin] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:56:17.620000 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3356f51c-f669-4312-82d8-f4bc9d225fcd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:56:17.620603 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3356f51c-f669-4312-82d8-f4bc9d225fcd admin admin] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:56:17.620878 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3356f51c-f669-4312-82d8-f4bc9d225fcd admin admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:56:17.621147 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3356f51c-f669-4312-82d8-f4bc9d225fcd admin admin] PATH_INFO: `/v3/roles` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:56:17.621835 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3356f51c-f669-4312-82d8-f4bc9d225fcd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:56:17.623484 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3356f51c-f669-4312-82d8-f4bc9d225fcd admin admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:56:17.633898 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 1018/2022] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Aug 1 16:56:17 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 16:56:17.638221 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0acb9f8a-febc-4cba-8d65-53c35ad329a9 admin admin] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:56:17.648225 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0acb9f8a-febc-4cba-8d65-53c35ad329a9 admin admin] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:56:17.653690 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb542a01-96ff-48e2-99c2-b1c627cba7bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:56:17.654388 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb542a01-96ff-48e2-99c2-b1c627cba7bd admin admin] REQUEST_METHOD: `PUT` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:56:17.654666 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb542a01-96ff-48e2-99c2-b1c627cba7bd admin admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:56:17.654942 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb542a01-96ff-48e2-99c2-b1c627cba7bd admin admin] PATH_INFO: `/v3/projects/aa20fc0789ac45c6bc3268fe9e815db3/users/fa5629136d484b6da03f2165f7db98fd/roles/440bb943dd5b46fea038b903b6eaaf81` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:56:17.671016 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb542a01-96ff-48e2-99c2-b1c627cba7bd admin admin] RBAC: Authorizing `identity:create_grant(project_id=aa20fc0789ac45c6bc3268fe9e815db3, user_id=fa5629136d484b6da03f2165f7db98fd, role_id=440bb943dd5b46fea038b903b6eaaf81)` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:56:17.673705 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb542a01-96ff-48e2-99c2-b1c627cba7bd admin admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:56:17.765692 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 1005/2023] 10.222.0.20 () {60 vars in 1483 bytes} [Mon Aug 1 16:56:17 2022] PUT /identity/v3/projects/aa20fc0789ac45c6bc3268fe9e815db3/users/fa5629136d484b6da03f2165f7db98fd/roles/440bb943dd5b46fea038b903b6eaaf81 => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 16:56:17.769764 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3356f51c-f669-4312-82d8-f4bc9d225fcd admin admin] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:56:17.775585 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG dbcounter [-] [87058] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=87058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:56:17.781082 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3356f51c-f669-4312-82d8-f4bc9d225fcd admin admin] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:56:17.835923 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da2147b1-a265-4204-8a83-c5b329ee4bc9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:56:17.836536 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2147b1-a265-4204-8a83-c5b329ee4bc9 admin admin] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:56:17.837420 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2147b1-a265-4204-8a83-c5b329ee4bc9 admin admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:56:17.837768 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2147b1-a265-4204-8a83-c5b329ee4bc9 admin admin] PATH_INFO: `/v3/roles` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:56:17.838484 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da2147b1-a265-4204-8a83-c5b329ee4bc9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:56:17.840068 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da2147b1-a265-4204-8a83-c5b329ee4bc9 admin admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:56:17.849950 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 1019/2024] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Aug 1 16:56:17 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 16:56:17.855021 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb542a01-96ff-48e2-99c2-b1c627cba7bd admin admin] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:56:17.865446 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb542a01-96ff-48e2-99c2-b1c627cba7bd admin admin] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:56:17.870445 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9cd7c6a-d65f-496b-ae85-a1348d76846c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:56:17.871104 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9cd7c6a-d65f-496b-ae85-a1348d76846c admin admin] REQUEST_METHOD: `PUT` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:56:17.871387 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9cd7c6a-d65f-496b-ae85-a1348d76846c admin admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:56:17.871665 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9cd7c6a-d65f-496b-ae85-a1348d76846c admin admin] PATH_INFO: `/v3/projects/aa20fc0789ac45c6bc3268fe9e815db3/users/fa5629136d484b6da03f2165f7db98fd/roles/440bb943dd5b46fea038b903b6eaaf81` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:56:17.873641 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9cd7c6a-d65f-496b-ae85-a1348d76846c admin admin] RBAC: Authorizing `identity:create_grant(project_id=aa20fc0789ac45c6bc3268fe9e815db3, user_id=fa5629136d484b6da03f2165f7db98fd, role_id=440bb943dd5b46fea038b903b6eaaf81)` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:56:17.876369 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9cd7c6a-d65f-496b-ae85-a1348d76846c admin admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:56:17.886605 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 1006/2025] 10.222.0.20 () {60 vars in 1483 bytes} [Mon Aug 1 16:56:17 2022] PUT /identity/v3/projects/aa20fc0789ac45c6bc3268fe9e815db3/users/fa5629136d484b6da03f2165f7db98fd/roles/440bb943dd5b46fea038b903b6eaaf81 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 16:56:17.897742 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58a2635-c057-4c2a-a20b-5579e81d74a5 None None] REQUEST_METHOD: `POST` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:56:17.898049 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58a2635-c057-4c2a-a20b-5579e81d74a5 None None] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:56:17.898354 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58a2635-c057-4c2a-a20b-5579e81d74a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:56:17.930123 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.auth.core [None req-e58a2635-c057-4c2a-a20b-5579e81d74a5 None None] MFA Rules not processed for user `fa5629136d484b6da03f2165f7db98fd`. Rule list: `[]` (Enabled: `True`). {{(pid=87058) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:56:17.930123 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.fernet_utils [None req-e58a2635-c057-4c2a-a20b-5579e81d74a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:56:18.001715 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 1020/2026] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 1 16:56:17 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 16:56:18.013526 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9cd7c6a-d65f-496b-ae85-a1348d76846c admin admin] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:56:18.015280 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.fernet_utils [None req-e9cd7c6a-d65f-496b-ae85-a1348d76846c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87057) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:56:18.072342 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9cd7c6a-d65f-496b-ae85-a1348d76846c admin admin] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:56:18.075940 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aaa04c1-bdc7-4fb9-915d-602fb9671638 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25970b52ca15448fa1abb1f974d64038', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b992d6a65df4bf9b27c13192d57c5f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:56:18.076552 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aaa04c1-bdc7-4fb9-915d-602fb9671638 service cinder] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:56:18.076842 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aaa04c1-bdc7-4fb9-915d-602fb9671638 service cinder] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:56:18.077118 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aaa04c1-bdc7-4fb9-915d-602fb9671638 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:56:18.085930 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aaa04c1-bdc7-4fb9-915d-602fb9671638 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:56:18.087432 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aaa04c1-bdc7-4fb9-915d-602fb9671638 service cinder] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:56:18.096729 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 1007/2027] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 1 16:56:18 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 16:56:18.161094 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e58a2635-c057-4c2a-a20b-5579e81d74a5 None None] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:56:18.227100 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e58a2635-c057-4c2a-a20b-5579e81d74a5 None None] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:56:18.232602 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d4bc01d-cd33-4ab7-a92f-79883de057d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:56:18.233280 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4bc01d-cd33-4ab7-a92f-79883de057d3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:56:18.233570 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4bc01d-cd33-4ab7-a92f-79883de057d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:56:18.233847 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4bc01d-cd33-4ab7-a92f-79883de057d3 admin admin] PATH_INFO: `/v3/users/ad15c781be3a4a67ab2d832948ddf94a` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:56:18.235121 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d4bc01d-cd33-4ab7-a92f-79883de057d3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ad15c781be3a4a67ab2d832948ddf94a)` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:56:18.237142 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d4bc01d-cd33-4ab7-a92f-79883de057d3 admin admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:56:18.321665 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG dbcounter [-] [87058] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=87058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:56:18.437045 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.notifications [None req-9d4bc01d-cd33-4ab7-a92f-79883de057d3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ad15c781be3a4a67ab2d832948ddf94a'} {{(pid=87058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:56:18.437793 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.notifications [None req-9d4bc01d-cd33-4ab7-a92f-79883de057d3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ad15c781be3a4a67ab2d832948ddf94a'} {{(pid=87058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:56:18.438455 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.notifications [None req-9d4bc01d-cd33-4ab7-a92f-79883de057d3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ad15c781be3a4a67ab2d832948ddf94a'} {{(pid=87058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:56:18.451060 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.notifications [None req-9d4bc01d-cd33-4ab7-a92f-79883de057d3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ad15c781be3a4a67ab2d832948ddf94a'} {{(pid=87058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:56:18.457875 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.notifications [None req-9d4bc01d-cd33-4ab7-a92f-79883de057d3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ad15c781be3a4a67ab2d832948ddf94a'} {{(pid=87058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:56:18.538103 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 1021/2028] 10.222.0.20 () {58 vars in 1224 bytes} [Mon Aug 1 16:56:18 2022] DELETE /identity/v3/users/ad15c781be3a4a67ab2d832948ddf94a => generated 0 bytes in 377 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 16:56:18.543014 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55cc303-6244-4138-9f1d-d8203311797f None None] REQUEST_METHOD: `POST` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:56:18.543374 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55cc303-6244-4138-9f1d-d8203311797f None None] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:56:18.543719 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55cc303-6244-4138-9f1d-d8203311797f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:56:18.567605 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.auth.core [None req-f55cc303-6244-4138-9f1d-d8203311797f None None] MFA Rules not processed for user `1007f8e379b74e169b98fb7c3c0ba5b4`. Rule list: `[]` (Enabled: `True`). {{(pid=87057) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:56:18.568849 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.fernet_utils [None req-f55cc303-6244-4138-9f1d-d8203311797f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87057) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:56:18.637941 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 1008/2029] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 1 16:56:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:56:18.650106 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d4bc01d-cd33-4ab7-a92f-79883de057d3 admin admin] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:56:18.652393 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.fernet_utils [None req-9d4bc01d-cd33-4ab7-a92f-79883de057d3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:56:18.717560 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d4bc01d-cd33-4ab7-a92f-79883de057d3 admin admin] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:56:18.721256 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e6a951-d3b1-4934-beb1-b6101b25250b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d9bd8d649154c979be3744bfd016f95', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b992d6a65df4bf9b27c13192d57c5f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:56:18.721881 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e6a951-d3b1-4934-beb1-b6101b25250b service neutron] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:56:18.722204 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e6a951-d3b1-4934-beb1-b6101b25250b service neutron] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:56:18.722481 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e6a951-d3b1-4934-beb1-b6101b25250b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:56:18.731537 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2e6a951-d3b1-4934-beb1-b6101b25250b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:56:18.733003 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2e6a951-d3b1-4934-beb1-b6101b25250b service neutron] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:56:18.742507 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 1022/2030] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 1 16:56:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 16:56:19.144035 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f55cc303-6244-4138-9f1d-d8203311797f None None] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:56:19.145482 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.fernet_utils [None req-f55cc303-6244-4138-9f1d-d8203311797f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87057) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:56:19.157583 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f55cc303-6244-4138-9f1d-d8203311797f None None] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:56:19.162306 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb6c0693-03bd-41b0-8449-e1b1ec4e446d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:56:19.162894 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6c0693-03bd-41b0-8449-e1b1ec4e446d admin admin] REQUEST_METHOD: `DELETE` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:56:19.163177 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6c0693-03bd-41b0-8449-e1b1ec4e446d admin admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:56:19.163453 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6c0693-03bd-41b0-8449-e1b1ec4e446d admin admin] PATH_INFO: `/v3/projects/ffa37dfeafcf49b085bf68046f0bb2ba` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:56:19.164625 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb6c0693-03bd-41b0-8449-e1b1ec4e446d admin admin] RBAC: Authorizing `identity:delete_project(project_id=ffa37dfeafcf49b085bf68046f0bb2ba)` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:56:19.166654 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb6c0693-03bd-41b0-8449-e1b1ec4e446d admin admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:56:19.188253 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG dbcounter [-] [87057] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=87057) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:56:19.264390 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.notifications [None req-bb6c0693-03bd-41b0-8449-e1b1ec4e446d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ffa37dfeafcf49b085bf68046f0bb2ba', 'request_id': 'req-bb6c0693-03bd-41b0-8449-e1b1ec4e446d'} {{(pid=87057) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:56:19.270519 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.notifications [None req-bb6c0693-03bd-41b0-8449-e1b1ec4e446d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ffa37dfeafcf49b085bf68046f0bb2ba', 'request_id': 'req-bb6c0693-03bd-41b0-8449-e1b1ec4e446d'} {{(pid=87057) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:56:19.276698 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.notifications [None req-bb6c0693-03bd-41b0-8449-e1b1ec4e446d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ffa37dfeafcf49b085bf68046f0bb2ba', 'request_id': 'req-bb6c0693-03bd-41b0-8449-e1b1ec4e446d'} {{(pid=87057) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:56:19.343309 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.notifications [None req-bb6c0693-03bd-41b0-8449-e1b1ec4e446d admin admin] The token cache is being invalidate because project ffa37dfeafcf49b085bf68046f0bb2ba was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87057) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 16:56:19.343594 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.notifications [None req-bb6c0693-03bd-41b0-8449-e1b1ec4e446d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87057) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:56:19.344218 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.notifications [None req-bb6c0693-03bd-41b0-8449-e1b1ec4e446d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87057) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:56:19.345763 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 1009/2031] 10.222.0.20 () {58 vars in 1233 bytes} [Mon Aug 1 16:56:19 2022] DELETE /identity/v3/projects/ffa37dfeafcf49b085bf68046f0bb2ba => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 16:56:19.350015 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e6a951-d3b1-4934-beb1-b6101b25250b service neutron] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:56:19.360750 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e6a951-d3b1-4934-beb1-b6101b25250b service neutron] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:56:19.419350 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36571b69-682e-4be8-8838-f2c90274c403 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:56:19.420175 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36571b69-682e-4be8-8838-f2c90274c403 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:56:19.420460 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36571b69-682e-4be8-8838-f2c90274c403 admin admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:56:19.420868 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36571b69-682e-4be8-8838-f2c90274c403 admin admin] PATH_INFO: `/v3/users/fa5629136d484b6da03f2165f7db98fd` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:56:19.422076 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36571b69-682e-4be8-8838-f2c90274c403 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fa5629136d484b6da03f2165f7db98fd)` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:56:19.423991 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36571b69-682e-4be8-8838-f2c90274c403 admin admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:56:19.573354 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.notifications [None req-36571b69-682e-4be8-8838-f2c90274c403 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fa5629136d484b6da03f2165f7db98fd'} {{(pid=87058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:56:19.574014 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.notifications [None req-36571b69-682e-4be8-8838-f2c90274c403 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fa5629136d484b6da03f2165f7db98fd'} {{(pid=87058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:56:19.574737 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.notifications [None req-36571b69-682e-4be8-8838-f2c90274c403 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fa5629136d484b6da03f2165f7db98fd'} {{(pid=87058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:56:19.576029 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG dbcounter [-] [87058] Writing DB stats keystone:DELETE=13,keystone:SELECT=86,keystone:INSERT=1 {{(pid=87058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:56:19.587446 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.notifications [None req-36571b69-682e-4be8-8838-f2c90274c403 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fa5629136d484b6da03f2165f7db98fd'} {{(pid=87058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:56:19.595385 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.notifications [None req-36571b69-682e-4be8-8838-f2c90274c403 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fa5629136d484b6da03f2165f7db98fd'} {{(pid=87058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:56:19.691861 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 1023/2032] 10.222.0.20 () {58 vars in 1224 bytes} [Mon Aug 1 16:56:19 2022] DELETE /identity/v3/users/fa5629136d484b6da03f2165f7db98fd => generated 0 bytes in 342 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 16:56:20.139652 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb6c0693-03bd-41b0-8449-e1b1ec4e446d admin admin] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:56:20.141114 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.fernet_utils [None req-bb6c0693-03bd-41b0-8449-e1b1ec4e446d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87057) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:56:20.155017 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb6c0693-03bd-41b0-8449-e1b1ec4e446d admin admin] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:56:20.214787 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da3ec440-fe7d-4845-b40d-db803a904336 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:56:20.215356 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3ec440-fe7d-4845-b40d-db803a904336 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:56:20.215632 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3ec440-fe7d-4845-b40d-db803a904336 admin admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:56:20.215899 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3ec440-fe7d-4845-b40d-db803a904336 admin admin] PATH_INFO: `/v3/projects/aa20fc0789ac45c6bc3268fe9e815db3` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:56:20.217048 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da3ec440-fe7d-4845-b40d-db803a904336 admin admin] RBAC: Authorizing `identity:delete_project(project_id=aa20fc0789ac45c6bc3268fe9e815db3)` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:56:20.219004 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da3ec440-fe7d-4845-b40d-db803a904336 admin admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:56:20.317547 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.notifications [None req-da3ec440-fe7d-4845-b40d-db803a904336 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'aa20fc0789ac45c6bc3268fe9e815db3', 'request_id': 'req-da3ec440-fe7d-4845-b40d-db803a904336'} {{(pid=87057) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:56:20.324072 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.notifications [None req-da3ec440-fe7d-4845-b40d-db803a904336 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'aa20fc0789ac45c6bc3268fe9e815db3', 'request_id': 'req-da3ec440-fe7d-4845-b40d-db803a904336'} {{(pid=87057) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:56:20.329623 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.notifications [None req-da3ec440-fe7d-4845-b40d-db803a904336 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'aa20fc0789ac45c6bc3268fe9e815db3', 'request_id': 'req-da3ec440-fe7d-4845-b40d-db803a904336'} {{(pid=87057) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:56:20.394216 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.notifications [None req-da3ec440-fe7d-4845-b40d-db803a904336 admin admin] The token cache is being invalidate because project aa20fc0789ac45c6bc3268fe9e815db3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87057) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 16:56:20.394520 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.notifications [None req-da3ec440-fe7d-4845-b40d-db803a904336 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87057) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:56:20.395130 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.notifications [None req-da3ec440-fe7d-4845-b40d-db803a904336 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87057) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:56:20.399934 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 1010/2033] 10.222.0.20 () {58 vars in 1233 bytes} [Mon Aug 1 16:56:20 2022] DELETE /identity/v3/projects/aa20fc0789ac45c6bc3268fe9e815db3 => generated 0 bytes in 258 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 16:56:20.402483 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd480a4-0027-46b5-9f5c-1d653d54d4f3 None None] REQUEST_METHOD: `POST` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:56:20.403285 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd480a4-0027-46b5-9f5c-1d653d54d4f3 None None] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:56:20.403706 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd480a4-0027-46b5-9f5c-1d653d54d4f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:56:20.423993 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.auth.core [None req-7dd480a4-0027-46b5-9f5c-1d653d54d4f3 None None] MFA Rules not processed for user `1007f8e379b74e169b98fb7c3c0ba5b4`. Rule list: `[]` (Enabled: `True`). {{(pid=87058) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:56:20.425190 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.fernet_utils [None req-7dd480a4-0027-46b5-9f5c-1d653d54d4f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:56:20.500230 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 1024/2034] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 1 16:56:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:56:20.514151 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40fe01a-98f3-44bd-bc8f-f00191754ba4 None None] REQUEST_METHOD: `POST` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:56:20.514518 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40fe01a-98f3-44bd-bc8f-f00191754ba4 None None] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:56:20.514701 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40fe01a-98f3-44bd-bc8f-f00191754ba4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:56:20.537799 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.auth.core [None req-a40fe01a-98f3-44bd-bc8f-f00191754ba4 None None] MFA Rules not processed for user `1007f8e379b74e169b98fb7c3c0ba5b4`. Rule list: `[]` (Enabled: `True`). {{(pid=87057) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:56:20.539026 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.fernet_utils [None req-a40fe01a-98f3-44bd-bc8f-f00191754ba4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87057) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:56:20.550241 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 1011/2035] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 1 16:56:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:56:20.552760 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd480a4-0027-46b5-9f5c-1d653d54d4f3 None None] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:56:20.563610 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd480a4-0027-46b5-9f5c-1d653d54d4f3 None None] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:56:20.567846 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ec2389b-fd52-4741-a613-0525f1b291a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:56:20.568457 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec2389b-fd52-4741-a613-0525f1b291a1 admin admin] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:56:20.568739 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec2389b-fd52-4741-a613-0525f1b291a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:56:20.569015 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec2389b-fd52-4741-a613-0525f1b291a1 admin admin] PATH_INFO: `/v3/domains` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:56:20.569718 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ec2389b-fd52-4741-a613-0525f1b291a1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87058) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:56:20.570023 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ec2389b-fd52-4741-a613-0525f1b291a1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:56:20.571643 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ec2389b-fd52-4741-a613-0525f1b291a1 admin admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:56:20.588185 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 1025/2036] 10.222.0.20 () {58 vars in 1153 bytes} [Mon Aug 1 16:56:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:56:20.589367 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a40fe01a-98f3-44bd-bc8f-f00191754ba4 None None] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:56:20.598957 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a40fe01a-98f3-44bd-bc8f-f00191754ba4 None None] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:56:20.603757 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca18c46-74fe-4784-8105-16f3598d8d9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:56:20.604434 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca18c46-74fe-4784-8105-16f3598d8d9c admin admin] REQUEST_METHOD: `POST` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:56:20.604690 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca18c46-74fe-4784-8105-16f3598d8d9c admin admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:56:20.604970 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca18c46-74fe-4784-8105-16f3598d8d9c admin admin] PATH_INFO: `/v3/projects` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:56:20.605640 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eca18c46-74fe-4784-8105-16f3598d8d9c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:56:20.607627 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eca18c46-74fe-4784-8105-16f3598d8d9c admin admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:56:20.730330 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 1012/2037] 10.222.0.20 () {60 vars in 1153 bytes} [Mon Aug 1 16:56:20 2022] POST /identity/v3/projects => generated 383 bytes in 141 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 16:56:20.734518 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ec2389b-fd52-4741-a613-0525f1b291a1 admin admin] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:56:20.745688 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ec2389b-fd52-4741-a613-0525f1b291a1 admin admin] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:56:20.821470 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7641ab81-0b22-4591-ab16-6d58dd535c6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:56:20.822313 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7641ab81-0b22-4591-ab16-6d58dd535c6d admin admin] REQUEST_METHOD: `POST` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:56:20.822729 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7641ab81-0b22-4591-ab16-6d58dd535c6d admin admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:56:20.823733 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7641ab81-0b22-4591-ab16-6d58dd535c6d admin admin] PATH_INFO: `/v3/users` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:56:20.824719 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7641ab81-0b22-4591-ab16-6d58dd535c6d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:56:20.826813 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7641ab81-0b22-4591-ab16-6d58dd535c6d admin admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:56:20.945767 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 1026/2038] 10.222.0.20 () {60 vars in 1144 bytes} [Mon Aug 1 16:56:20 2022] POST /identity/v3/users => generated 292 bytes in 212 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 16:56:20.946564 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca18c46-74fe-4784-8105-16f3598d8d9c admin admin] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:56:20.958117 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca18c46-74fe-4784-8105-16f3598d8d9c admin admin] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:56:20.963136 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20bed757-27d0-432b-8d5d-ef095f79a19a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:56:20.964415 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20bed757-27d0-432b-8d5d-ef095f79a19a admin admin] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:56:20.964650 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20bed757-27d0-432b-8d5d-ef095f79a19a admin admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:56:20.965545 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20bed757-27d0-432b-8d5d-ef095f79a19a admin admin] PATH_INFO: `/v3/roles` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:56:20.965806 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20bed757-27d0-432b-8d5d-ef095f79a19a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:56:20.967506 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20bed757-27d0-432b-8d5d-ef095f79a19a admin admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:56:20.970105 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG dbcounter [-] [87057] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=87057) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:56:20.981487 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 1013/2039] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Aug 1 16:56:20 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 01 16:56:20.986563 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7641ab81-0b22-4591-ab16-6d58dd535c6d admin admin] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:56:21.011869 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7641ab81-0b22-4591-ab16-6d58dd535c6d admin admin] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:56:21.019389 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e2c056-32b8-40a2-868b-6fc69916a1af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:56:21.026826 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e2c056-32b8-40a2-868b-6fc69916a1af admin admin] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:56:21.027325 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e2c056-32b8-40a2-868b-6fc69916a1af admin admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:56:21.028439 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e2c056-32b8-40a2-868b-6fc69916a1af admin admin] PATH_INFO: `/v3/roles` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:56:21.029407 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63e2c056-32b8-40a2-868b-6fc69916a1af admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:56:21.031600 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63e2c056-32b8-40a2-868b-6fc69916a1af admin admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:56:21.048576 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 1027/2040] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Aug 1 16:56:20 2022] GET /identity/v3/roles => generated 1420 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 16:56:21.051646 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20bed757-27d0-432b-8d5d-ef095f79a19a admin admin] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:56:21.063243 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20bed757-27d0-432b-8d5d-ef095f79a19a admin admin] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:56:21.069340 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8454a58-72a4-4bd3-be53-e6232acb4292 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:56:21.070053 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8454a58-72a4-4bd3-be53-e6232acb4292 admin admin] REQUEST_METHOD: `PUT` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:56:21.070654 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8454a58-72a4-4bd3-be53-e6232acb4292 admin admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:56:21.070654 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8454a58-72a4-4bd3-be53-e6232acb4292 admin admin] PATH_INFO: `/v3/projects/8d7ac28c00c545e38bcc86f37e6ebc71/users/170a3f2012be42eab60c97a4d718ecc5/roles/0999d3da63a545f18ff3252afdc90518` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:56:21.090062 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8454a58-72a4-4bd3-be53-e6232acb4292 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8d7ac28c00c545e38bcc86f37e6ebc71, user_id=170a3f2012be42eab60c97a4d718ecc5, role_id=0999d3da63a545f18ff3252afdc90518)` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:56:21.093550 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8454a58-72a4-4bd3-be53-e6232acb4292 admin admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:56:21.175506 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 1014/2041] 10.222.0.20 () {60 vars in 1483 bytes} [Mon Aug 1 16:56:21 2022] PUT /identity/v3/projects/8d7ac28c00c545e38bcc86f37e6ebc71/users/170a3f2012be42eab60c97a4d718ecc5/roles/0999d3da63a545f18ff3252afdc90518 => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 16:56:21.187734 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88023335-448f-4241-8721-cffa0a9301bb None None] REQUEST_METHOD: `POST` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:56:21.188257 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88023335-448f-4241-8721-cffa0a9301bb None None] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:56:21.190206 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88023335-448f-4241-8721-cffa0a9301bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:56:21.235840 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.auth.core [None req-88023335-448f-4241-8721-cffa0a9301bb None None] MFA Rules not processed for user `170a3f2012be42eab60c97a4d718ecc5`. Rule list: `[]` (Enabled: `True`). {{(pid=87058) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:56:21.237104 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.fernet_utils [None req-88023335-448f-4241-8721-cffa0a9301bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:56:21.258331 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG dbcounter [-] [87058] Writing DB stats keystone:SELECT=93,keystone:DELETE=3,keystone:INSERT=4 {{(pid=87058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:56:21.316817 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 1028/2042] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 1 16:56:21 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:56:21.321282 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8454a58-72a4-4bd3-be53-e6232acb4292 admin admin] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:56:21.390524 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8454a58-72a4-4bd3-be53-e6232acb4292 admin admin] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:56:21.395509 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05361d13-68e3-461b-a398-ab3b63f0edf7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:56:21.396133 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05361d13-68e3-461b-a398-ab3b63f0edf7 admin admin] REQUEST_METHOD: `POST` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:56:21.396418 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05361d13-68e3-461b-a398-ab3b63f0edf7 admin admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:56:21.396697 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05361d13-68e3-461b-a398-ab3b63f0edf7 admin admin] PATH_INFO: `/v3/projects` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:56:21.397369 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05361d13-68e3-461b-a398-ab3b63f0edf7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:56:21.399382 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05361d13-68e3-461b-a398-ab3b63f0edf7 admin admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:56:21.540053 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 1015/2043] 10.222.0.20 () {60 vars in 1153 bytes} [Mon Aug 1 16:56:21 2022] POST /identity/v3/projects => generated 383 bytes in 219 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 16:56:21.543926 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88023335-448f-4241-8721-cffa0a9301bb None None] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:56:21.556419 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88023335-448f-4241-8721-cffa0a9301bb None None] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:56:21.617664 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52feced3-96fe-44e6-b765-e00a502d1ea6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:56:21.618523 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52feced3-96fe-44e6-b765-e00a502d1ea6 admin admin] REQUEST_METHOD: `POST` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:56:21.618802 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52feced3-96fe-44e6-b765-e00a502d1ea6 admin admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:56:21.619078 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52feced3-96fe-44e6-b765-e00a502d1ea6 admin admin] PATH_INFO: `/v3/users` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:56:21.619743 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52feced3-96fe-44e6-b765-e00a502d1ea6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:56:21.621729 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52feced3-96fe-44e6-b765-e00a502d1ea6 admin admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:56:21.709118 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 1029/2044] 10.222.0.20 () {60 vars in 1144 bytes} [Mon Aug 1 16:56:21 2022] POST /identity/v3/users => generated 298 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 16:56:21.713360 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05361d13-68e3-461b-a398-ab3b63f0edf7 admin admin] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:56:21.724629 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05361d13-68e3-461b-a398-ab3b63f0edf7 admin admin] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:56:21.729466 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d679e74-f8ae-4b67-a57d-908ff01890d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:56:21.730105 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d679e74-f8ae-4b67-a57d-908ff01890d8 admin admin] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:56:21.730390 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d679e74-f8ae-4b67-a57d-908ff01890d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:56:21.730668 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d679e74-f8ae-4b67-a57d-908ff01890d8 admin admin] PATH_INFO: `/v3/roles` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:56:21.731371 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d679e74-f8ae-4b67-a57d-908ff01890d8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:56:21.732984 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d679e74-f8ae-4b67-a57d-908ff01890d8 admin admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:56:21.743770 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 1016/2045] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Aug 1 16:56:21 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 16:56:21.748492 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52feced3-96fe-44e6-b765-e00a502d1ea6 admin admin] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:56:21.774473 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52feced3-96fe-44e6-b765-e00a502d1ea6 admin admin] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:56:21.780920 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbae7d33-077b-47bb-9ef3-28b9a0d00163 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:56:21.781716 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbae7d33-077b-47bb-9ef3-28b9a0d00163 admin admin] REQUEST_METHOD: `PUT` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:56:21.782202 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbae7d33-077b-47bb-9ef3-28b9a0d00163 admin admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:56:21.787353 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbae7d33-077b-47bb-9ef3-28b9a0d00163 admin admin] PATH_INFO: `/v3/projects/e89c29fe77eb491dbb53bfb954b3a57a/users/ab391e18776f469d8aa3a3beef9c7c96/roles/440bb943dd5b46fea038b903b6eaaf81` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:56:21.801767 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbae7d33-077b-47bb-9ef3-28b9a0d00163 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e89c29fe77eb491dbb53bfb954b3a57a, user_id=ab391e18776f469d8aa3a3beef9c7c96, role_id=440bb943dd5b46fea038b903b6eaaf81)` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:56:21.804418 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbae7d33-077b-47bb-9ef3-28b9a0d00163 admin admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:56:21.868381 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d679e74-f8ae-4b67-a57d-908ff01890d8 admin admin] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:56:21.873483 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.fernet_utils [None req-8d679e74-f8ae-4b67-a57d-908ff01890d8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87057) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:56:21.886241 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d679e74-f8ae-4b67-a57d-908ff01890d8 admin admin] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:56:21.892179 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-538543ec-7dc0-4054-bb6e-a4038df017f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:56:21.893068 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538543ec-7dc0-4054-bb6e-a4038df017f4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:56:21.893586 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538543ec-7dc0-4054-bb6e-a4038df017f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:56:21.895280 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538543ec-7dc0-4054-bb6e-a4038df017f4 admin admin] PATH_INFO: `/v3/users/cf1a2114c87e46f28e884428b4bd4fd3` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:56:21.895280 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-538543ec-7dc0-4054-bb6e-a4038df017f4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cf1a2114c87e46f28e884428b4bd4fd3)` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:56:21.897268 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-538543ec-7dc0-4054-bb6e-a4038df017f4 admin admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:56:21.907733 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 1030/2046] 10.222.0.20 () {60 vars in 1483 bytes} [Mon Aug 1 16:56:21 2022] PUT /identity/v3/projects/e89c29fe77eb491dbb53bfb954b3a57a/users/ab391e18776f469d8aa3a3beef9c7c96/roles/440bb943dd5b46fea038b903b6eaaf81 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 16:56:21.912231 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbae7d33-077b-47bb-9ef3-28b9a0d00163 admin admin] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:56:21.977730 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbae7d33-077b-47bb-9ef3-28b9a0d00163 admin admin] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:56:21.983579 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d19fd6d-2366-409b-a18a-34968579875f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:56:21.984204 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d19fd6d-2366-409b-a18a-34968579875f admin admin] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:56:21.984487 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d19fd6d-2366-409b-a18a-34968579875f admin admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:56:21.984760 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d19fd6d-2366-409b-a18a-34968579875f admin admin] PATH_INFO: `/v3/roles` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:56:21.985587 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d19fd6d-2366-409b-a18a-34968579875f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:56:21.987252 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d19fd6d-2366-409b-a18a-34968579875f admin admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:56:21.998382 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 1031/2047] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Aug 1 16:56:21 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 16:56:22.002848 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d19fd6d-2366-409b-a18a-34968579875f admin admin] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:56:22.018387 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d19fd6d-2366-409b-a18a-34968579875f admin admin] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:56:22.023357 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f736886a-a511-434a-b70b-3c31f4dd1277 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:56:22.024312 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f736886a-a511-434a-b70b-3c31f4dd1277 admin admin] REQUEST_METHOD: `PUT` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:56:22.024586 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f736886a-a511-434a-b70b-3c31f4dd1277 admin admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:56:22.024866 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f736886a-a511-434a-b70b-3c31f4dd1277 admin admin] PATH_INFO: `/v3/projects/e89c29fe77eb491dbb53bfb954b3a57a/users/ab391e18776f469d8aa3a3beef9c7c96/roles/440bb943dd5b46fea038b903b6eaaf81` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:56:22.027358 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f736886a-a511-434a-b70b-3c31f4dd1277 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e89c29fe77eb491dbb53bfb954b3a57a, user_id=ab391e18776f469d8aa3a3beef9c7c96, role_id=440bb943dd5b46fea038b903b6eaaf81)` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:56:22.030400 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f736886a-a511-434a-b70b-3c31f4dd1277 admin admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:56:22.041169 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 1032/2048] 10.222.0.20 () {60 vars in 1483 bytes} [Mon Aug 1 16:56:22 2022] PUT /identity/v3/projects/e89c29fe77eb491dbb53bfb954b3a57a/users/ab391e18776f469d8aa3a3beef9c7c96/roles/440bb943dd5b46fea038b903b6eaaf81 => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 16:56:22.053521 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6476c6f2-eb4c-40f6-92fa-7da8d551e224 None None] REQUEST_METHOD: `POST` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:56:22.053813 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6476c6f2-eb4c-40f6-92fa-7da8d551e224 None None] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:56:22.054123 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6476c6f2-eb4c-40f6-92fa-7da8d551e224 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:56:22.076999 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG dbcounter [-] [87058] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=87058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:56:22.087325 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.auth.core [None req-6476c6f2-eb4c-40f6-92fa-7da8d551e224 None None] MFA Rules not processed for user `ab391e18776f469d8aa3a3beef9c7c96`. Rule list: `[]` (Enabled: `True`). {{(pid=87058) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:56:22.089359 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.fernet_utils [None req-6476c6f2-eb4c-40f6-92fa-7da8d551e224 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:56:22.136129 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.notifications [None req-538543ec-7dc0-4054-bb6e-a4038df017f4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cf1a2114c87e46f28e884428b4bd4fd3'} {{(pid=87057) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:56:22.136767 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.notifications [None req-538543ec-7dc0-4054-bb6e-a4038df017f4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cf1a2114c87e46f28e884428b4bd4fd3'} {{(pid=87057) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:56:22.170106 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 1033/2049] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 1 16:56:22 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 16:56:22.192311 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.notifications [None req-538543ec-7dc0-4054-bb6e-a4038df017f4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cf1a2114c87e46f28e884428b4bd4fd3'} {{(pid=87057) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:56:22.193315 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.notifications [None req-538543ec-7dc0-4054-bb6e-a4038df017f4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cf1a2114c87e46f28e884428b4bd4fd3'} {{(pid=87057) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:56:22.198995 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6476c6f2-eb4c-40f6-92fa-7da8d551e224 None None] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:56:22.201656 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.fernet_utils [None req-6476c6f2-eb4c-40f6-92fa-7da8d551e224 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:56:22.211721 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.notifications [None req-538543ec-7dc0-4054-bb6e-a4038df017f4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cf1a2114c87e46f28e884428b4bd4fd3'} {{(pid=87057) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:56:22.226803 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 1017/2050] 10.222.0.20 () {58 vars in 1224 bytes} [Mon Aug 1 16:56:21 2022] DELETE /identity/v3/users/cf1a2114c87e46f28e884428b4bd4fd3 => generated 0 bytes in 359 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 16:56:22.232701 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5659a0e-6b13-43fc-9372-38e362fc77b6 None None] REQUEST_METHOD: `POST` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:56:22.235881 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5659a0e-6b13-43fc-9372-38e362fc77b6 None None] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:56:22.235881 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5659a0e-6b13-43fc-9372-38e362fc77b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:56:22.260651 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.auth.core [None req-e5659a0e-6b13-43fc-9372-38e362fc77b6 None None] MFA Rules not processed for user `1007f8e379b74e169b98fb7c3c0ba5b4`. Rule list: `[]` (Enabled: `True`). {{(pid=87057) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:56:22.262638 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.fernet_utils [None req-e5659a0e-6b13-43fc-9372-38e362fc77b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87057) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:56:22.281013 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG dbcounter [-] [87057] Writing DB stats keystone:SELECT=87,keystone:INSERT=3,keystone:DELETE=10 {{(pid=87057) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:56:22.288564 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6476c6f2-eb4c-40f6-92fa-7da8d551e224 None None] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:56:22.369299 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 1018/2051] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 1 16:56:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:56:22.380727 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5659a0e-6b13-43fc-9372-38e362fc77b6 None None] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:56:22.382435 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.fernet_utils [None req-e5659a0e-6b13-43fc-9372-38e362fc77b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87057) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:56:22.416126 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e573fa57-ac3c-4a00-a310-b4fb9ad71515 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25970b52ca15448fa1abb1f974d64038', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b992d6a65df4bf9b27c13192d57c5f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:56:22.417603 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e573fa57-ac3c-4a00-a310-b4fb9ad71515 service cinder] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:56:22.418255 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e573fa57-ac3c-4a00-a310-b4fb9ad71515 service cinder] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:56:22.418539 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e573fa57-ac3c-4a00-a310-b4fb9ad71515 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:56:22.420601 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.fernet_utils [None req-e573fa57-ac3c-4a00-a310-b4fb9ad71515 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:56:22.466112 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5659a0e-6b13-43fc-9372-38e362fc77b6 None None] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:56:22.471367 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4baad869-a9da-420a-a655-a78198093697 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d9bd8d649154c979be3744bfd016f95', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b992d6a65df4bf9b27c13192d57c5f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:56:22.472143 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4baad869-a9da-420a-a655-a78198093697 service neutron] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:56:22.472482 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4baad869-a9da-420a-a655-a78198093697 service neutron] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:56:22.472779 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4baad869-a9da-420a-a655-a78198093697 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:56:22.490313 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4baad869-a9da-420a-a655-a78198093697 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:56:22.493332 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4baad869-a9da-420a-a655-a78198093697 service neutron] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:56:22.507675 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 1019/2052] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 1 16:56:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 16:56:22.509083 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e573fa57-ac3c-4a00-a310-b4fb9ad71515 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:56:22.511234 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e573fa57-ac3c-4a00-a310-b4fb9ad71515 service cinder] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:56:22.518063 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG dbcounter [-] [87058] Writing DB stats keystone:SELECT=100 {{(pid=87058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:56:22.529338 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 1034/2053] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 1 16:56:22 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 331 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 01 16:56:22.710987 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4baad869-a9da-420a-a655-a78198093697 service neutron] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:56:22.712613 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.fernet_utils [None req-4baad869-a9da-420a-a655-a78198093697 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87057) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:56:22.725616 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4baad869-a9da-420a-a655-a78198093697 service neutron] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:56:22.729256 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11977587-8a29-4ec2-ab2e-97e784b13f07 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25970b52ca15448fa1abb1f974d64038', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b992d6a65df4bf9b27c13192d57c5f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:56:22.729935 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11977587-8a29-4ec2-ab2e-97e784b13f07 service cinder] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:56:22.730247 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11977587-8a29-4ec2-ab2e-97e784b13f07 service cinder] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:56:22.730598 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11977587-8a29-4ec2-ab2e-97e784b13f07 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:56:22.732410 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.fernet_utils [None req-11977587-8a29-4ec2-ab2e-97e784b13f07 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87057) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:56:22.847103 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11977587-8a29-4ec2-ab2e-97e784b13f07 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:56:22.847103 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11977587-8a29-4ec2-ab2e-97e784b13f07 service cinder] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:56:22.864306 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 1020/2054] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 1 16:56:22 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 154 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 16:56:23.175156 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e573fa57-ac3c-4a00-a310-b4fb9ad71515 service cinder] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:56:23.179001 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.fernet_utils [None req-e573fa57-ac3c-4a00-a310-b4fb9ad71515 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:56:23.191004 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e573fa57-ac3c-4a00-a310-b4fb9ad71515 service cinder] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:56:23.195575 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d6469fe-16ae-4b2a-942c-69442a0cce3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:56:23.207408 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6469fe-16ae-4b2a-942c-69442a0cce3e admin admin] REQUEST_METHOD: `DELETE` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:56:23.207408 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6469fe-16ae-4b2a-942c-69442a0cce3e admin admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:56:23.207408 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6469fe-16ae-4b2a-942c-69442a0cce3e admin admin] PATH_INFO: `/v3/projects/061e8ec4ecb7446d843e2161a8566f82` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:56:23.207408 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d6469fe-16ae-4b2a-942c-69442a0cce3e admin admin] RBAC: Authorizing `identity:delete_project(project_id=061e8ec4ecb7446d843e2161a8566f82)` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:56:23.207408 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d6469fe-16ae-4b2a-942c-69442a0cce3e admin admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:56:23.324604 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.notifications [None req-0d6469fe-16ae-4b2a-942c-69442a0cce3e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '061e8ec4ecb7446d843e2161a8566f82', 'request_id': 'req-0d6469fe-16ae-4b2a-942c-69442a0cce3e'} {{(pid=87058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:56:23.332247 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.notifications [None req-0d6469fe-16ae-4b2a-942c-69442a0cce3e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '061e8ec4ecb7446d843e2161a8566f82', 'request_id': 'req-0d6469fe-16ae-4b2a-942c-69442a0cce3e'} {{(pid=87058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:56:23.343475 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.notifications [None req-0d6469fe-16ae-4b2a-942c-69442a0cce3e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '061e8ec4ecb7446d843e2161a8566f82', 'request_id': 'req-0d6469fe-16ae-4b2a-942c-69442a0cce3e'} {{(pid=87058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:56:23.394954 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.notifications [None req-0d6469fe-16ae-4b2a-942c-69442a0cce3e admin admin] The token cache is being invalidate because project 061e8ec4ecb7446d843e2161a8566f82 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87058) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 16:56:23.395570 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.notifications [None req-0d6469fe-16ae-4b2a-942c-69442a0cce3e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:56:23.402171 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.notifications [None req-0d6469fe-16ae-4b2a-942c-69442a0cce3e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:56:23.402171 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 1035/2055] 10.222.0.20 () {58 vars in 1233 bytes} [Mon Aug 1 16:56:23 2022] DELETE /identity/v3/projects/061e8ec4ecb7446d843e2161a8566f82 => generated 0 bytes in 223 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 16:56:23.402369 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11977587-8a29-4ec2-ab2e-97e784b13f07 service cinder] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:56:23.414924 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11977587-8a29-4ec2-ab2e-97e784b13f07 service cinder] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:56:23.484181 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b605e0ab-770a-4c39-a00c-6794ade759e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:56:23.486345 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b605e0ab-770a-4c39-a00c-6794ade759e5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:56:23.487980 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b605e0ab-770a-4c39-a00c-6794ade759e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:56:23.487980 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b605e0ab-770a-4c39-a00c-6794ade759e5 admin admin] PATH_INFO: `/v3/users/109046d3dcad4e189a6476fc0e98e5ed` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:56:23.488984 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b605e0ab-770a-4c39-a00c-6794ade759e5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=109046d3dcad4e189a6476fc0e98e5ed)` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:56:23.491123 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b605e0ab-770a-4c39-a00c-6794ade759e5 admin admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:56:23.495515 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG dbcounter [-] [87057] Writing DB stats keystone:SELECT=100 {{(pid=87057) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:56:23.543700 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d6469fe-16ae-4b2a-942c-69442a0cce3e admin admin] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:56:23.545157 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.fernet_utils [None req-0d6469fe-16ae-4b2a-942c-69442a0cce3e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:56:23.560477 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d6469fe-16ae-4b2a-942c-69442a0cce3e admin admin] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:56:23.585517 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fecb991-e616-49d5-abcc-608232f7dcfb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:56:23.585517 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fecb991-e616-49d5-abcc-608232f7dcfb admin admin] REQUEST_METHOD: `DELETE` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:56:23.585755 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fecb991-e616-49d5-abcc-608232f7dcfb admin admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:56:23.585903 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fecb991-e616-49d5-abcc-608232f7dcfb admin admin] PATH_INFO: `/v3/users/ee6f46b57c0d4350babc45b5aa3e78d4` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:56:23.587380 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fecb991-e616-49d5-abcc-608232f7dcfb admin admin] RBAC: Authorizing `identity:delete_user(user_id=ee6f46b57c0d4350babc45b5aa3e78d4)` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:56:23.590340 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fecb991-e616-49d5-abcc-608232f7dcfb admin admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:56:23.710693 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.notifications [None req-b605e0ab-770a-4c39-a00c-6794ade759e5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '109046d3dcad4e189a6476fc0e98e5ed'} {{(pid=87057) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:56:23.711364 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.notifications [None req-b605e0ab-770a-4c39-a00c-6794ade759e5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '109046d3dcad4e189a6476fc0e98e5ed'} {{(pid=87057) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:56:23.786965 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.notifications [None req-4fecb991-e616-49d5-abcc-608232f7dcfb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ee6f46b57c0d4350babc45b5aa3e78d4'} {{(pid=87058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:56:23.787820 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.notifications [None req-4fecb991-e616-49d5-abcc-608232f7dcfb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ee6f46b57c0d4350babc45b5aa3e78d4'} {{(pid=87058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:56:23.788528 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.notifications [None req-4fecb991-e616-49d5-abcc-608232f7dcfb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ee6f46b57c0d4350babc45b5aa3e78d4'} {{(pid=87058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:56:23.799307 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.notifications [None req-b605e0ab-770a-4c39-a00c-6794ade759e5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '109046d3dcad4e189a6476fc0e98e5ed'} {{(pid=87057) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:56:23.803365 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.notifications [None req-b605e0ab-770a-4c39-a00c-6794ade759e5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '109046d3dcad4e189a6476fc0e98e5ed'} {{(pid=87057) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:56:23.804490 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.notifications [None req-4fecb991-e616-49d5-abcc-608232f7dcfb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ee6f46b57c0d4350babc45b5aa3e78d4'} {{(pid=87058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:56:23.813613 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.notifications [None req-b605e0ab-770a-4c39-a00c-6794ade759e5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '109046d3dcad4e189a6476fc0e98e5ed'} {{(pid=87057) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:56:23.814287 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.notifications [None req-4fecb991-e616-49d5-abcc-608232f7dcfb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ee6f46b57c0d4350babc45b5aa3e78d4'} {{(pid=87058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:56:23.834676 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 1021/2056] 10.222.0.20 () {58 vars in 1224 bytes} [Mon Aug 1 16:56:23 2022] DELETE /identity/v3/users/109046d3dcad4e189a6476fc0e98e5ed => generated 0 bytes in 434 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 16:56:23.902893 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 1036/2057] 10.222.0.20 () {58 vars in 1224 bytes} [Mon Aug 1 16:56:23 2022] DELETE /identity/v3/users/ee6f46b57c0d4350babc45b5aa3e78d4 => generated 0 bytes in 360 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 16:56:23.907489 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121e7df5-a152-462d-bf25-0dd43700bd06 None None] REQUEST_METHOD: `POST` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:56:23.907772 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121e7df5-a152-462d-bf25-0dd43700bd06 None None] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:56:23.908075 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121e7df5-a152-462d-bf25-0dd43700bd06 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:56:23.934263 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.auth.core [None req-121e7df5-a152-462d-bf25-0dd43700bd06 None None] MFA Rules not processed for user `1007f8e379b74e169b98fb7c3c0ba5b4`. Rule list: `[]` (Enabled: `True`). {{(pid=87057) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:56:23.936803 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.fernet_utils [None req-121e7df5-a152-462d-bf25-0dd43700bd06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87057) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:56:24.001517 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 1022/2058] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 1 16:56:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 16:56:24.012889 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fecb991-e616-49d5-abcc-608232f7dcfb admin admin] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:56:24.014722 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.fernet_utils [None req-4fecb991-e616-49d5-abcc-608232f7dcfb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:56:24.093633 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fecb991-e616-49d5-abcc-608232f7dcfb admin admin] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:56:24.099244 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3df6aa9-af1c-4b45-a15d-8a5917067a80 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d9bd8d649154c979be3744bfd016f95', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b992d6a65df4bf9b27c13192d57c5f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:56:24.100334 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3df6aa9-af1c-4b45-a15d-8a5917067a80 service neutron] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:56:24.100732 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3df6aa9-af1c-4b45-a15d-8a5917067a80 service neutron] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:56:24.101112 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3df6aa9-af1c-4b45-a15d-8a5917067a80 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:56:24.112616 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3df6aa9-af1c-4b45-a15d-8a5917067a80 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:56:24.114343 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3df6aa9-af1c-4b45-a15d-8a5917067a80 service neutron] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:56:24.119541 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG dbcounter [-] [87058] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=87058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:56:24.132042 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 1037/2059] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 1 16:56:24 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 16:56:24.331247 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-121e7df5-a152-462d-bf25-0dd43700bd06 None None] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:56:24.332904 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.fernet_utils [None req-121e7df5-a152-462d-bf25-0dd43700bd06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87057) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:56:24.347913 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-121e7df5-a152-462d-bf25-0dd43700bd06 None None] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:56:24.353315 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00200725-e289-46e4-99aa-89edde85a41e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:56:24.354745 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00200725-e289-46e4-99aa-89edde85a41e admin admin] REQUEST_METHOD: `DELETE` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:56:24.355501 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00200725-e289-46e4-99aa-89edde85a41e admin admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:56:24.355862 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00200725-e289-46e4-99aa-89edde85a41e admin admin] PATH_INFO: `/v3/users/1e64c30e6b704a40a57115f030356f21` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:56:24.357526 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00200725-e289-46e4-99aa-89edde85a41e admin admin] RBAC: Authorizing `identity:delete_user(user_id=1e64c30e6b704a40a57115f030356f21)` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:56:24.360179 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00200725-e289-46e4-99aa-89edde85a41e admin admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:56:24.461546 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3df6aa9-af1c-4b45-a15d-8a5917067a80 service neutron] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:56:24.463140 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.fernet_utils [None req-e3df6aa9-af1c-4b45-a15d-8a5917067a80 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:56:24.477446 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3df6aa9-af1c-4b45-a15d-8a5917067a80 service neutron] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:56:24.507482 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-592f4d38-1d0a-4936-87ff-dc3b00f41914 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:56:24.508160 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-592f4d38-1d0a-4936-87ff-dc3b00f41914 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:56:24.508431 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-592f4d38-1d0a-4936-87ff-dc3b00f41914 admin admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:56:24.509226 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-592f4d38-1d0a-4936-87ff-dc3b00f41914 admin admin] PATH_INFO: `/v3/projects/7a466783f2934e2ba939506eadfc4326` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:56:24.509923 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-592f4d38-1d0a-4936-87ff-dc3b00f41914 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7a466783f2934e2ba939506eadfc4326)` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:56:24.512791 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-592f4d38-1d0a-4936-87ff-dc3b00f41914 admin admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:56:24.619509 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.notifications [None req-00200725-e289-46e4-99aa-89edde85a41e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1e64c30e6b704a40a57115f030356f21'} {{(pid=87057) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:56:24.621320 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.notifications [None req-00200725-e289-46e4-99aa-89edde85a41e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1e64c30e6b704a40a57115f030356f21'} {{(pid=87057) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:56:24.679223 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.notifications [None req-592f4d38-1d0a-4936-87ff-dc3b00f41914 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7a466783f2934e2ba939506eadfc4326', 'request_id': 'req-592f4d38-1d0a-4936-87ff-dc3b00f41914'} {{(pid=87058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:56:24.684748 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.notifications [None req-00200725-e289-46e4-99aa-89edde85a41e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1e64c30e6b704a40a57115f030356f21'} {{(pid=87057) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:56:24.690127 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.notifications [None req-00200725-e289-46e4-99aa-89edde85a41e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1e64c30e6b704a40a57115f030356f21'} {{(pid=87057) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:56:24.695536 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.notifications [None req-592f4d38-1d0a-4936-87ff-dc3b00f41914 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7a466783f2934e2ba939506eadfc4326', 'request_id': 'req-592f4d38-1d0a-4936-87ff-dc3b00f41914'} {{(pid=87058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:56:24.709317 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.notifications [None req-592f4d38-1d0a-4936-87ff-dc3b00f41914 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7a466783f2934e2ba939506eadfc4326', 'request_id': 'req-592f4d38-1d0a-4936-87ff-dc3b00f41914'} {{(pid=87058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:56:24.715918 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.notifications [None req-00200725-e289-46e4-99aa-89edde85a41e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1e64c30e6b704a40a57115f030356f21'} {{(pid=87057) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:56:24.737799 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG dbcounter [-] [87057] Writing DB stats keystone:SELECT=79,keystone:DELETE=19,keystone:INSERT=2 {{(pid=87057) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:56:24.760124 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 1023/2060] 10.222.0.20 () {58 vars in 1224 bytes} [Mon Aug 1 16:56:24 2022] DELETE /identity/v3/users/1e64c30e6b704a40a57115f030356f21 => generated 0 bytes in 427 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 16:56:24.760124 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00200725-e289-46e4-99aa-89edde85a41e admin admin] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:56:24.761439 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.fernet_utils [None req-00200725-e289-46e4-99aa-89edde85a41e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87057) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:56:24.799627 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.notifications [None req-592f4d38-1d0a-4936-87ff-dc3b00f41914 admin admin] The token cache is being invalidate because project 7a466783f2934e2ba939506eadfc4326 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87058) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 16:56:24.799934 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.notifications [None req-592f4d38-1d0a-4936-87ff-dc3b00f41914 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:56:24.801558 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.notifications [None req-592f4d38-1d0a-4936-87ff-dc3b00f41914 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:56:24.804021 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 1038/2061] 10.222.0.20 () {58 vars in 1233 bytes} [Mon Aug 1 16:56:24 2022] DELETE /identity/v3/projects/7a466783f2934e2ba939506eadfc4326 => generated 0 bytes in 343 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 16:56:24.805847 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d85883d-6fdb-448c-b918-8f5d704d3a92 None None] REQUEST_METHOD: `POST` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:56:24.806181 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d85883d-6fdb-448c-b918-8f5d704d3a92 None None] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:56:24.806642 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d85883d-6fdb-448c-b918-8f5d704d3a92 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:56:24.841267 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.auth.core [None req-9d85883d-6fdb-448c-b918-8f5d704d3a92 None None] MFA Rules not processed for user `1007f8e379b74e169b98fb7c3c0ba5b4`. Rule list: `[]` (Enabled: `True`). {{(pid=87058) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:56:24.843129 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.fernet_utils [None req-9d85883d-6fdb-448c-b918-8f5d704d3a92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:56:24.931232 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00200725-e289-46e4-99aa-89edde85a41e admin admin] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:56:24.970753 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-922d89d8-8e44-46e1-8f85-b8256c308d61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:56:24.973980 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922d89d8-8e44-46e1-8f85-b8256c308d61 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:56:24.974310 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922d89d8-8e44-46e1-8f85-b8256c308d61 admin admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:56:24.974591 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922d89d8-8e44-46e1-8f85-b8256c308d61 admin admin] PATH_INFO: `/v3/projects/3934f1fe56ee4c08b86dbecfd42639b1` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:56:24.978380 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-922d89d8-8e44-46e1-8f85-b8256c308d61 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3934f1fe56ee4c08b86dbecfd42639b1)` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:56:24.983129 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-922d89d8-8e44-46e1-8f85-b8256c308d61 admin admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:56:25.021116 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 1039/2062] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 1 16:56:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 217 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:56:25.022657 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e093e2-6613-4f13-a327-833c898d68cf None None] REQUEST_METHOD: `POST` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:56:25.032791 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e093e2-6613-4f13-a327-833c898d68cf None None] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:56:25.032791 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e093e2-6613-4f13-a327-833c898d68cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:56:25.084314 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.auth.core [None req-45e093e2-6613-4f13-a327-833c898d68cf None None] MFA Rules not processed for user `1007f8e379b74e169b98fb7c3c0ba5b4`. Rule list: `[]` (Enabled: `True`). {{(pid=87058) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:56:25.084314 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.fernet_utils [None req-45e093e2-6613-4f13-a327-833c898d68cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:56:25.084314 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 1040/2063] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 1 16:56:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:56:25.084314 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e093e2-6613-4f13-a327-833c898d68cf None None] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:56:25.084314 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.fernet_utils [None req-45e093e2-6613-4f13-a327-833c898d68cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:56:25.170963 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.notifications [None req-922d89d8-8e44-46e1-8f85-b8256c308d61 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3934f1fe56ee4c08b86dbecfd42639b1', 'request_id': 'req-922d89d8-8e44-46e1-8f85-b8256c308d61'} {{(pid=87057) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:56:25.178158 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.notifications [None req-922d89d8-8e44-46e1-8f85-b8256c308d61 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3934f1fe56ee4c08b86dbecfd42639b1', 'request_id': 'req-922d89d8-8e44-46e1-8f85-b8256c308d61'} {{(pid=87057) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:56:25.184890 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.notifications [None req-922d89d8-8e44-46e1-8f85-b8256c308d61 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3934f1fe56ee4c08b86dbecfd42639b1', 'request_id': 'req-922d89d8-8e44-46e1-8f85-b8256c308d61'} {{(pid=87057) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:56:25.185032 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e093e2-6613-4f13-a327-833c898d68cf None None] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:56:25.217166 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG dbcounter [-] [87058] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=87058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:56:25.237495 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb99b7bb-ef05-4a91-ab3f-fbcac5d32c73 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d9bd8d649154c979be3744bfd016f95', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b992d6a65df4bf9b27c13192d57c5f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:56:25.238154 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb99b7bb-ef05-4a91-ab3f-fbcac5d32c73 service neutron] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:56:25.238449 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb99b7bb-ef05-4a91-ab3f-fbcac5d32c73 service neutron] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:56:25.238731 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb99b7bb-ef05-4a91-ab3f-fbcac5d32c73 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:56:25.302312 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.notifications [None req-922d89d8-8e44-46e1-8f85-b8256c308d61 admin admin] The token cache is being invalidate because project 3934f1fe56ee4c08b86dbecfd42639b1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87057) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 16:56:25.302312 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.notifications [None req-922d89d8-8e44-46e1-8f85-b8256c308d61 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87057) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:56:25.302312 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.notifications [None req-922d89d8-8e44-46e1-8f85-b8256c308d61 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87057) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:56:25.302312 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 1024/2064] 10.222.0.20 () {58 vars in 1233 bytes} [Mon Aug 1 16:56:24 2022] DELETE /identity/v3/projects/3934f1fe56ee4c08b86dbecfd42639b1 => generated 0 bytes in 544 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 16:56:25.303258 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ae3a53-6048-4a65-8f53-65679b558f55 None None] REQUEST_METHOD: `POST` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:56:25.303546 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ae3a53-6048-4a65-8f53-65679b558f55 None None] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:56:25.303972 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ae3a53-6048-4a65-8f53-65679b558f55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:56:25.326079 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb99b7bb-ef05-4a91-ab3f-fbcac5d32c73 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:56:25.327655 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb99b7bb-ef05-4a91-ab3f-fbcac5d32c73 service neutron] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:56:25.327771 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.auth.core [None req-70ae3a53-6048-4a65-8f53-65679b558f55 None None] MFA Rules not processed for user `1007f8e379b74e169b98fb7c3c0ba5b4`. Rule list: `[]` (Enabled: `True`). {{(pid=87057) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:56:25.331281 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.fernet_utils [None req-70ae3a53-6048-4a65-8f53-65679b558f55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87057) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:56:25.339325 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 1041/2065] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 1 16:56:25 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 266 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 16:56:25.340309 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb99b7bb-ef05-4a91-ab3f-fbcac5d32c73 service neutron] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:56:25.341977 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.fernet_utils [None req-bb99b7bb-ef05-4a91-ab3f-fbcac5d32c73 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:56:25.342467 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 1025/2066] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 1 16:56:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:56:25.348915 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ae3a53-6048-4a65-8f53-65679b558f55 None None] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:56:25.358484 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb99b7bb-ef05-4a91-ab3f-fbcac5d32c73 service neutron] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:56:25.361106 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ae3a53-6048-4a65-8f53-65679b558f55 None None] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:56:25.363471 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a07f6f-21ce-45bd-a2a4-7ad7c35d4307 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:56:25.364135 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a07f6f-21ce-45bd-a2a4-7ad7c35d4307 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:56:25.364401 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a07f6f-21ce-45bd-a2a4-7ad7c35d4307 admin admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:56:25.364950 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a07f6f-21ce-45bd-a2a4-7ad7c35d4307 admin admin] PATH_INFO: `/v3/users/401067c5615749f8bcf0a851330a34f6` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:56:25.366348 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2a07f6f-21ce-45bd-a2a4-7ad7c35d4307 admin admin] RBAC: Authorizing `identity:delete_user(user_id=401067c5615749f8bcf0a851330a34f6)` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:56:25.368551 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2a07f6f-21ce-45bd-a2a4-7ad7c35d4307 admin admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:56:25.377900 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-118fef01-c862-49fd-8a30-3befabf40a94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:56:25.378662 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118fef01-c862-49fd-8a30-3befabf40a94 admin admin] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:56:25.379150 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118fef01-c862-49fd-8a30-3befabf40a94 admin admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:56:25.379906 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118fef01-c862-49fd-8a30-3befabf40a94 admin admin] PATH_INFO: `/v3/domains` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:56:25.380959 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-118fef01-c862-49fd-8a30-3befabf40a94 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87057) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:56:25.383072 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-118fef01-c862-49fd-8a30-3befabf40a94 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:56:25.383416 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-118fef01-c862-49fd-8a30-3befabf40a94 admin admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:56:25.407151 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 1026/2067] 10.222.0.20 () {58 vars in 1153 bytes} [Mon Aug 1 16:56:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 16:56:25.411468 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-118fef01-c862-49fd-8a30-3befabf40a94 admin admin] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:56:25.422071 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-118fef01-c862-49fd-8a30-3befabf40a94 admin admin] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:56:25.426946 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-714e8943-378d-4258-87e9-2023e638f83e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:56:25.427584 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714e8943-378d-4258-87e9-2023e638f83e admin admin] REQUEST_METHOD: `POST` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:56:25.427846 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714e8943-378d-4258-87e9-2023e638f83e admin admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:56:25.428121 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714e8943-378d-4258-87e9-2023e638f83e admin admin] PATH_INFO: `/v3/projects` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:56:25.428827 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-714e8943-378d-4258-87e9-2023e638f83e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:56:25.430756 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-714e8943-378d-4258-87e9-2023e638f83e admin admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:56:25.522225 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 1027/2068] 10.222.0.20 () {60 vars in 1153 bytes} [Mon Aug 1 16:56:25 2022] POST /identity/v3/projects => generated 375 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 16:56:25.526948 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321fcdff-7c9e-4a7f-91e8-c5dbec467d59 None None] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:56:25.527457 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321fcdff-7c9e-4a7f-91e8-c5dbec467d59 None None] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:56:25.527736 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321fcdff-7c9e-4a7f-91e8-c5dbec467d59 None None] PATH_INFO: `/` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:56:25.529034 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 1028/2069] 10.222.0.20 () {54 vars in 921 bytes} [Mon Aug 1 16:56:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 16:56:25.529896 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-321fcdff-7c9e-4a7f-91e8-c5dbec467d59 None None] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:56:25.577022 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.notifications [None req-f2a07f6f-21ce-45bd-a2a4-7ad7c35d4307 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '401067c5615749f8bcf0a851330a34f6'} {{(pid=87058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:56:25.578512 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.notifications [None req-f2a07f6f-21ce-45bd-a2a4-7ad7c35d4307 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '401067c5615749f8bcf0a851330a34f6'} {{(pid=87058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:56:25.579569 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.notifications [None req-f2a07f6f-21ce-45bd-a2a4-7ad7c35d4307 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '401067c5615749f8bcf0a851330a34f6'} {{(pid=87058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:56:25.594091 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.notifications [None req-f2a07f6f-21ce-45bd-a2a4-7ad7c35d4307 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '401067c5615749f8bcf0a851330a34f6'} {{(pid=87058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:56:25.602793 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-321fcdff-7c9e-4a7f-91e8-c5dbec467d59 None None] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:56:25.603701 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.notifications [None req-f2a07f6f-21ce-45bd-a2a4-7ad7c35d4307 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '401067c5615749f8bcf0a851330a34f6'} {{(pid=87058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:56:25.607880 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27590da9-4129-42c2-a96d-777d2ece954c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:56:25.608510 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27590da9-4129-42c2-a96d-777d2ece954c admin admin] REQUEST_METHOD: `POST` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:56:25.608870 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27590da9-4129-42c2-a96d-777d2ece954c admin admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:56:25.609050 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27590da9-4129-42c2-a96d-777d2ece954c admin admin] PATH_INFO: `/v3/users` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:56:25.609721 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27590da9-4129-42c2-a96d-777d2ece954c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:56:25.611823 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27590da9-4129-42c2-a96d-777d2ece954c admin admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:56:25.678339 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 1042/2070] 10.222.0.20 () {58 vars in 1224 bytes} [Mon Aug 1 16:56:25 2022] DELETE /identity/v3/users/401067c5615749f8bcf0a851330a34f6 => generated 0 bytes in 337 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 16:56:25.681989 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9f9cb2-0b69-4b08-a652-985f288cf4f0 None None] REQUEST_METHOD: `POST` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:56:25.681989 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9f9cb2-0b69-4b08-a652-985f288cf4f0 None None] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:56:25.681989 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9f9cb2-0b69-4b08-a652-985f288cf4f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:56:25.708261 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 1029/2071] 10.222.0.20 () {60 vars in 1144 bytes} [Mon Aug 1 16:56:25 2022] POST /identity/v3/users => generated 288 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 16:56:25.711161 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27590da9-4129-42c2-a96d-777d2ece954c admin admin] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:56:25.711378 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.auth.core [None req-fd9f9cb2-0b69-4b08-a652-985f288cf4f0 None None] MFA Rules not processed for user `8362293261614623bcdabc4120a97a9d`. Rule list: `[]` (Enabled: `True`). {{(pid=87058) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:56:25.713357 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.fernet_utils [None req-fd9f9cb2-0b69-4b08-a652-985f288cf4f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:56:25.714494 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.fernet_utils [None req-27590da9-4129-42c2-a96d-777d2ece954c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87057) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:56:25.743972 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27590da9-4129-42c2-a96d-777d2ece954c admin admin] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:56:25.799686 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 1043/2072] 10.222.0.20 () {58 vars in 1018 bytes} [Mon Aug 1 16:56:25 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:56:25.800547 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd9f9cb2-0b69-4b08-a652-985f288cf4f0 None None] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:56:25.802244 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.fernet_utils [None req-fd9f9cb2-0b69-4b08-a652-985f288cf4f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:56:25.822772 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99bc6330-b8e1-4287-af2b-4523737db670 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:56:25.825148 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99bc6330-b8e1-4287-af2b-4523737db670 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:56:25.825275 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99bc6330-b8e1-4287-af2b-4523737db670 admin admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:56:25.825545 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99bc6330-b8e1-4287-af2b-4523737db670 admin admin] PATH_INFO: `/v3/users/bcdd827171964509bc43c7c24bd34da6` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:56:25.827616 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99bc6330-b8e1-4287-af2b-4523737db670 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bcdd827171964509bc43c7c24bd34da6)` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:56:25.829906 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99bc6330-b8e1-4287-af2b-4523737db670 admin admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:56:25.877710 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd9f9cb2-0b69-4b08-a652-985f288cf4f0 None None] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:56:25.882331 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3471bc0-fc88-4e6c-9b3d-9dff6e0acdee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:56:25.882952 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3471bc0-fc88-4e6c-9b3d-9dff6e0acdee admin admin] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:56:25.883226 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3471bc0-fc88-4e6c-9b3d-9dff6e0acdee admin admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:56:25.883493 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3471bc0-fc88-4e6c-9b3d-9dff6e0acdee admin admin] PATH_INFO: `/v3/roles` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:56:25.884275 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3471bc0-fc88-4e6c-9b3d-9dff6e0acdee admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:56:25.886509 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3471bc0-fc88-4e6c-9b3d-9dff6e0acdee admin admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:56:25.896801 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 1044/2073] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Aug 1 16:56:25 2022] GET /identity/v3/roles => generated 1420 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 16:56:25.897644 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3471bc0-fc88-4e6c-9b3d-9dff6e0acdee admin admin] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:56:25.899355 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.fernet_utils [None req-e3471bc0-fc88-4e6c-9b3d-9dff6e0acdee admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:56:25.915379 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3471bc0-fc88-4e6c-9b3d-9dff6e0acdee admin admin] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:56:25.921161 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f06bcc1c-05c2-4928-848f-24d0355da13f service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8362293261614623bcdabc4120a97a9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b992d6a65df4bf9b27c13192d57c5f0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:56:25.921774 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06bcc1c-05c2-4928-848f-24d0355da13f service placement] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:56:25.922087 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06bcc1c-05c2-4928-848f-24d0355da13f service placement] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:56:25.922366 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06bcc1c-05c2-4928-848f-24d0355da13f service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:56:25.931248 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f06bcc1c-05c2-4928-848f-24d0355da13f service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:56:25.934870 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f06bcc1c-05c2-4928-848f-24d0355da13f service placement] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:56:25.944391 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 1045/2074] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 1 16:56:25 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 16:56:25.945212 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f06bcc1c-05c2-4928-848f-24d0355da13f service placement] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:56:25.957053 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f06bcc1c-05c2-4928-848f-24d0355da13f service placement] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:56:25.964932 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65141669-7133-45f7-9cad-3008ff9fc513 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:56:25.964932 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65141669-7133-45f7-9cad-3008ff9fc513 admin admin] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:56:25.964932 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65141669-7133-45f7-9cad-3008ff9fc513 admin admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:56:25.964932 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65141669-7133-45f7-9cad-3008ff9fc513 admin admin] PATH_INFO: `/v3/roles` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:56:25.965288 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65141669-7133-45f7-9cad-3008ff9fc513 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:56:25.966395 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65141669-7133-45f7-9cad-3008ff9fc513 admin admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:56:25.984040 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 1046/2075] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Aug 1 16:56:25 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 16:56:25.993229 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65141669-7133-45f7-9cad-3008ff9fc513 admin admin] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:56:25.999007 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65141669-7133-45f7-9cad-3008ff9fc513 admin admin] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:56:26.003661 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1cdaabd-3d53-45e3-9828-800e183aa2a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:56:26.004384 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1cdaabd-3d53-45e3-9828-800e183aa2a1 admin admin] REQUEST_METHOD: `PUT` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:56:26.004662 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1cdaabd-3d53-45e3-9828-800e183aa2a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:56:26.004936 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1cdaabd-3d53-45e3-9828-800e183aa2a1 admin admin] PATH_INFO: `/v3/projects/9efbcdf51b7346959297c5d0d67a9643/users/30ef28ff292a4c2598083f3c8ff0de9e/roles/0999d3da63a545f18ff3252afdc90518` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:56:26.024860 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1cdaabd-3d53-45e3-9828-800e183aa2a1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9efbcdf51b7346959297c5d0d67a9643, user_id=30ef28ff292a4c2598083f3c8ff0de9e, role_id=0999d3da63a545f18ff3252afdc90518)` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:56:26.031769 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1cdaabd-3d53-45e3-9828-800e183aa2a1 admin admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:56:26.096337 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.notifications [None req-99bc6330-b8e1-4287-af2b-4523737db670 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bcdd827171964509bc43c7c24bd34da6'} {{(pid=87057) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:56:26.097296 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.notifications [None req-99bc6330-b8e1-4287-af2b-4523737db670 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bcdd827171964509bc43c7c24bd34da6'} {{(pid=87057) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:56:26.115418 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 1047/2076] 10.222.0.20 () {60 vars in 1483 bytes} [Mon Aug 1 16:56:25 2022] PUT /identity/v3/projects/9efbcdf51b7346959297c5d0d67a9643/users/30ef28ff292a4c2598083f3c8ff0de9e/roles/0999d3da63a545f18ff3252afdc90518 => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 16:56:26.116396 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1cdaabd-3d53-45e3-9828-800e183aa2a1 admin admin] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:56:26.121336 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.fernet_utils [None req-c1cdaabd-3d53-45e3-9828-800e183aa2a1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:56:26.167296 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.notifications [None req-99bc6330-b8e1-4287-af2b-4523737db670 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bcdd827171964509bc43c7c24bd34da6'} {{(pid=87057) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:56:26.168028 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.notifications [None req-99bc6330-b8e1-4287-af2b-4523737db670 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bcdd827171964509bc43c7c24bd34da6'} {{(pid=87057) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:56:26.176249 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.notifications [None req-99bc6330-b8e1-4287-af2b-4523737db670 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bcdd827171964509bc43c7c24bd34da6'} {{(pid=87057) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:56:26.205638 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 1030/2077] 10.222.0.20 () {58 vars in 1224 bytes} [Mon Aug 1 16:56:25 2022] DELETE /identity/v3/users/bcdd827171964509bc43c7c24bd34da6 => generated 0 bytes in 493 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 16:56:26.205638 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb086870-ba8e-4d1b-999e-5d3698ff3009 None None] REQUEST_METHOD: `POST` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:56:26.205638 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb086870-ba8e-4d1b-999e-5d3698ff3009 None None] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:56:26.205638 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb086870-ba8e-4d1b-999e-5d3698ff3009 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:56:26.231467 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1cdaabd-3d53-45e3-9828-800e183aa2a1 admin admin] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:56:26.248402 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.auth.core [None req-bb086870-ba8e-4d1b-999e-5d3698ff3009 None None] MFA Rules not processed for user `30ef28ff292a4c2598083f3c8ff0de9e`. Rule list: `[]` (Enabled: `True`). {{(pid=87057) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:56:26.249705 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.fernet_utils [None req-bb086870-ba8e-4d1b-999e-5d3698ff3009 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87057) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:56:26.310794 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a3b77bc-0b16-41d3-a2f5-3764a5cee5b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:56:26.311832 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3b77bc-0b16-41d3-a2f5-3764a5cee5b0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:56:26.318340 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3b77bc-0b16-41d3-a2f5-3764a5cee5b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:56:26.318340 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3b77bc-0b16-41d3-a2f5-3764a5cee5b0 admin admin] PATH_INFO: `/v3/projects/6e818621422f4ce7835c03e96bd4afbf` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:56:26.318340 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a3b77bc-0b16-41d3-a2f5-3764a5cee5b0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6e818621422f4ce7835c03e96bd4afbf)` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:56:26.318340 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a3b77bc-0b16-41d3-a2f5-3764a5cee5b0 admin admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:56:26.346178 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 1031/2078] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 1 16:56:26 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:56:26.347043 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb086870-ba8e-4d1b-999e-5d3698ff3009 None None] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:56:26.348458 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.fernet_utils [None req-bb086870-ba8e-4d1b-999e-5d3698ff3009 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87057) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:56:26.363047 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb086870-ba8e-4d1b-999e-5d3698ff3009 None None] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:56:26.368069 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64934869-85b6-429e-9fc3-139f9ad9f763 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:56:26.368719 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64934869-85b6-429e-9fc3-139f9ad9f763 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:56:26.369003 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64934869-85b6-429e-9fc3-139f9ad9f763 admin admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:56:26.369280 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64934869-85b6-429e-9fc3-139f9ad9f763 admin admin] PATH_INFO: `/v3/users/170a3f2012be42eab60c97a4d718ecc5` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:56:26.371343 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64934869-85b6-429e-9fc3-139f9ad9f763 admin admin] RBAC: Authorizing `identity:delete_user(user_id=170a3f2012be42eab60c97a4d718ecc5)` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:56:26.373281 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64934869-85b6-429e-9fc3-139f9ad9f763 admin admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:56:26.447032 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.notifications [None req-9a3b77bc-0b16-41d3-a2f5-3764a5cee5b0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6e818621422f4ce7835c03e96bd4afbf', 'request_id': 'req-9a3b77bc-0b16-41d3-a2f5-3764a5cee5b0'} {{(pid=87058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:56:26.453451 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.notifications [None req-9a3b77bc-0b16-41d3-a2f5-3764a5cee5b0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6e818621422f4ce7835c03e96bd4afbf', 'request_id': 'req-9a3b77bc-0b16-41d3-a2f5-3764a5cee5b0'} {{(pid=87058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:56:26.458797 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.notifications [None req-9a3b77bc-0b16-41d3-a2f5-3764a5cee5b0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6e818621422f4ce7835c03e96bd4afbf', 'request_id': 'req-9a3b77bc-0b16-41d3-a2f5-3764a5cee5b0'} {{(pid=87058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:56:26.571314 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.notifications [None req-9a3b77bc-0b16-41d3-a2f5-3764a5cee5b0 admin admin] The token cache is being invalidate because project 6e818621422f4ce7835c03e96bd4afbf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87058) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 16:56:26.571605 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.notifications [None req-9a3b77bc-0b16-41d3-a2f5-3764a5cee5b0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:56:26.572257 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.notifications [None req-9a3b77bc-0b16-41d3-a2f5-3764a5cee5b0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:56:26.574704 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 1048/2079] 10.222.0.20 () {58 vars in 1233 bytes} [Mon Aug 1 16:56:26 2022] DELETE /identity/v3/projects/6e818621422f4ce7835c03e96bd4afbf => generated 0 bytes in 459 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 16:56:26.578473 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a3b77bc-0b16-41d3-a2f5-3764a5cee5b0 admin admin] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:56:26.579797 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.fernet_utils [None req-9a3b77bc-0b16-41d3-a2f5-3764a5cee5b0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:56:26.580248 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG dbcounter [-] [87057] Writing DB stats keystone:DELETE=8,keystone:SELECT=90,keystone:INSERT=2 {{(pid=87057) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:56:26.593497 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.notifications [None req-64934869-85b6-429e-9fc3-139f9ad9f763 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '170a3f2012be42eab60c97a4d718ecc5'} {{(pid=87057) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:56:26.594324 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.notifications [None req-64934869-85b6-429e-9fc3-139f9ad9f763 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '170a3f2012be42eab60c97a4d718ecc5'} {{(pid=87057) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:56:26.644720 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a3b77bc-0b16-41d3-a2f5-3764a5cee5b0 admin admin] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:56:26.649546 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac04acc2-1ec6-4838-bbe1-616e3dfa1106 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:56:26.650344 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac04acc2-1ec6-4838-bbe1-616e3dfa1106 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:56:26.650628 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac04acc2-1ec6-4838-bbe1-616e3dfa1106 admin admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:56:26.650931 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac04acc2-1ec6-4838-bbe1-616e3dfa1106 admin admin] PATH_INFO: `/v3/projects/655b8720c7704ad4898b6c6079adbc35` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:56:26.652298 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac04acc2-1ec6-4838-bbe1-616e3dfa1106 admin admin] RBAC: Authorizing `identity:delete_project(project_id=655b8720c7704ad4898b6c6079adbc35)` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:56:26.654296 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac04acc2-1ec6-4838-bbe1-616e3dfa1106 admin admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:56:26.686320 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.notifications [None req-64934869-85b6-429e-9fc3-139f9ad9f763 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '170a3f2012be42eab60c97a4d718ecc5'} {{(pid=87057) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:56:26.686936 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.notifications [None req-64934869-85b6-429e-9fc3-139f9ad9f763 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '170a3f2012be42eab60c97a4d718ecc5'} {{(pid=87057) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:56:26.694520 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.notifications [None req-64934869-85b6-429e-9fc3-139f9ad9f763 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '170a3f2012be42eab60c97a4d718ecc5'} {{(pid=87057) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:56:26.713092 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 1032/2080] 10.222.0.20 () {58 vars in 1224 bytes} [Mon Aug 1 16:56:26 2022] DELETE /identity/v3/users/170a3f2012be42eab60c97a4d718ecc5 => generated 0 bytes in 366 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 16:56:26.714153 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64934869-85b6-429e-9fc3-139f9ad9f763 admin admin] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:56:26.715803 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.fernet_utils [None req-64934869-85b6-429e-9fc3-139f9ad9f763 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87057) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:56:26.764416 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.notifications [None req-ac04acc2-1ec6-4838-bbe1-616e3dfa1106 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '655b8720c7704ad4898b6c6079adbc35', 'request_id': 'req-ac04acc2-1ec6-4838-bbe1-616e3dfa1106'} {{(pid=87058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:56:26.771814 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.notifications [None req-ac04acc2-1ec6-4838-bbe1-616e3dfa1106 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '655b8720c7704ad4898b6c6079adbc35', 'request_id': 'req-ac04acc2-1ec6-4838-bbe1-616e3dfa1106'} {{(pid=87058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:56:26.775810 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64934869-85b6-429e-9fc3-139f9ad9f763 admin admin] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:56:26.778175 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.notifications [None req-ac04acc2-1ec6-4838-bbe1-616e3dfa1106 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '655b8720c7704ad4898b6c6079adbc35', 'request_id': 'req-ac04acc2-1ec6-4838-bbe1-616e3dfa1106'} {{(pid=87058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:56:26.828776 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a0a48d0-c6c2-492f-a731-8d31461a8503 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25970b52ca15448fa1abb1f974d64038', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b992d6a65df4bf9b27c13192d57c5f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:56:26.829528 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0a48d0-c6c2-492f-a731-8d31461a8503 service cinder] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:56:26.829807 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0a48d0-c6c2-492f-a731-8d31461a8503 service cinder] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:56:26.830110 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0a48d0-c6c2-492f-a731-8d31461a8503 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:56:26.831999 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.fernet_utils [None req-5a0a48d0-c6c2-492f-a731-8d31461a8503 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87057) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:56:26.879846 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.notifications [None req-ac04acc2-1ec6-4838-bbe1-616e3dfa1106 admin admin] The token cache is being invalidate because project 655b8720c7704ad4898b6c6079adbc35 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87058) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 16:56:26.880235 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.notifications [None req-ac04acc2-1ec6-4838-bbe1-616e3dfa1106 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:56:26.880985 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.notifications [None req-ac04acc2-1ec6-4838-bbe1-616e3dfa1106 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:56:26.882636 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 1049/2081] 10.222.0.20 () {58 vars in 1233 bytes} [Mon Aug 1 16:56:26 2022] DELETE /identity/v3/projects/655b8720c7704ad4898b6c6079adbc35 => generated 0 bytes in 307 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 16:56:26.883467 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac04acc2-1ec6-4838-bbe1-616e3dfa1106 admin admin] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:56:26.884762 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.fernet_utils [None req-ac04acc2-1ec6-4838-bbe1-616e3dfa1106 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:56:26.896488 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a0a48d0-c6c2-492f-a731-8d31461a8503 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:56:26.897995 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a0a48d0-c6c2-492f-a731-8d31461a8503 service cinder] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:56:26.908140 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 1033/2082] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 1 16:56:26 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 194 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 16:56:26.909027 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a0a48d0-c6c2-492f-a731-8d31461a8503 service cinder] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:56:26.919197 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.fernet_utils [None req-5a0a48d0-c6c2-492f-a731-8d31461a8503 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87057) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:56:26.957464 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac04acc2-1ec6-4838-bbe1-616e3dfa1106 admin admin] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:56:26.962411 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b20f486-0821-495c-8871-5d29821d6f7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:56:26.963059 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b20f486-0821-495c-8871-5d29821d6f7c admin admin] REQUEST_METHOD: `DELETE` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:56:26.965143 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b20f486-0821-495c-8871-5d29821d6f7c admin admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:56:26.965143 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b20f486-0821-495c-8871-5d29821d6f7c admin admin] PATH_INFO: `/v3/projects/7d94184fe46443a6a339d35908cc2b13` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:56:26.965668 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b20f486-0821-495c-8871-5d29821d6f7c admin admin] RBAC: Authorizing `identity:delete_project(project_id=7d94184fe46443a6a339d35908cc2b13)` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:56:26.967597 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b20f486-0821-495c-8871-5d29821d6f7c admin admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:56:26.991732 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a0a48d0-c6c2-492f-a731-8d31461a8503 service cinder] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:56:26.997308 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-916eeacb-a6dc-4e2d-b554-dafd15879d85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:56:26.998007 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916eeacb-a6dc-4e2d-b554-dafd15879d85 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:56:26.998273 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916eeacb-a6dc-4e2d-b554-dafd15879d85 admin admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:56:26.998553 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916eeacb-a6dc-4e2d-b554-dafd15879d85 admin admin] PATH_INFO: `/v3/users/4a08b8b5945b4b438bd47b8ce5ed73f3` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:56:26.999741 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-916eeacb-a6dc-4e2d-b554-dafd15879d85 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4a08b8b5945b4b438bd47b8ce5ed73f3)` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:56:27.001692 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-916eeacb-a6dc-4e2d-b554-dafd15879d85 admin admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:56:27.076354 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG dbcounter [-] [87058] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:56:27.077779 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.notifications [None req-9b20f486-0821-495c-8871-5d29821d6f7c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7d94184fe46443a6a339d35908cc2b13', 'request_id': 'req-9b20f486-0821-495c-8871-5d29821d6f7c'} {{(pid=87058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:56:27.084043 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.notifications [None req-9b20f486-0821-495c-8871-5d29821d6f7c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7d94184fe46443a6a339d35908cc2b13', 'request_id': 'req-9b20f486-0821-495c-8871-5d29821d6f7c'} {{(pid=87058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:56:27.090641 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.notifications [None req-9b20f486-0821-495c-8871-5d29821d6f7c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7d94184fe46443a6a339d35908cc2b13', 'request_id': 'req-9b20f486-0821-495c-8871-5d29821d6f7c'} {{(pid=87058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:56:27.146952 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.notifications [None req-916eeacb-a6dc-4e2d-b554-dafd15879d85 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4a08b8b5945b4b438bd47b8ce5ed73f3'} {{(pid=87057) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:56:27.147724 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.notifications [None req-916eeacb-a6dc-4e2d-b554-dafd15879d85 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4a08b8b5945b4b438bd47b8ce5ed73f3'} {{(pid=87057) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:56:27.172490 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.notifications [None req-9b20f486-0821-495c-8871-5d29821d6f7c admin admin] The token cache is being invalidate because project 7d94184fe46443a6a339d35908cc2b13 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87058) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 16:56:27.172783 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.notifications [None req-9b20f486-0821-495c-8871-5d29821d6f7c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:56:27.173427 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.notifications [None req-9b20f486-0821-495c-8871-5d29821d6f7c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:56:27.175013 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 1050/2083] 10.222.0.20 () {58 vars in 1233 bytes} [Mon Aug 1 16:56:26 2022] DELETE /identity/v3/projects/7d94184fe46443a6a339d35908cc2b13 => generated 0 bytes in 292 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 16:56:27.178067 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77900c1-074b-48a2-b3cd-973d78d6ecf0 None None] REQUEST_METHOD: `POST` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:56:27.178067 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77900c1-074b-48a2-b3cd-973d78d6ecf0 None None] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:56:27.178067 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77900c1-074b-48a2-b3cd-973d78d6ecf0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:56:27.196178 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.auth.core [None req-a77900c1-074b-48a2-b3cd-973d78d6ecf0 None None] MFA Rules not processed for user `1007f8e379b74e169b98fb7c3c0ba5b4`. Rule list: `[]` (Enabled: `True`). {{(pid=87058) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:56:27.197452 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.fernet_utils [None req-a77900c1-074b-48a2-b3cd-973d78d6ecf0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:56:27.234752 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.notifications [None req-916eeacb-a6dc-4e2d-b554-dafd15879d85 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4a08b8b5945b4b438bd47b8ce5ed73f3'} {{(pid=87057) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:56:27.235320 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.notifications [None req-916eeacb-a6dc-4e2d-b554-dafd15879d85 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4a08b8b5945b4b438bd47b8ce5ed73f3'} {{(pid=87057) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:56:27.243192 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.notifications [None req-916eeacb-a6dc-4e2d-b554-dafd15879d85 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4a08b8b5945b4b438bd47b8ce5ed73f3'} {{(pid=87057) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:56:27.256240 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 1051/2084] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 1 16:56:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:56:27.257503 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eefe92f-7318-4eeb-9ecd-6de2ec9c3350 None None] REQUEST_METHOD: `POST` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:56:27.257794 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eefe92f-7318-4eeb-9ecd-6de2ec9c3350 None None] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:56:27.258095 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eefe92f-7318-4eeb-9ecd-6de2ec9c3350 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:56:27.263399 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 1034/2085] 10.222.0.20 () {58 vars in 1224 bytes} [Mon Aug 1 16:56:26 2022] DELETE /identity/v3/users/4a08b8b5945b4b438bd47b8ce5ed73f3 => generated 0 bytes in 354 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 16:56:27.263399 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-916eeacb-a6dc-4e2d-b554-dafd15879d85 admin admin] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:56:27.274103 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.fernet_utils [None req-916eeacb-a6dc-4e2d-b554-dafd15879d85 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87057) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:56:27.283321 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.auth.core [None req-5eefe92f-7318-4eeb-9ecd-6de2ec9c3350 None None] MFA Rules not processed for user `1007f8e379b74e169b98fb7c3c0ba5b4`. Rule list: `[]` (Enabled: `True`). {{(pid=87058) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:56:27.284763 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.fernet_utils [None req-5eefe92f-7318-4eeb-9ecd-6de2ec9c3350 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:56:27.336418 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-916eeacb-a6dc-4e2d-b554-dafd15879d85 admin admin] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:56:27.343312 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f54e4ab3-a8be-486e-b40a-3a514ed59326 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:56:27.343979 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54e4ab3-a8be-486e-b40a-3a514ed59326 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:56:27.344271 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54e4ab3-a8be-486e-b40a-3a514ed59326 admin admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:56:27.344534 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54e4ab3-a8be-486e-b40a-3a514ed59326 admin admin] PATH_INFO: `/v3/users/20b77010606549cb8f5d23dad50f16d9` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:56:27.346567 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f54e4ab3-a8be-486e-b40a-3a514ed59326 admin admin] RBAC: Authorizing `identity:delete_user(user_id=20b77010606549cb8f5d23dad50f16d9)` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:56:27.349162 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f54e4ab3-a8be-486e-b40a-3a514ed59326 admin admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:56:27.387033 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 1052/2086] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 1 16:56:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:56:27.387868 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eefe92f-7318-4eeb-9ecd-6de2ec9c3350 None None] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:56:27.390128 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.fernet_utils [None req-5eefe92f-7318-4eeb-9ecd-6de2ec9c3350 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:56:27.453143 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eefe92f-7318-4eeb-9ecd-6de2ec9c3350 None None] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:56:27.457121 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d860bca-4037-4854-a1e3-3eff2c1da16b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d9bd8d649154c979be3744bfd016f95', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b992d6a65df4bf9b27c13192d57c5f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:56:27.475439 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d860bca-4037-4854-a1e3-3eff2c1da16b service neutron] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:56:27.475439 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d860bca-4037-4854-a1e3-3eff2c1da16b service neutron] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:56:27.475439 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d860bca-4037-4854-a1e3-3eff2c1da16b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:56:27.475439 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d860bca-4037-4854-a1e3-3eff2c1da16b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:56:27.475439 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d860bca-4037-4854-a1e3-3eff2c1da16b service neutron] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:56:27.480620 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 1053/2087] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 1 16:56:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 16:56:27.481859 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd066701-45a3-4f2d-9505-9e77e06d2ecd None None] REQUEST_METHOD: `POST` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:56:27.482174 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd066701-45a3-4f2d-9505-9e77e06d2ecd None None] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:56:27.482454 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd066701-45a3-4f2d-9505-9e77e06d2ecd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:56:27.523897 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.auth.core [None req-dd066701-45a3-4f2d-9505-9e77e06d2ecd None None] MFA Rules not processed for user `1007f8e379b74e169b98fb7c3c0ba5b4`. Rule list: `[]` (Enabled: `True`). {{(pid=87058) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:56:27.525380 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.fernet_utils [None req-dd066701-45a3-4f2d-9505-9e77e06d2ecd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:56:27.534883 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 1054/2088] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 1 16:56:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 54 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:56:27.539647 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd066701-45a3-4f2d-9505-9e77e06d2ecd None None] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:56:27.552822 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd066701-45a3-4f2d-9505-9e77e06d2ecd None None] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:56:27.560140 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d96ba1-3cb2-4cbf-be5f-a91cf61ff880 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:56:27.560844 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d96ba1-3cb2-4cbf-be5f-a91cf61ff880 admin admin] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:56:27.561260 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d96ba1-3cb2-4cbf-be5f-a91cf61ff880 admin admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:56:27.561590 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d96ba1-3cb2-4cbf-be5f-a91cf61ff880 admin admin] PATH_INFO: `/v3/domains` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:56:27.562336 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44d96ba1-3cb2-4cbf-be5f-a91cf61ff880 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87058) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:56:27.562699 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44d96ba1-3cb2-4cbf-be5f-a91cf61ff880 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:56:27.564413 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44d96ba1-3cb2-4cbf-be5f-a91cf61ff880 admin admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:56:27.591820 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.notifications [None req-f54e4ab3-a8be-486e-b40a-3a514ed59326 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '20b77010606549cb8f5d23dad50f16d9'} {{(pid=87057) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:56:27.596069 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 1055/2089] 10.222.0.20 () {58 vars in 1153 bytes} [Mon Aug 1 16:56:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 16:56:27.600318 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d96ba1-3cb2-4cbf-be5f-a91cf61ff880 admin admin] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:56:27.603183 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.notifications [None req-f54e4ab3-a8be-486e-b40a-3a514ed59326 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '20b77010606549cb8f5d23dad50f16d9'} {{(pid=87057) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:56:27.613394 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d96ba1-3cb2-4cbf-be5f-a91cf61ff880 admin admin] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:56:27.618750 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e3fdc9-784a-4df7-ba06-86e87eab9a9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:56:27.619385 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e3fdc9-784a-4df7-ba06-86e87eab9a9f admin admin] REQUEST_METHOD: `POST` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:56:27.620218 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e3fdc9-784a-4df7-ba06-86e87eab9a9f admin admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:56:27.620548 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e3fdc9-784a-4df7-ba06-86e87eab9a9f admin admin] PATH_INFO: `/v3/projects` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:56:27.621508 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86e3fdc9-784a-4df7-ba06-86e87eab9a9f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:56:27.623928 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86e3fdc9-784a-4df7-ba06-86e87eab9a9f admin admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:56:27.759773 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.notifications [None req-f54e4ab3-a8be-486e-b40a-3a514ed59326 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '20b77010606549cb8f5d23dad50f16d9'} {{(pid=87057) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:56:27.760495 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.notifications [None req-f54e4ab3-a8be-486e-b40a-3a514ed59326 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '20b77010606549cb8f5d23dad50f16d9'} {{(pid=87057) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:56:27.771198 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 1056/2090] 10.222.0.20 () {60 vars in 1153 bytes} [Mon Aug 1 16:56:27 2022] POST /identity/v3/projects => generated 375 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 16:56:27.772668 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e3fdc9-784a-4df7-ba06-86e87eab9a9f admin admin] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:56:27.773980 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.notifications [None req-f54e4ab3-a8be-486e-b40a-3a514ed59326 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '20b77010606549cb8f5d23dad50f16d9'} {{(pid=87057) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:56:27.774472 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.fernet_utils [None req-86e3fdc9-784a-4df7-ba06-86e87eab9a9f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:56:27.795354 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 1035/2091] 10.222.0.20 () {58 vars in 1224 bytes} [Mon Aug 1 16:56:27 2022] DELETE /identity/v3/users/20b77010606549cb8f5d23dad50f16d9 => generated 0 bytes in 532 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 16:56:27.883980 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e3fdc9-784a-4df7-ba06-86e87eab9a9f admin admin] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:56:27.961620 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6a3d5bd-ec87-4db1-ac13-4954d70262a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:56:27.962110 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a3d5bd-ec87-4db1-ac13-4954d70262a4 admin admin] REQUEST_METHOD: `POST` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:56:27.962421 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a3d5bd-ec87-4db1-ac13-4954d70262a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:56:27.962793 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a3d5bd-ec87-4db1-ac13-4954d70262a4 admin admin] PATH_INFO: `/v3/users` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:56:27.964153 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6a3d5bd-ec87-4db1-ac13-4954d70262a4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:56:27.966317 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6a3d5bd-ec87-4db1-ac13-4954d70262a4 admin admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:56:28.059199 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG dbcounter [-] [87057] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=87057) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:56:28.065366 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 1057/2092] 10.222.0.20 () {60 vars in 1144 bytes} [Mon Aug 1 16:56:27 2022] POST /identity/v3/users => generated 288 bytes in 293 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 16:56:28.069498 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f54e4ab3-a8be-486e-b40a-3a514ed59326 admin admin] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:56:28.080291 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f54e4ab3-a8be-486e-b40a-3a514ed59326 admin admin] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:56:28.084937 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b8158e-348a-4558-add1-6932f6da22bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:56:28.085543 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b8158e-348a-4558-add1-6932f6da22bb admin admin] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:56:28.085820 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b8158e-348a-4558-add1-6932f6da22bb admin admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:56:28.086146 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b8158e-348a-4558-add1-6932f6da22bb admin admin] PATH_INFO: `/v3/roles` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:56:28.090510 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0b8158e-348a-4558-add1-6932f6da22bb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:56:28.090510 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0b8158e-348a-4558-add1-6932f6da22bb admin admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:56:28.099289 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 1036/2093] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Aug 1 16:56:28 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 16:56:28.103142 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6a3d5bd-ec87-4db1-ac13-4954d70262a4 admin admin] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:56:28.114772 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6a3d5bd-ec87-4db1-ac13-4954d70262a4 admin admin] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:56:28.120794 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76273dbe-82bc-4839-b6ca-9105c6a01e0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:56:28.121412 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76273dbe-82bc-4839-b6ca-9105c6a01e0a admin admin] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:56:28.121811 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76273dbe-82bc-4839-b6ca-9105c6a01e0a admin admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:56:28.122126 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76273dbe-82bc-4839-b6ca-9105c6a01e0a admin admin] PATH_INFO: `/v3/roles` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:56:28.122828 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76273dbe-82bc-4839-b6ca-9105c6a01e0a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:56:28.124587 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76273dbe-82bc-4839-b6ca-9105c6a01e0a admin admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:56:28.138005 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 1058/2094] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Aug 1 16:56:28 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 16:56:28.141813 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b8158e-348a-4558-add1-6932f6da22bb admin admin] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:56:28.152466 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b8158e-348a-4558-add1-6932f6da22bb admin admin] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:56:28.157333 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e367908-d19e-4dc6-9fa8-37b0cd769d4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:56:28.157880 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e367908-d19e-4dc6-9fa8-37b0cd769d4b admin admin] REQUEST_METHOD: `PUT` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:56:28.158278 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e367908-d19e-4dc6-9fa8-37b0cd769d4b admin admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:56:28.158456 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e367908-d19e-4dc6-9fa8-37b0cd769d4b admin admin] PATH_INFO: `/v3/projects/f94c60f534f840c7a0459b24c85a1b93/users/0ea895229f3f407f80314a8723d90e9d/roles/0999d3da63a545f18ff3252afdc90518` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:56:28.174235 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e367908-d19e-4dc6-9fa8-37b0cd769d4b admin admin] RBAC: Authorizing `identity:create_grant(project_id=f94c60f534f840c7a0459b24c85a1b93, user_id=0ea895229f3f407f80314a8723d90e9d, role_id=0999d3da63a545f18ff3252afdc90518)` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:56:28.176901 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e367908-d19e-4dc6-9fa8-37b0cd769d4b admin admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:56:28.211817 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76273dbe-82bc-4839-b6ca-9105c6a01e0a admin admin] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:56:28.212128 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.fernet_utils [None req-76273dbe-82bc-4839-b6ca-9105c6a01e0a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:56:28.244103 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76273dbe-82bc-4839-b6ca-9105c6a01e0a admin admin] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:56:28.251351 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b811f68-23a9-4e2b-9a2d-e493d329e706 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:56:28.252004 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b811f68-23a9-4e2b-9a2d-e493d329e706 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:56:28.252288 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b811f68-23a9-4e2b-9a2d-e493d329e706 admin admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:56:28.252562 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b811f68-23a9-4e2b-9a2d-e493d329e706 admin admin] PATH_INFO: `/v3/projects/fa80280bf94d4df7a889c93ab3b2f84a` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:56:28.253744 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b811f68-23a9-4e2b-9a2d-e493d329e706 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fa80280bf94d4df7a889c93ab3b2f84a)` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:56:28.255684 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b811f68-23a9-4e2b-9a2d-e493d329e706 admin admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:56:28.265379 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 1037/2095] 10.222.0.20 () {60 vars in 1483 bytes} [Mon Aug 1 16:56:28 2022] PUT /identity/v3/projects/f94c60f534f840c7a0459b24c85a1b93/users/0ea895229f3f407f80314a8723d90e9d/roles/0999d3da63a545f18ff3252afdc90518 => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 16:56:28.266288 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e367908-d19e-4dc6-9fa8-37b0cd769d4b admin admin] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:56:28.268137 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.fernet_utils [None req-9e367908-d19e-4dc6-9fa8-37b0cd769d4b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87057) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:56:28.340528 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e367908-d19e-4dc6-9fa8-37b0cd769d4b admin admin] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:56:28.345550 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec53af0-3628-4a07-ad59-3c8c66ec8f3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:56:28.346233 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec53af0-3628-4a07-ad59-3c8c66ec8f3c admin admin] REQUEST_METHOD: `DELETE` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:56:28.346511 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec53af0-3628-4a07-ad59-3c8c66ec8f3c admin admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:56:28.346786 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec53af0-3628-4a07-ad59-3c8c66ec8f3c admin admin] PATH_INFO: `/v3/projects/8d7ac28c00c545e38bcc86f37e6ebc71` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:56:28.348164 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ec53af0-3628-4a07-ad59-3c8c66ec8f3c admin admin] RBAC: Authorizing `identity:delete_project(project_id=8d7ac28c00c545e38bcc86f37e6ebc71)` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:56:28.350168 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ec53af0-3628-4a07-ad59-3c8c66ec8f3c admin admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:56:28.364550 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.notifications [None req-0b811f68-23a9-4e2b-9a2d-e493d329e706 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fa80280bf94d4df7a889c93ab3b2f84a', 'request_id': 'req-0b811f68-23a9-4e2b-9a2d-e493d329e706'} {{(pid=87058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:56:28.372095 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.notifications [None req-0b811f68-23a9-4e2b-9a2d-e493d329e706 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fa80280bf94d4df7a889c93ab3b2f84a', 'request_id': 'req-0b811f68-23a9-4e2b-9a2d-e493d329e706'} {{(pid=87058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:56:28.377712 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.notifications [None req-0b811f68-23a9-4e2b-9a2d-e493d329e706 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fa80280bf94d4df7a889c93ab3b2f84a', 'request_id': 'req-0b811f68-23a9-4e2b-9a2d-e493d329e706'} {{(pid=87058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:56:28.445287 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.notifications [None req-0b811f68-23a9-4e2b-9a2d-e493d329e706 admin admin] The token cache is being invalidate because project fa80280bf94d4df7a889c93ab3b2f84a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87058) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 16:56:28.445598 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.notifications [None req-0b811f68-23a9-4e2b-9a2d-e493d329e706 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:56:28.446379 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.notifications [None req-0b811f68-23a9-4e2b-9a2d-e493d329e706 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:56:28.448414 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 1059/2096] 10.222.0.20 () {58 vars in 1233 bytes} [Mon Aug 1 16:56:28 2022] DELETE /identity/v3/projects/fa80280bf94d4df7a889c93ab3b2f84a => generated 0 bytes in 239 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 16:56:28.451773 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b811f68-23a9-4e2b-9a2d-e493d329e706 admin admin] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:56:28.452582 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG dbcounter [-] [87058] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:56:28.453435 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.fernet_utils [None req-0b811f68-23a9-4e2b-9a2d-e493d329e706 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:56:28.458961 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.notifications [None req-8ec53af0-3628-4a07-ad59-3c8c66ec8f3c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8d7ac28c00c545e38bcc86f37e6ebc71', 'request_id': 'req-8ec53af0-3628-4a07-ad59-3c8c66ec8f3c'} {{(pid=87057) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:56:28.466355 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.notifications [None req-8ec53af0-3628-4a07-ad59-3c8c66ec8f3c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8d7ac28c00c545e38bcc86f37e6ebc71', 'request_id': 'req-8ec53af0-3628-4a07-ad59-3c8c66ec8f3c'} {{(pid=87057) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:56:28.471982 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.notifications [None req-8ec53af0-3628-4a07-ad59-3c8c66ec8f3c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8d7ac28c00c545e38bcc86f37e6ebc71', 'request_id': 'req-8ec53af0-3628-4a07-ad59-3c8c66ec8f3c'} {{(pid=87057) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:56:28.518001 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b811f68-23a9-4e2b-9a2d-e493d329e706 admin admin] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:56:28.565722 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.notifications [None req-8ec53af0-3628-4a07-ad59-3c8c66ec8f3c admin admin] The token cache is being invalidate because project 8d7ac28c00c545e38bcc86f37e6ebc71 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87057) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 16:56:28.566134 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.notifications [None req-8ec53af0-3628-4a07-ad59-3c8c66ec8f3c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87057) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:56:28.566856 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.notifications [None req-8ec53af0-3628-4a07-ad59-3c8c66ec8f3c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87057) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:56:28.568994 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 1038/2097] 10.222.0.20 () {58 vars in 1233 bytes} [Mon Aug 1 16:56:28 2022] DELETE /identity/v3/projects/8d7ac28c00c545e38bcc86f37e6ebc71 => generated 0 bytes in 303 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 16:56:28.570390 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b4aea4-56ed-4a31-8f07-7862d688f447 None None] REQUEST_METHOD: `POST` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:56:28.570739 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b4aea4-56ed-4a31-8f07-7862d688f447 None None] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:56:28.570953 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b4aea4-56ed-4a31-8f07-7862d688f447 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:56:28.573730 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34aab992-0a62-4b28-98fc-da872e3e8561 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:56:28.574438 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34aab992-0a62-4b28-98fc-da872e3e8561 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:56:28.574719 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34aab992-0a62-4b28-98fc-da872e3e8561 admin admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:56:28.574995 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34aab992-0a62-4b28-98fc-da872e3e8561 admin admin] PATH_INFO: `/v3/projects/dc905974d67c4366b4a6a91c43706466` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:56:28.576149 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34aab992-0a62-4b28-98fc-da872e3e8561 admin admin] RBAC: Authorizing `identity:delete_project(project_id=dc905974d67c4366b4a6a91c43706466)` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:56:28.578211 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34aab992-0a62-4b28-98fc-da872e3e8561 admin admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:56:28.610219 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.auth.core [None req-85b4aea4-56ed-4a31-8f07-7862d688f447 None None] MFA Rules not processed for user `0ea895229f3f407f80314a8723d90e9d`. Rule list: `[]` (Enabled: `True`). {{(pid=87057) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:56:28.611453 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.fernet_utils [None req-85b4aea4-56ed-4a31-8f07-7862d688f447 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87057) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:56:28.683569 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 1039/2098] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 1 16:56:28 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:56:28.684853 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb001d14-97d1-4280-a280-41124c09fb94 None None] REQUEST_METHOD: `POST` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:56:28.685136 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb001d14-97d1-4280-a280-41124c09fb94 None None] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:56:28.685412 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb001d14-97d1-4280-a280-41124c09fb94 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:56:28.711500 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.auth.core [None req-fb001d14-97d1-4280-a280-41124c09fb94 None None] MFA Rules not processed for user `1007f8e379b74e169b98fb7c3c0ba5b4`. Rule list: `[]` (Enabled: `True`). {{(pid=87057) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:56:28.712723 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.fernet_utils [None req-fb001d14-97d1-4280-a280-41124c09fb94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87057) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:56:28.723574 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.notifications [None req-34aab992-0a62-4b28-98fc-da872e3e8561 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dc905974d67c4366b4a6a91c43706466', 'request_id': 'req-34aab992-0a62-4b28-98fc-da872e3e8561'} {{(pid=87058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:56:28.730784 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.notifications [None req-34aab992-0a62-4b28-98fc-da872e3e8561 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dc905974d67c4366b4a6a91c43706466', 'request_id': 'req-34aab992-0a62-4b28-98fc-da872e3e8561'} {{(pid=87058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:56:28.748989 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.notifications [None req-34aab992-0a62-4b28-98fc-da872e3e8561 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dc905974d67c4366b4a6a91c43706466', 'request_id': 'req-34aab992-0a62-4b28-98fc-da872e3e8561'} {{(pid=87058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:56:28.773851 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 1040/2099] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 1 16:56:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:56:28.775034 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb001d14-97d1-4280-a280-41124c09fb94 None None] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:56:28.776825 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.fernet_utils [None req-fb001d14-97d1-4280-a280-41124c09fb94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87057) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:56:28.788895 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb001d14-97d1-4280-a280-41124c09fb94 None None] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:56:28.793813 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-562aaa55-4177-4182-a916-4ebe85d39e70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:56:28.794493 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562aaa55-4177-4182-a916-4ebe85d39e70 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:56:28.794774 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562aaa55-4177-4182-a916-4ebe85d39e70 admin admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:56:28.795050 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562aaa55-4177-4182-a916-4ebe85d39e70 admin admin] PATH_INFO: `/v3/users/ab391e18776f469d8aa3a3beef9c7c96` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:56:28.796239 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-562aaa55-4177-4182-a916-4ebe85d39e70 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ab391e18776f469d8aa3a3beef9c7c96)` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:56:28.798207 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-562aaa55-4177-4182-a916-4ebe85d39e70 admin admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:56:28.807689 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.notifications [None req-34aab992-0a62-4b28-98fc-da872e3e8561 admin admin] The token cache is being invalidate because project dc905974d67c4366b4a6a91c43706466 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87058) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 16:56:28.808225 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.notifications [None req-34aab992-0a62-4b28-98fc-da872e3e8561 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:56:28.809105 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.notifications [None req-34aab992-0a62-4b28-98fc-da872e3e8561 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:56:28.811137 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 1060/2100] 10.222.0.20 () {58 vars in 1233 bytes} [Mon Aug 1 16:56:28 2022] DELETE /identity/v3/projects/dc905974d67c4366b4a6a91c43706466 => generated 0 bytes in 362 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 16:56:28.815414 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34aab992-0a62-4b28-98fc-da872e3e8561 admin admin] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:56:28.816836 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.fernet_utils [None req-34aab992-0a62-4b28-98fc-da872e3e8561 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:56:28.830051 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34aab992-0a62-4b28-98fc-da872e3e8561 admin admin] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:56:28.835143 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ac4481-8747-41d1-a9d3-d22671a1e533 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:56:28.835804 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ac4481-8747-41d1-a9d3-d22671a1e533 admin admin] REQUEST_METHOD: `POST` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:56:28.836088 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ac4481-8747-41d1-a9d3-d22671a1e533 admin admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:56:28.836359 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ac4481-8747-41d1-a9d3-d22671a1e533 admin admin] PATH_INFO: `/v3/projects` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:56:28.837032 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6ac4481-8747-41d1-a9d3-d22671a1e533 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:56:28.839080 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6ac4481-8747-41d1-a9d3-d22671a1e533 admin admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:56:28.930668 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 1061/2101] 10.222.0.20 () {60 vars in 1153 bytes} [Mon Aug 1 16:56:28 2022] POST /identity/v3/projects => generated 377 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 16:56:28.930668 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0bf0aa-ae15-4ead-84ef-b5c8ab39e34f None None] REQUEST_METHOD: `POST` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:56:28.930668 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0bf0aa-ae15-4ead-84ef-b5c8ab39e34f None None] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:56:28.930668 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0bf0aa-ae15-4ead-84ef-b5c8ab39e34f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:56:28.951762 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.notifications [None req-562aaa55-4177-4182-a916-4ebe85d39e70 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ab391e18776f469d8aa3a3beef9c7c96'} {{(pid=87057) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:56:28.952808 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.notifications [None req-562aaa55-4177-4182-a916-4ebe85d39e70 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ab391e18776f469d8aa3a3beef9c7c96'} {{(pid=87057) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:56:28.954270 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.auth.core [None req-2f0bf0aa-ae15-4ead-84ef-b5c8ab39e34f None None] MFA Rules not processed for user `1007f8e379b74e169b98fb7c3c0ba5b4`. Rule list: `[]` (Enabled: `True`). {{(pid=87058) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:56:28.955885 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.fernet_utils [None req-2f0bf0aa-ae15-4ead-84ef-b5c8ab39e34f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:56:29.014733 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG dbcounter [-] [87057] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87057) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:56:29.023999 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 1062/2102] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 1 16:56:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:56:29.025267 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b8a135-26a1-42d7-a9a0-a5cdae743cb2 None None] REQUEST_METHOD: `POST` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:56:29.025562 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b8a135-26a1-42d7-a9a0-a5cdae743cb2 None None] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:56:29.025873 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b8a135-26a1-42d7-a9a0-a5cdae743cb2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:56:29.045472 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.auth.core [None req-d7b8a135-26a1-42d7-a9a0-a5cdae743cb2 None None] MFA Rules not processed for user `1007f8e379b74e169b98fb7c3c0ba5b4`. Rule list: `[]` (Enabled: `True`). {{(pid=87058) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:56:29.048235 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.fernet_utils [None req-d7b8a135-26a1-42d7-a9a0-a5cdae743cb2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:56:29.050271 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG dbcounter [-] [87058] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=87058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:56:29.050737 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.notifications [None req-562aaa55-4177-4182-a916-4ebe85d39e70 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ab391e18776f469d8aa3a3beef9c7c96'} {{(pid=87057) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:56:29.051868 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.notifications [None req-562aaa55-4177-4182-a916-4ebe85d39e70 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ab391e18776f469d8aa3a3beef9c7c96'} {{(pid=87057) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:56:29.061778 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 1063/2103] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 1 16:56:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:56:29.062517 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b8a135-26a1-42d7-a9a0-a5cdae743cb2 None None] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:56:29.062946 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.notifications [None req-562aaa55-4177-4182-a916-4ebe85d39e70 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ab391e18776f469d8aa3a3beef9c7c96'} {{(pid=87057) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:56:29.063975 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.fernet_utils [None req-d7b8a135-26a1-42d7-a9a0-a5cdae743cb2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:56:29.085207 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b8a135-26a1-42d7-a9a0-a5cdae743cb2 None None] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:56:29.087801 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc31975e-419b-472a-869c-ca0effe2f5c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:56:29.088464 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc31975e-419b-472a-869c-ca0effe2f5c1 admin admin] REQUEST_METHOD: `POST` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:56:29.088744 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc31975e-419b-472a-869c-ca0effe2f5c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:56:29.089018 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc31975e-419b-472a-869c-ca0effe2f5c1 admin admin] PATH_INFO: `/v3/users` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:56:29.089961 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc31975e-419b-472a-869c-ca0effe2f5c1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:56:29.090437 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 1041/2104] 10.222.0.20 () {58 vars in 1224 bytes} [Mon Aug 1 16:56:28 2022] DELETE /identity/v3/users/ab391e18776f469d8aa3a3beef9c7c96 => generated 0 bytes in 316 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 16:56:29.091350 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-562aaa55-4177-4182-a916-4ebe85d39e70 admin admin] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:56:29.092801 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.fernet_utils [None req-562aaa55-4177-4182-a916-4ebe85d39e70 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87057) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:56:29.100557 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc31975e-419b-472a-869c-ca0effe2f5c1 admin admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:56:29.187506 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-562aaa55-4177-4182-a916-4ebe85d39e70 admin admin] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:56:29.193517 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b0b18d8-a544-416d-a5a4-a88e6733db95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:56:29.194342 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0b18d8-a544-416d-a5a4-a88e6733db95 admin admin] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:56:29.194749 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0b18d8-a544-416d-a5a4-a88e6733db95 admin admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:56:29.195105 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0b18d8-a544-416d-a5a4-a88e6733db95 admin admin] PATH_INFO: `/v3/domains` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:56:29.195831 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b0b18d8-a544-416d-a5a4-a88e6733db95 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87057) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:56:29.196143 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b0b18d8-a544-416d-a5a4-a88e6733db95 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:56:29.197842 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b0b18d8-a544-416d-a5a4-a88e6733db95 admin admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:56:29.214357 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 1042/2105] 10.222.0.20 () {58 vars in 1153 bytes} [Mon Aug 1 16:56:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 120 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:56:29.214357 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66dfc047-e93c-493a-819a-b1a6fbbd6fc4 None None] REQUEST_METHOD: `POST` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:56:29.214357 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66dfc047-e93c-493a-819a-b1a6fbbd6fc4 None None] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:56:29.214357 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66dfc047-e93c-493a-819a-b1a6fbbd6fc4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:56:29.223590 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 1064/2106] 10.222.0.20 () {60 vars in 1144 bytes} [Mon Aug 1 16:56:29 2022] POST /identity/v3/users => generated 289 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 16:56:29.225868 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc31975e-419b-472a-869c-ca0effe2f5c1 admin admin] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:56:29.239523 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.auth.core [None req-66dfc047-e93c-493a-819a-b1a6fbbd6fc4 None None] MFA Rules not processed for user `1007f8e379b74e169b98fb7c3c0ba5b4`. Rule list: `[]` (Enabled: `True`). {{(pid=87057) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:56:29.240845 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.fernet_utils [None req-66dfc047-e93c-493a-819a-b1a6fbbd6fc4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87057) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:56:29.250781 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 1043/2107] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 1 16:56:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:56:29.251660 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66dfc047-e93c-493a-819a-b1a6fbbd6fc4 None None] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:56:29.259975 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc31975e-419b-472a-869c-ca0effe2f5c1 admin admin] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:56:29.263926 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66dfc047-e93c-493a-819a-b1a6fbbd6fc4 None None] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:56:29.270169 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd76bafe-280b-48ba-8f2c-b6c39eeb0ff9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:56:29.270818 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd76bafe-280b-48ba-8f2c-b6c39eeb0ff9 admin admin] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:56:29.271262 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd76bafe-280b-48ba-8f2c-b6c39eeb0ff9 admin admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:56:29.271405 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd76bafe-280b-48ba-8f2c-b6c39eeb0ff9 admin admin] PATH_INFO: `/v3/roles` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:56:29.272132 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd76bafe-280b-48ba-8f2c-b6c39eeb0ff9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:56:29.273791 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd76bafe-280b-48ba-8f2c-b6c39eeb0ff9 admin admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:56:29.277300 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-171c7ee5-a355-46df-a62a-a539a1cb9062 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:56:29.277930 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171c7ee5-a355-46df-a62a-a539a1cb9062 admin admin] REQUEST_METHOD: `POST` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:56:29.278818 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171c7ee5-a355-46df-a62a-a539a1cb9062 admin admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:56:29.279466 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171c7ee5-a355-46df-a62a-a539a1cb9062 admin admin] PATH_INFO: `/v3/projects` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:56:29.280329 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-171c7ee5-a355-46df-a62a-a539a1cb9062 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:56:29.283153 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-171c7ee5-a355-46df-a62a-a539a1cb9062 admin admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:56:29.292546 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 1044/2108] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Aug 1 16:56:29 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 16:56:29.293405 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd76bafe-280b-48ba-8f2c-b6c39eeb0ff9 admin admin] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:56:29.305481 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd76bafe-280b-48ba-8f2c-b6c39eeb0ff9 admin admin] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:56:29.310863 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cb071b2-ea27-4f17-972e-efe27773afb8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:56:29.311496 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb071b2-ea27-4f17-972e-efe27773afb8 admin admin] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:56:29.311779 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb071b2-ea27-4f17-972e-efe27773afb8 admin admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:56:29.312055 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb071b2-ea27-4f17-972e-efe27773afb8 admin admin] PATH_INFO: `/v3/domains` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:56:29.312792 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cb071b2-ea27-4f17-972e-efe27773afb8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87057) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:56:29.313086 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cb071b2-ea27-4f17-972e-efe27773afb8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:56:29.314739 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cb071b2-ea27-4f17-972e-efe27773afb8 admin admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:56:29.332506 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 1045/2109] 10.222.0.20 () {58 vars in 1153 bytes} [Mon Aug 1 16:56:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:56:29.333402 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cb071b2-ea27-4f17-972e-efe27773afb8 admin admin] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:56:29.347081 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cb071b2-ea27-4f17-972e-efe27773afb8 admin admin] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:56:29.354664 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-997e9e82-3f38-4415-a8e8-85dc0235ff8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:56:29.355422 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997e9e82-3f38-4415-a8e8-85dc0235ff8f admin admin] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:56:29.355770 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997e9e82-3f38-4415-a8e8-85dc0235ff8f admin admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:56:29.356153 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997e9e82-3f38-4415-a8e8-85dc0235ff8f admin admin] PATH_INFO: `/v3/roles` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:56:29.356940 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-997e9e82-3f38-4415-a8e8-85dc0235ff8f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:56:29.359560 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-997e9e82-3f38-4415-a8e8-85dc0235ff8f admin admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:56:29.375211 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 1046/2110] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Aug 1 16:56:29 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 16:56:29.376476 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-997e9e82-3f38-4415-a8e8-85dc0235ff8f admin admin] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:56:29.388625 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-997e9e82-3f38-4415-a8e8-85dc0235ff8f admin admin] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:56:29.394256 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c989f70-8f79-4393-ad06-f87f4e41278d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:56:29.395041 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c989f70-8f79-4393-ad06-f87f4e41278d admin admin] REQUEST_METHOD: `POST` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:56:29.395325 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c989f70-8f79-4393-ad06-f87f4e41278d admin admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:56:29.395602 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c989f70-8f79-4393-ad06-f87f4e41278d admin admin] PATH_INFO: `/v3/projects` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:56:29.396300 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c989f70-8f79-4393-ad06-f87f4e41278d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:56:29.398358 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c989f70-8f79-4393-ad06-f87f4e41278d admin admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:56:29.418938 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 1065/2111] 10.222.0.20 () {60 vars in 1153 bytes} [Mon Aug 1 16:56:29 2022] POST /identity/v3/projects => generated 381 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 16:56:29.420320 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-171c7ee5-a355-46df-a62a-a539a1cb9062 admin admin] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:56:29.440823 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG dbcounter [-] [87057] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87057) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:56:29.509648 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 1047/2112] 10.222.0.20 () {60 vars in 1153 bytes} [Mon Aug 1 16:56:29 2022] POST /identity/v3/projects => generated 385 bytes in 134 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 16:56:29.510947 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c989f70-8f79-4393-ad06-f87f4e41278d admin admin] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:56:29.528351 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG dbcounter [-] [87058] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=87058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:56:29.540889 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-171c7ee5-a355-46df-a62a-a539a1cb9062 admin admin] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:56:29.597619 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c989f70-8f79-4393-ad06-f87f4e41278d admin admin] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:56:29.605854 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5057425d-6fec-4924-8a79-95e6e73b358e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:56:29.606958 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5057425d-6fec-4924-8a79-95e6e73b358e admin admin] REQUEST_METHOD: `POST` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:56:29.607442 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5057425d-6fec-4924-8a79-95e6e73b358e admin admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:56:29.608066 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5057425d-6fec-4924-8a79-95e6e73b358e admin admin] PATH_INFO: `/v3/users` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:56:29.608962 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5057425d-6fec-4924-8a79-95e6e73b358e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:56:29.611104 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5057425d-6fec-4924-8a79-95e6e73b358e admin admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:56:29.624665 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b34a395-41da-4f76-af36-90112aa061f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:56:29.626652 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b34a395-41da-4f76-af36-90112aa061f3 admin admin] REQUEST_METHOD: `PUT` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:56:29.629618 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b34a395-41da-4f76-af36-90112aa061f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:56:29.630215 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b34a395-41da-4f76-af36-90112aa061f3 admin admin] PATH_INFO: `/v3/projects/80ce9192be4849e1ae25548e57454fc9/users/59641a4bc54243ac8632560ef6e24bbe/roles/0999d3da63a545f18ff3252afdc90518` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:56:29.673722 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b34a395-41da-4f76-af36-90112aa061f3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=80ce9192be4849e1ae25548e57454fc9, user_id=59641a4bc54243ac8632560ef6e24bbe, role_id=0999d3da63a545f18ff3252afdc90518)` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:56:29.675125 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b34a395-41da-4f76-af36-90112aa061f3 admin admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:56:29.729204 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 1048/2113] 10.222.0.20 () {60 vars in 1144 bytes} [Mon Aug 1 16:56:29 2022] POST /identity/v3/users => generated 291 bytes in 218 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 16:56:29.729204 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5057425d-6fec-4924-8a79-95e6e73b358e admin admin] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:56:29.741500 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5057425d-6fec-4924-8a79-95e6e73b358e admin admin] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:56:29.749224 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf6a62bd-c0e2-4703-8518-06ac0ae03cef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:56:29.749854 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6a62bd-c0e2-4703-8518-06ac0ae03cef admin admin] REQUEST_METHOD: `POST` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:56:29.750190 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6a62bd-c0e2-4703-8518-06ac0ae03cef admin admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:56:29.750468 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6a62bd-c0e2-4703-8518-06ac0ae03cef admin admin] PATH_INFO: `/v3/users` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:56:29.751145 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf6a62bd-c0e2-4703-8518-06ac0ae03cef admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:56:29.753107 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf6a62bd-c0e2-4703-8518-06ac0ae03cef admin admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:56:29.762976 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 1066/2114] 10.222.0.20 () {60 vars in 1483 bytes} [Mon Aug 1 16:56:29 2022] PUT /identity/v3/projects/80ce9192be4849e1ae25548e57454fc9/users/59641a4bc54243ac8632560ef6e24bbe/roles/0999d3da63a545f18ff3252afdc90518 => generated 0 bytes in 339 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 16:56:29.762976 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b34a395-41da-4f76-af36-90112aa061f3 admin admin] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:56:29.763833 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.fernet_utils [None req-2b34a395-41da-4f76-af36-90112aa061f3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:56:29.848715 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b34a395-41da-4f76-af36-90112aa061f3 admin admin] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:56:29.853626 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a6513e-c6ff-4003-9ef6-7beba50507cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:56:29.854352 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a6513e-c6ff-4003-9ef6-7beba50507cb admin admin] REQUEST_METHOD: `DELETE` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:56:29.854635 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a6513e-c6ff-4003-9ef6-7beba50507cb admin admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:56:29.854908 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a6513e-c6ff-4003-9ef6-7beba50507cb admin admin] PATH_INFO: `/v3/projects/e89c29fe77eb491dbb53bfb954b3a57a` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:56:29.856127 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45a6513e-c6ff-4003-9ef6-7beba50507cb admin admin] RBAC: Authorizing `identity:delete_project(project_id=e89c29fe77eb491dbb53bfb954b3a57a)` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:56:29.858078 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45a6513e-c6ff-4003-9ef6-7beba50507cb admin admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:56:29.882954 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 1049/2115] 10.222.0.20 () {60 vars in 1144 bytes} [Mon Aug 1 16:56:29 2022] POST /identity/v3/users => generated 293 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 16:56:29.884330 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf6a62bd-c0e2-4703-8518-06ac0ae03cef admin admin] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:56:29.911101 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf6a62bd-c0e2-4703-8518-06ac0ae03cef admin admin] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:56:29.917860 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff20ba1-eb14-46bc-b467-0b13359aff9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:56:29.918775 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff20ba1-eb14-46bc-b467-0b13359aff9d admin admin] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:56:29.919058 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff20ba1-eb14-46bc-b467-0b13359aff9d admin admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:56:29.919336 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff20ba1-eb14-46bc-b467-0b13359aff9d admin admin] PATH_INFO: `/v3/roles` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:56:29.920153 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ff20ba1-eb14-46bc-b467-0b13359aff9d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:56:29.922685 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ff20ba1-eb14-46bc-b467-0b13359aff9d admin admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:56:29.939037 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 1050/2116] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Aug 1 16:56:29 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 16:56:29.940502 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fefa0ac5-214f-4037-acb3-ef2da201a42c None None] REQUEST_METHOD: `POST` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:56:29.940970 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fefa0ac5-214f-4037-acb3-ef2da201a42c None None] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:56:29.941404 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fefa0ac5-214f-4037-acb3-ef2da201a42c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:56:29.970456 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.auth.core [None req-fefa0ac5-214f-4037-acb3-ef2da201a42c None None] MFA Rules not processed for user `59641a4bc54243ac8632560ef6e24bbe`. Rule list: `[]` (Enabled: `True`). {{(pid=87057) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:56:29.971792 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.fernet_utils [None req-fefa0ac5-214f-4037-acb3-ef2da201a42c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87057) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:56:29.996615 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.notifications [None req-45a6513e-c6ff-4003-9ef6-7beba50507cb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e89c29fe77eb491dbb53bfb954b3a57a', 'request_id': 'req-45a6513e-c6ff-4003-9ef6-7beba50507cb'} {{(pid=87058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:56:30.004216 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.notifications [None req-45a6513e-c6ff-4003-9ef6-7beba50507cb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e89c29fe77eb491dbb53bfb954b3a57a', 'request_id': 'req-45a6513e-c6ff-4003-9ef6-7beba50507cb'} {{(pid=87058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:56:30.007839 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG dbcounter [-] [87057] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=87057) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:56:30.011521 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.notifications [None req-45a6513e-c6ff-4003-9ef6-7beba50507cb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e89c29fe77eb491dbb53bfb954b3a57a', 'request_id': 'req-45a6513e-c6ff-4003-9ef6-7beba50507cb'} {{(pid=87058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:56:30.044307 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 1051/2117] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 1 16:56:29 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:56:30.045165 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fefa0ac5-214f-4037-acb3-ef2da201a42c None None] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:56:30.055571 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fefa0ac5-214f-4037-acb3-ef2da201a42c None None] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:56:30.111277 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f54d7cd-f6b4-45b4-83ed-5d0c24f8d3e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:56:30.111825 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f54d7cd-f6b4-45b4-83ed-5d0c24f8d3e8 admin admin] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:56:30.112129 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f54d7cd-f6b4-45b4-83ed-5d0c24f8d3e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:56:30.112386 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f54d7cd-f6b4-45b4-83ed-5d0c24f8d3e8 admin admin] PATH_INFO: `/v3/roles` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:56:30.113087 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f54d7cd-f6b4-45b4-83ed-5d0c24f8d3e8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:56:30.114742 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f54d7cd-f6b4-45b4-83ed-5d0c24f8d3e8 admin admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:56:30.126072 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 1052/2118] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Aug 1 16:56:30 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 16:56:30.126072 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f54d7cd-f6b4-45b4-83ed-5d0c24f8d3e8 admin admin] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:56:30.136236 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f54d7cd-f6b4-45b4-83ed-5d0c24f8d3e8 admin admin] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:56:30.141014 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8695777-0a1b-4430-abfd-eae5fbc56e65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:56:30.141835 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8695777-0a1b-4430-abfd-eae5fbc56e65 admin admin] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:56:30.142133 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8695777-0a1b-4430-abfd-eae5fbc56e65 admin admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:56:30.142411 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8695777-0a1b-4430-abfd-eae5fbc56e65 admin admin] PATH_INFO: `/v3/roles` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:56:30.143119 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8695777-0a1b-4430-abfd-eae5fbc56e65 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:56:30.144728 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8695777-0a1b-4430-abfd-eae5fbc56e65 admin admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:56:30.150222 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG dbcounter [-] [87058] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:56:30.159080 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 1053/2119] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Aug 1 16:56:30 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 16:56:30.159080 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8695777-0a1b-4430-abfd-eae5fbc56e65 admin admin] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:56:30.159316 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.notifications [None req-45a6513e-c6ff-4003-9ef6-7beba50507cb admin admin] The token cache is being invalidate because project e89c29fe77eb491dbb53bfb954b3a57a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87058) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 16:56:30.159316 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.notifications [None req-45a6513e-c6ff-4003-9ef6-7beba50507cb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:56:30.159974 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.notifications [None req-45a6513e-c6ff-4003-9ef6-7beba50507cb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:56:30.161149 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.fernet_utils [None req-a8695777-0a1b-4430-abfd-eae5fbc56e65 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87057) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:56:30.161566 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 1067/2120] 10.222.0.20 () {58 vars in 1233 bytes} [Mon Aug 1 16:56:29 2022] DELETE /identity/v3/projects/e89c29fe77eb491dbb53bfb954b3a57a => generated 0 bytes in 402 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 16:56:30.167455 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a6513e-c6ff-4003-9ef6-7beba50507cb admin admin] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:56:30.167455 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.fernet_utils [None req-45a6513e-c6ff-4003-9ef6-7beba50507cb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:56:30.174844 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8695777-0a1b-4430-abfd-eae5fbc56e65 admin admin] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:56:30.179828 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fffc1f3-4f20-44ba-8bf0-a8379ab92cd3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:56:30.182126 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fffc1f3-4f20-44ba-8bf0-a8379ab92cd3 admin admin] REQUEST_METHOD: `POST` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:56:30.182126 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fffc1f3-4f20-44ba-8bf0-a8379ab92cd3 admin admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:56:30.182126 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fffc1f3-4f20-44ba-8bf0-a8379ab92cd3 admin admin] PATH_INFO: `/v3/projects` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:56:30.182126 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fffc1f3-4f20-44ba-8bf0-a8379ab92cd3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:56:30.183778 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fffc1f3-4f20-44ba-8bf0-a8379ab92cd3 admin admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:56:30.197150 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a6513e-c6ff-4003-9ef6-7beba50507cb admin admin] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:56:30.203325 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2477ab0-88e9-4423-abe2-05808d47d94a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:56:30.203965 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2477ab0-88e9-4423-abe2-05808d47d94a admin admin] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:56:30.204259 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2477ab0-88e9-4423-abe2-05808d47d94a admin admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:56:30.204578 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2477ab0-88e9-4423-abe2-05808d47d94a admin admin] PATH_INFO: `/v3/roles` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:56:30.205502 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2477ab0-88e9-4423-abe2-05808d47d94a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:56:30.207407 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2477ab0-88e9-4423-abe2-05808d47d94a admin admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:56:30.223457 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 1068/2121] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Aug 1 16:56:30 2022] GET /identity/v3/roles => generated 1420 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 16:56:30.224628 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2477ab0-88e9-4423-abe2-05808d47d94a admin admin] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:56:30.226378 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.fernet_utils [None req-e2477ab0-88e9-4423-abe2-05808d47d94a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:56:30.241375 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2477ab0-88e9-4423-abe2-05808d47d94a admin admin] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:56:30.247232 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b33fdc2-c7df-4421-ac39-5053ff67f054 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:56:30.247908 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b33fdc2-c7df-4421-ac39-5053ff67f054 admin admin] REQUEST_METHOD: `PUT` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:56:30.248188 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b33fdc2-c7df-4421-ac39-5053ff67f054 admin admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:56:30.248462 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b33fdc2-c7df-4421-ac39-5053ff67f054 admin admin] PATH_INFO: `/v3/projects/c6cb6a9491384a3f829fc713bb3bbcaa/users/bea8963139a74858bfe6c4b3e0cfbf99/roles/0999d3da63a545f18ff3252afdc90518` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:56:30.273723 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b33fdc2-c7df-4421-ac39-5053ff67f054 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c6cb6a9491384a3f829fc713bb3bbcaa, user_id=bea8963139a74858bfe6c4b3e0cfbf99, role_id=0999d3da63a545f18ff3252afdc90518)` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:56:30.277379 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b33fdc2-c7df-4421-ac39-5053ff67f054 admin admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:56:30.299407 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 1054/2122] 10.222.0.20 () {60 vars in 1153 bytes} [Mon Aug 1 16:56:30 2022] POST /identity/v3/projects => generated 373 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 16:56:30.300731 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e3e0e8-b159-4145-8390-a34d7dcfce80 None None] REQUEST_METHOD: `POST` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:56:30.301015 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e3e0e8-b159-4145-8390-a34d7dcfce80 None None] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:56:30.301326 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e3e0e8-b159-4145-8390-a34d7dcfce80 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:56:30.329102 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.auth.core [None req-88e3e0e8-b159-4145-8390-a34d7dcfce80 None None] MFA Rules not processed for user `1007f8e379b74e169b98fb7c3c0ba5b4`. Rule list: `[]` (Enabled: `True`). {{(pid=87057) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:56:30.330430 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.fernet_utils [None req-88e3e0e8-b159-4145-8390-a34d7dcfce80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87057) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:56:30.394111 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 1069/2123] 10.222.0.20 () {60 vars in 1483 bytes} [Mon Aug 1 16:56:30 2022] PUT /identity/v3/projects/c6cb6a9491384a3f829fc713bb3bbcaa/users/bea8963139a74858bfe6c4b3e0cfbf99/roles/0999d3da63a545f18ff3252afdc90518 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 16:56:30.394111 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b33fdc2-c7df-4421-ac39-5053ff67f054 admin admin] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:56:30.446308 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG dbcounter [-] [87058] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=87058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:56:30.474851 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 1055/2124] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 1 16:56:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:56:30.476710 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88e3e0e8-b159-4145-8390-a34d7dcfce80 None None] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:56:30.487740 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG dbcounter [-] [87057] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=87057) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:56:30.488287 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b33fdc2-c7df-4421-ac39-5053ff67f054 admin admin] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:56:30.493941 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88e3e0e8-b159-4145-8390-a34d7dcfce80 None None] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:56:30.494232 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da243e8-c0c9-4a7a-be63-4597dbe2197b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:56:30.495018 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da243e8-c0c9-4a7a-be63-4597dbe2197b admin admin] REQUEST_METHOD: `PUT` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:56:30.495478 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da243e8-c0c9-4a7a-be63-4597dbe2197b admin admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:56:30.495783 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da243e8-c0c9-4a7a-be63-4597dbe2197b admin admin] PATH_INFO: `/v3/projects/4b65f22ad0cd43f6a29ac52e5b93a2ae/users/cfd197d9c4574483985c82c5d0bcaacc/roles/0999d3da63a545f18ff3252afdc90518` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:56:30.498226 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dfba4ed-91c7-4fe2-9ea6-0de4acea6088 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:56:30.498854 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dfba4ed-91c7-4fe2-9ea6-0de4acea6088 admin admin] REQUEST_METHOD: `POST` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:56:30.499150 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dfba4ed-91c7-4fe2-9ea6-0de4acea6088 admin admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:56:30.499411 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dfba4ed-91c7-4fe2-9ea6-0de4acea6088 admin admin] PATH_INFO: `/v3/users` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:56:30.500352 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dfba4ed-91c7-4fe2-9ea6-0de4acea6088 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:56:30.502125 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dfba4ed-91c7-4fe2-9ea6-0de4acea6088 admin admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:56:30.512432 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2da243e8-c0c9-4a7a-be63-4597dbe2197b admin admin] RBAC: Authorizing `identity:create_grant(project_id=4b65f22ad0cd43f6a29ac52e5b93a2ae, user_id=cfd197d9c4574483985c82c5d0bcaacc, role_id=0999d3da63a545f18ff3252afdc90518)` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:56:30.516092 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2da243e8-c0c9-4a7a-be63-4597dbe2197b admin admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:56:30.612452 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 1056/2125] 10.222.0.20 () {60 vars in 1144 bytes} [Mon Aug 1 16:56:30 2022] POST /identity/v3/users => generated 293 bytes in 137 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 16:56:30.614209 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f61429d-b752-48a0-be82-d25a4cb62024 None None] REQUEST_METHOD: `POST` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:56:30.614653 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f61429d-b752-48a0-be82-d25a4cb62024 None None] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:56:30.615344 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f61429d-b752-48a0-be82-d25a4cb62024 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:56:30.645484 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 1070/2126] 10.222.0.20 () {60 vars in 1483 bytes} [Mon Aug 1 16:56:30 2022] PUT /identity/v3/projects/4b65f22ad0cd43f6a29ac52e5b93a2ae/users/cfd197d9c4574483985c82c5d0bcaacc/roles/0999d3da63a545f18ff3252afdc90518 => generated 0 bytes in 255 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 16:56:30.647918 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.auth.core [None req-8f61429d-b752-48a0-be82-d25a4cb62024 None None] MFA Rules not processed for user `bea8963139a74858bfe6c4b3e0cfbf99`. Rule list: `[]` (Enabled: `True`). {{(pid=87057) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:56:30.649364 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.fernet_utils [None req-8f61429d-b752-48a0-be82-d25a4cb62024 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87057) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:56:30.657198 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0963a3c-17cd-4cdd-8e28-23e99a325089 None None] REQUEST_METHOD: `POST` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:56:30.657483 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0963a3c-17cd-4cdd-8e28-23e99a325089 None None] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:56:30.657762 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0963a3c-17cd-4cdd-8e28-23e99a325089 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:56:30.762343 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 1057/2127] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 1 16:56:30 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:56:30.762343 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f61429d-b752-48a0-be82-d25a4cb62024 None None] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:56:30.763463 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.auth.core [None req-c0963a3c-17cd-4cdd-8e28-23e99a325089 None None] MFA Rules not processed for user `1007f8e379b74e169b98fb7c3c0ba5b4`. Rule list: `[]` (Enabled: `True`). {{(pid=87058) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:56:30.764758 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.fernet_utils [None req-c0963a3c-17cd-4cdd-8e28-23e99a325089 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:56:30.883823 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 1071/2128] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 1 16:56:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 239 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:56:30.886186 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4988174f-1dd8-4ad3-bed6-44762bf4beb2 None None] REQUEST_METHOD: `POST` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:56:30.886546 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4988174f-1dd8-4ad3-bed6-44762bf4beb2 None None] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:56:30.886830 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4988174f-1dd8-4ad3-bed6-44762bf4beb2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:56:30.889846 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f61429d-b752-48a0-be82-d25a4cb62024 None None] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:56:30.905461 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d1645f0-dc8b-4094-9b4d-6237a1835936 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:56:30.910263 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1645f0-dc8b-4094-9b4d-6237a1835936 admin admin] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:56:30.910263 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1645f0-dc8b-4094-9b4d-6237a1835936 admin admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:56:30.910263 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1645f0-dc8b-4094-9b4d-6237a1835936 admin admin] PATH_INFO: `/v3/roles` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:56:30.910263 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d1645f0-dc8b-4094-9b4d-6237a1835936 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:56:30.913595 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d1645f0-dc8b-4094-9b4d-6237a1835936 admin admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:56:30.930277 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.auth.core [None req-4988174f-1dd8-4ad3-bed6-44762bf4beb2 None None] MFA Rules not processed for user `cfd197d9c4574483985c82c5d0bcaacc`. Rule list: `[]` (Enabled: `True`). {{(pid=87058) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:56:30.930277 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.fernet_utils [None req-4988174f-1dd8-4ad3-bed6-44762bf4beb2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:56:30.936927 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 1058/2129] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Aug 1 16:56:30 2022] GET /identity/v3/roles => generated 1420 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 16:56:30.938172 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d1645f0-dc8b-4094-9b4d-6237a1835936 admin admin] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:56:30.940281 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.fernet_utils [None req-4d1645f0-dc8b-4094-9b4d-6237a1835936 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87057) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:56:31.032934 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d1645f0-dc8b-4094-9b4d-6237a1835936 admin admin] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:56:31.038363 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5716ca6-fb40-4080-ae43-a3daa6c709eb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25970b52ca15448fa1abb1f974d64038', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b992d6a65df4bf9b27c13192d57c5f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:56:31.039065 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5716ca6-fb40-4080-ae43-a3daa6c709eb service cinder] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:56:31.039399 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5716ca6-fb40-4080-ae43-a3daa6c709eb service cinder] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:56:31.039677 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5716ca6-fb40-4080-ae43-a3daa6c709eb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:56:31.052736 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5716ca6-fb40-4080-ae43-a3daa6c709eb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:56:31.053266 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 1072/2130] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 1 16:56:30 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:56:31.054243 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4988174f-1dd8-4ad3-bed6-44762bf4beb2 None None] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:56:31.055091 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5716ca6-fb40-4080-ae43-a3daa6c709eb service cinder] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:56:31.068046 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4988174f-1dd8-4ad3-bed6-44762bf4beb2 None None] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:56:31.072181 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 1059/2131] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 1 16:56:30 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 134 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 16:56:31.073499 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5716ca6-fb40-4080-ae43-a3daa6c709eb service cinder] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:56:31.075143 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG dbcounter [-] [87058] Writing DB stats keystone:SELECT=87,keystone:DELETE=7,keystone:INSERT=6 {{(pid=87058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:56:31.085689 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5716ca6-fb40-4080-ae43-a3daa6c709eb service cinder] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:56:31.091106 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff8acc2-ae44-40e7-b648-f06ffffdb513 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:56:31.092857 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff8acc2-ae44-40e7-b648-f06ffffdb513 admin admin] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:56:31.098166 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff8acc2-ae44-40e7-b648-f06ffffdb513 admin admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:56:31.098166 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff8acc2-ae44-40e7-b648-f06ffffdb513 admin admin] PATH_INFO: `/v3/domains` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:56:31.098449 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b600dd10-4d70-4cc9-ab21-1a0e5e4843ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:56:31.098559 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dff8acc2-ae44-40e7-b648-f06ffffdb513 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87058) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:56:31.098657 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b600dd10-4d70-4cc9-ab21-1a0e5e4843ff admin admin] REQUEST_METHOD: `PUT` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:56:31.098657 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b600dd10-4d70-4cc9-ab21-1a0e5e4843ff admin admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:56:31.098657 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b600dd10-4d70-4cc9-ab21-1a0e5e4843ff admin admin] PATH_INFO: `/v3/projects/74bafa2382ed4e988c211442bebec6ab/users/151b09136bba4bf6bdc0772a2df571e7/roles/440bb943dd5b46fea038b903b6eaaf81` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:56:31.098879 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dff8acc2-ae44-40e7-b648-f06ffffdb513 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:56:31.103636 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dff8acc2-ae44-40e7-b648-f06ffffdb513 admin admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:56:31.125664 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b600dd10-4d70-4cc9-ab21-1a0e5e4843ff admin admin] RBAC: Authorizing `identity:create_grant(project_id=74bafa2382ed4e988c211442bebec6ab, user_id=151b09136bba4bf6bdc0772a2df571e7, role_id=440bb943dd5b46fea038b903b6eaaf81)` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:56:31.129537 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 1073/2132] 10.222.0.20 () {58 vars in 1153 bytes} [Mon Aug 1 16:56:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 76 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:56:31.131325 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b600dd10-4d70-4cc9-ab21-1a0e5e4843ff admin admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:56:31.133807 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff8acc2-ae44-40e7-b648-f06ffffdb513 admin admin] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:56:31.150544 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff8acc2-ae44-40e7-b648-f06ffffdb513 admin admin] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:56:31.160208 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ef2ecc1-d68c-42c8-b0e2-b0d0023246d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:56:31.160901 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef2ecc1-d68c-42c8-b0e2-b0d0023246d1 admin admin] REQUEST_METHOD: `POST` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:56:31.161638 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef2ecc1-d68c-42c8-b0e2-b0d0023246d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:56:31.161638 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef2ecc1-d68c-42c8-b0e2-b0d0023246d1 admin admin] PATH_INFO: `/v3/projects` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:56:31.162323 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ef2ecc1-d68c-42c8-b0e2-b0d0023246d1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:56:31.164304 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ef2ecc1-d68c-42c8-b0e2-b0d0023246d1 admin admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:56:31.207272 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 1060/2133] 10.222.0.20 () {60 vars in 1483 bytes} [Mon Aug 1 16:56:31 2022] PUT /identity/v3/projects/74bafa2382ed4e988c211442bebec6ab/users/151b09136bba4bf6bdc0772a2df571e7/roles/440bb943dd5b46fea038b903b6eaaf81 => generated 0 bytes in 134 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 16:56:31.208592 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b600dd10-4d70-4cc9-ab21-1a0e5e4843ff admin admin] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:56:31.211138 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.fernet_utils [None req-b600dd10-4d70-4cc9-ab21-1a0e5e4843ff admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87057) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:56:31.286915 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 1074/2134] 10.222.0.20 () {60 vars in 1153 bytes} [Mon Aug 1 16:56:31 2022] POST /identity/v3/projects => generated 377 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 16:56:31.286915 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ef2ecc1-d68c-42c8-b0e2-b0d0023246d1 admin admin] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:56:31.304079 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG dbcounter [-] [87057] Writing DB stats keystone:SELECT=94,keystone:DELETE=4,keystone:INSERT=2 {{(pid=87057) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:56:31.318791 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b600dd10-4d70-4cc9-ab21-1a0e5e4843ff admin admin] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:56:31.397906 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2c6a5d0-89b4-44df-8253-8dbbd17ea566 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25970b52ca15448fa1abb1f974d64038', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b992d6a65df4bf9b27c13192d57c5f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:56:31.397906 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c6a5d0-89b4-44df-8253-8dbbd17ea566 service cinder] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:56:31.397906 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c6a5d0-89b4-44df-8253-8dbbd17ea566 service cinder] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:56:31.397906 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c6a5d0-89b4-44df-8253-8dbbd17ea566 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:56:31.401861 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ef2ecc1-d68c-42c8-b0e2-b0d0023246d1 admin admin] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:56:31.417121 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e85185ac-89c3-4100-b101-c8af40037c2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:56:31.417753 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85185ac-89c3-4100-b101-c8af40037c2e admin admin] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:56:31.418060 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85185ac-89c3-4100-b101-c8af40037c2e admin admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:56:31.418331 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85185ac-89c3-4100-b101-c8af40037c2e admin admin] PATH_INFO: `/v3/roles` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:56:31.419042 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e85185ac-89c3-4100-b101-c8af40037c2e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:56:31.420711 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e85185ac-89c3-4100-b101-c8af40037c2e admin admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:56:31.435092 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 1075/2135] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Aug 1 16:56:31 2022] GET /identity/v3/roles => generated 1420 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 16:56:31.436187 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e85185ac-89c3-4100-b101-c8af40037c2e admin admin] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:56:31.447064 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e85185ac-89c3-4100-b101-c8af40037c2e admin admin] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:56:31.453618 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f3812e8-cbb3-49f2-839a-b0b04c5efa71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:56:31.454466 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3812e8-cbb3-49f2-839a-b0b04c5efa71 admin admin] REQUEST_METHOD: `POST` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:56:31.454920 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3812e8-cbb3-49f2-839a-b0b04c5efa71 admin admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:56:31.455395 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3812e8-cbb3-49f2-839a-b0b04c5efa71 admin admin] PATH_INFO: `/v3/users` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:56:31.456267 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f3812e8-cbb3-49f2-839a-b0b04c5efa71 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:56:31.458375 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f3812e8-cbb3-49f2-839a-b0b04c5efa71 admin admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:56:31.493804 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2c6a5d0-89b4-44df-8253-8dbbd17ea566 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:56:31.496105 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2c6a5d0-89b4-44df-8253-8dbbd17ea566 service cinder] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:56:31.506742 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 1061/2136] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 1 16:56:31 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 298 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 16:56:31.512289 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2c6a5d0-89b4-44df-8253-8dbbd17ea566 service cinder] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:56:31.532349 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2c6a5d0-89b4-44df-8253-8dbbd17ea566 service cinder] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:56:31.539927 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee81c6ff-2f48-49c1-8cb1-c48d8287ebf2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:56:31.539927 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee81c6ff-2f48-49c1-8cb1-c48d8287ebf2 admin admin] REQUEST_METHOD: `PUT` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:56:31.539927 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee81c6ff-2f48-49c1-8cb1-c48d8287ebf2 admin admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:56:31.539927 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee81c6ff-2f48-49c1-8cb1-c48d8287ebf2 admin admin] PATH_INFO: `/v3/projects/74bafa2382ed4e988c211442bebec6ab/users/151b09136bba4bf6bdc0772a2df571e7/roles/440bb943dd5b46fea038b903b6eaaf81` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:56:31.543214 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee81c6ff-2f48-49c1-8cb1-c48d8287ebf2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=74bafa2382ed4e988c211442bebec6ab, user_id=151b09136bba4bf6bdc0772a2df571e7, role_id=440bb943dd5b46fea038b903b6eaaf81)` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:56:31.546645 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee81c6ff-2f48-49c1-8cb1-c48d8287ebf2 admin admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:56:31.569479 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 1062/2137] 10.222.0.20 () {60 vars in 1483 bytes} [Mon Aug 1 16:56:31 2022] PUT /identity/v3/projects/74bafa2382ed4e988c211442bebec6ab/users/151b09136bba4bf6bdc0772a2df571e7/roles/440bb943dd5b46fea038b903b6eaaf81 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 16:56:31.570011 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 1076/2138] 10.222.0.20 () {60 vars in 1144 bytes} [Mon Aug 1 16:56:31 2022] POST /identity/v3/users => generated 289 bytes in 134 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 16:56:31.570525 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee81c6ff-2f48-49c1-8cb1-c48d8287ebf2 admin admin] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:56:31.571500 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f3812e8-cbb3-49f2-839a-b0b04c5efa71 admin admin] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:56:31.608765 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f3812e8-cbb3-49f2-839a-b0b04c5efa71 admin admin] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:56:31.702683 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee81c6ff-2f48-49c1-8cb1-c48d8287ebf2 admin admin] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:56:31.708482 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88c91bab-c296-413c-b6b2-4cc1e555fce9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:56:31.709074 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c91bab-c296-413c-b6b2-4cc1e555fce9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:56:31.709358 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c91bab-c296-413c-b6b2-4cc1e555fce9 admin admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:56:31.709635 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c91bab-c296-413c-b6b2-4cc1e555fce9 admin admin] PATH_INFO: `/v3/users/cfd197d9c4574483985c82c5d0bcaacc` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:56:31.711041 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88c91bab-c296-413c-b6b2-4cc1e555fce9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cfd197d9c4574483985c82c5d0bcaacc)` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:56:31.713330 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88c91bab-c296-413c-b6b2-4cc1e555fce9 admin admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:56:31.717596 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d34744a-b263-4583-ba09-f1c8e1063daf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:56:31.719000 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d34744a-b263-4583-ba09-f1c8e1063daf admin admin] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:56:31.720491 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d34744a-b263-4583-ba09-f1c8e1063daf admin admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:56:31.720872 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d34744a-b263-4583-ba09-f1c8e1063daf admin admin] PATH_INFO: `/v3/roles` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:56:31.721909 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d34744a-b263-4583-ba09-f1c8e1063daf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:56:31.727946 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d34744a-b263-4583-ba09-f1c8e1063daf admin admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:56:31.751624 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 1077/2139] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Aug 1 16:56:31 2022] GET /identity/v3/roles => generated 1420 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 16:56:31.753146 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3201f30c-1269-4695-936c-f8d81b50e2bd None None] REQUEST_METHOD: `POST` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:56:31.753438 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3201f30c-1269-4695-936c-f8d81b50e2bd None None] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:56:31.753714 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3201f30c-1269-4695-936c-f8d81b50e2bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:56:31.785219 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.auth.core [None req-3201f30c-1269-4695-936c-f8d81b50e2bd None None] MFA Rules not processed for user `151b09136bba4bf6bdc0772a2df571e7`. Rule list: `[]` (Enabled: `True`). {{(pid=87058) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:56:31.786605 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.fernet_utils [None req-3201f30c-1269-4695-936c-f8d81b50e2bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:56:31.859450 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 1078/2140] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 1 16:56:31 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:56:31.860111 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3201f30c-1269-4695-936c-f8d81b50e2bd None None] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:56:31.870800 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3201f30c-1269-4695-936c-f8d81b50e2bd None None] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:56:31.875535 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e241c45-8898-4b5f-afe1-580ea00011ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:56:31.876183 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e241c45-8898-4b5f-afe1-580ea00011ca admin admin] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:56:31.876465 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e241c45-8898-4b5f-afe1-580ea00011ca admin admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:56:31.876739 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e241c45-8898-4b5f-afe1-580ea00011ca admin admin] PATH_INFO: `/v3/roles` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:56:31.877500 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e241c45-8898-4b5f-afe1-580ea00011ca admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:56:31.879180 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e241c45-8898-4b5f-afe1-580ea00011ca admin admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:56:31.891085 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 1079/2141] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Aug 1 16:56:31 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 16:56:31.895220 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e241c45-8898-4b5f-afe1-580ea00011ca admin admin] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:56:31.909243 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e241c45-8898-4b5f-afe1-580ea00011ca admin admin] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:56:31.914852 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-313490be-ccfa-4261-af83-99568c12ad02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:56:31.915699 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313490be-ccfa-4261-af83-99568c12ad02 admin admin] REQUEST_METHOD: `PUT` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:56:31.919040 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313490be-ccfa-4261-af83-99568c12ad02 admin admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:56:31.919040 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313490be-ccfa-4261-af83-99568c12ad02 admin admin] PATH_INFO: `/v3/projects/62ba61ebdbd14cea8d409bfb55b75e0a/users/7df0f7fce7854e4bb7a34c3040b0f9d0/roles/0999d3da63a545f18ff3252afdc90518` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:56:31.941997 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-313490be-ccfa-4261-af83-99568c12ad02 admin admin] RBAC: Authorizing `identity:create_grant(project_id=62ba61ebdbd14cea8d409bfb55b75e0a, user_id=7df0f7fce7854e4bb7a34c3040b0f9d0, role_id=0999d3da63a545f18ff3252afdc90518)` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:56:31.946174 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-313490be-ccfa-4261-af83-99568c12ad02 admin admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:56:31.985242 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.notifications [None req-88c91bab-c296-413c-b6b2-4cc1e555fce9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cfd197d9c4574483985c82c5d0bcaacc'} {{(pid=87057) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:56:31.986023 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.notifications [None req-88c91bab-c296-413c-b6b2-4cc1e555fce9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cfd197d9c4574483985c82c5d0bcaacc'} {{(pid=87057) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:56:32.077640 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.notifications [None req-88c91bab-c296-413c-b6b2-4cc1e555fce9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cfd197d9c4574483985c82c5d0bcaacc'} {{(pid=87057) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:56:32.083727 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.notifications [None req-88c91bab-c296-413c-b6b2-4cc1e555fce9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cfd197d9c4574483985c82c5d0bcaacc'} {{(pid=87057) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:56:32.095847 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 1080/2142] 10.222.0.20 () {60 vars in 1483 bytes} [Mon Aug 1 16:56:31 2022] PUT /identity/v3/projects/62ba61ebdbd14cea8d409bfb55b75e0a/users/7df0f7fce7854e4bb7a34c3040b0f9d0/roles/0999d3da63a545f18ff3252afdc90518 => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 16:56:32.098495 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-313490be-ccfa-4261-af83-99568c12ad02 admin admin] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:56:32.115674 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.fernet_utils [None req-313490be-ccfa-4261-af83-99568c12ad02 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:56:32.115775 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.notifications [None req-88c91bab-c296-413c-b6b2-4cc1e555fce9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cfd197d9c4574483985c82c5d0bcaacc'} {{(pid=87057) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:56:32.137568 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 1063/2143] 10.222.0.20 () {58 vars in 1224 bytes} [Mon Aug 1 16:56:31 2022] DELETE /identity/v3/users/cfd197d9c4574483985c82c5d0bcaacc => generated 0 bytes in 565 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 16:56:32.137568 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c2da1a-56b8-4c6d-8b3e-4a767afe6f5f None None] REQUEST_METHOD: `POST` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:56:32.137568 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c2da1a-56b8-4c6d-8b3e-4a767afe6f5f None None] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:56:32.137568 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c2da1a-56b8-4c6d-8b3e-4a767afe6f5f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:56:32.184090 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-313490be-ccfa-4261-af83-99568c12ad02 admin admin] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:56:32.186940 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.auth.core [None req-06c2da1a-56b8-4c6d-8b3e-4a767afe6f5f None None] MFA Rules not processed for user `7df0f7fce7854e4bb7a34c3040b0f9d0`. Rule list: `[]` (Enabled: `True`). {{(pid=87057) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:56:32.190254 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.fernet_utils [None req-06c2da1a-56b8-4c6d-8b3e-4a767afe6f5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87057) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:56:32.282797 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a548d63-4690-41e9-9880-7b40484e0d9a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25970b52ca15448fa1abb1f974d64038', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b992d6a65df4bf9b27c13192d57c5f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:56:32.283430 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a548d63-4690-41e9-9880-7b40484e0d9a service cinder] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:56:32.283874 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a548d63-4690-41e9-9880-7b40484e0d9a service cinder] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:56:32.284346 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a548d63-4690-41e9-9880-7b40484e0d9a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:56:32.286291 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.fernet_utils [None req-2a548d63-4690-41e9-9880-7b40484e0d9a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:56:32.306916 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 1064/2144] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 1 16:56:32 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:56:32.312646 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6d467e-6b32-440a-b964-6036a07fc6fa None None] REQUEST_METHOD: `POST` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:56:32.312939 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6d467e-6b32-440a-b964-6036a07fc6fa None None] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:56:32.313218 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6d467e-6b32-440a-b964-6036a07fc6fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:56:32.340428 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.auth.core [None req-8b6d467e-6b32-440a-b964-6036a07fc6fa None None] MFA Rules not processed for user `1007f8e379b74e169b98fb7c3c0ba5b4`. Rule list: `[]` (Enabled: `True`). {{(pid=87057) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:56:32.342136 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.fernet_utils [None req-8b6d467e-6b32-440a-b964-6036a07fc6fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87057) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:56:32.360364 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG dbcounter [-] [87058] Writing DB stats keystone:SELECT=95,keystone:INSERT=1,keystone:DELETE=4 {{(pid=87058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:56:32.397237 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a548d63-4690-41e9-9880-7b40484e0d9a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:56:32.399152 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a548d63-4690-41e9-9880-7b40484e0d9a service cinder] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:56:32.411539 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 1081/2145] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 1 16:56:32 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 16:56:32.413378 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 1065/2146] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 1 16:56:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:56:32.422415 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a548d63-4690-41e9-9880-7b40484e0d9a service cinder] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:56:32.424560 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b6d467e-6b32-440a-b964-6036a07fc6fa None None] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:56:32.425593 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.fernet_utils [None req-2a548d63-4690-41e9-9880-7b40484e0d9a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:56:32.426843 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.fernet_utils [None req-8b6d467e-6b32-440a-b964-6036a07fc6fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87057) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:56:32.441269 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a548d63-4690-41e9-9880-7b40484e0d9a service cinder] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:56:32.448449 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e29e27-7dae-4965-adef-5118e8556d0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:56:32.449051 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e29e27-7dae-4965-adef-5118e8556d0b admin admin] REQUEST_METHOD: `POST` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:56:32.449341 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e29e27-7dae-4965-adef-5118e8556d0b admin admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:56:32.453071 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e29e27-7dae-4965-adef-5118e8556d0b admin admin] PATH_INFO: `/v3/projects` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:56:32.453071 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5e29e27-7dae-4965-adef-5118e8556d0b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:56:32.453071 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5e29e27-7dae-4965-adef-5118e8556d0b admin admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:56:32.498227 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b6d467e-6b32-440a-b964-6036a07fc6fa None None] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:56:32.503040 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-819d5fb7-3b92-4bc4-a122-48301ff19d52 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d9bd8d649154c979be3744bfd016f95', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b992d6a65df4bf9b27c13192d57c5f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:56:32.503753 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819d5fb7-3b92-4bc4-a122-48301ff19d52 service neutron] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:56:32.504094 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819d5fb7-3b92-4bc4-a122-48301ff19d52 service neutron] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:56:32.504440 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819d5fb7-3b92-4bc4-a122-48301ff19d52 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:56:32.514570 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-819d5fb7-3b92-4bc4-a122-48301ff19d52 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:56:32.516119 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-819d5fb7-3b92-4bc4-a122-48301ff19d52 service neutron] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:56:32.526902 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 1066/2147] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 1 16:56:32 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 16:56:32.533815 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 1082/2148] 10.222.0.20 () {60 vars in 1153 bytes} [Mon Aug 1 16:56:32 2022] POST /identity/v3/projects => generated 377 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 16:56:32.537782 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-819d5fb7-3b92-4bc4-a122-48301ff19d52 service neutron] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:56:32.551161 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-819d5fb7-3b92-4bc4-a122-48301ff19d52 service neutron] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:56:32.553434 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG dbcounter [-] [87057] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=87057) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:56:32.619288 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc12f546-649d-4405-8561-8ccf80abe6a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:56:32.619939 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc12f546-649d-4405-8561-8ccf80abe6a0 admin admin] REQUEST_METHOD: `POST` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:56:32.620220 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc12f546-649d-4405-8561-8ccf80abe6a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:56:32.620496 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc12f546-649d-4405-8561-8ccf80abe6a0 admin admin] PATH_INFO: `/v3/users` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:56:32.621184 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc12f546-649d-4405-8561-8ccf80abe6a0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:56:32.623222 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc12f546-649d-4405-8561-8ccf80abe6a0 admin admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:56:32.727907 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 1067/2149] 10.222.0.20 () {60 vars in 1144 bytes} [Mon Aug 1 16:56:32 2022] POST /identity/v3/users => generated 289 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 16:56:32.732154 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e29e27-7dae-4965-adef-5118e8556d0b admin admin] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:56:32.745246 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e29e27-7dae-4965-adef-5118e8556d0b admin admin] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:56:32.751017 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18e82066-16aa-41db-bced-e57afcf9fd75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:56:32.751851 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e82066-16aa-41db-bced-e57afcf9fd75 admin admin] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:56:32.752129 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e82066-16aa-41db-bced-e57afcf9fd75 admin admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:56:32.752399 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e82066-16aa-41db-bced-e57afcf9fd75 admin admin] PATH_INFO: `/v3/roles` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:56:32.753243 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18e82066-16aa-41db-bced-e57afcf9fd75 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:56:32.755099 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18e82066-16aa-41db-bced-e57afcf9fd75 admin admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:56:32.771259 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 1083/2150] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Aug 1 16:56:32 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 16:56:32.775418 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc12f546-649d-4405-8561-8ccf80abe6a0 admin admin] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:56:32.795505 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc12f546-649d-4405-8561-8ccf80abe6a0 admin admin] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:56:32.803995 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-335ea148-a797-4f38-994a-d83d33be755d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:56:32.804756 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335ea148-a797-4f38-994a-d83d33be755d admin admin] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:56:32.805068 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335ea148-a797-4f38-994a-d83d33be755d admin admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:56:32.805363 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335ea148-a797-4f38-994a-d83d33be755d admin admin] PATH_INFO: `/v3/roles` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:56:32.807722 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-335ea148-a797-4f38-994a-d83d33be755d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:56:32.809700 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-335ea148-a797-4f38-994a-d83d33be755d admin admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:56:32.821304 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 1068/2151] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Aug 1 16:56:32 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 16:56:32.826702 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18e82066-16aa-41db-bced-e57afcf9fd75 admin admin] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:56:32.839137 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18e82066-16aa-41db-bced-e57afcf9fd75 admin admin] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:56:32.844392 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d15102e-2df4-4aab-ad6b-6118b9463ba4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:56:32.845071 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d15102e-2df4-4aab-ad6b-6118b9463ba4 admin admin] REQUEST_METHOD: `PUT` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:56:32.845352 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d15102e-2df4-4aab-ad6b-6118b9463ba4 admin admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:56:32.845627 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d15102e-2df4-4aab-ad6b-6118b9463ba4 admin admin] PATH_INFO: `/v3/projects/a1978a2f0e23448ba0ecd38d452287fa/users/5126b0c08930428ab0e0bd3ca1bb3555/roles/0999d3da63a545f18ff3252afdc90518` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:56:32.863647 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d15102e-2df4-4aab-ad6b-6118b9463ba4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a1978a2f0e23448ba0ecd38d452287fa, user_id=5126b0c08930428ab0e0bd3ca1bb3555, role_id=0999d3da63a545f18ff3252afdc90518)` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:56:32.866688 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d15102e-2df4-4aab-ad6b-6118b9463ba4 admin admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:56:32.977403 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 1084/2152] 10.222.0.20 () {60 vars in 1483 bytes} [Mon Aug 1 16:56:32 2022] PUT /identity/v3/projects/a1978a2f0e23448ba0ecd38d452287fa/users/5126b0c08930428ab0e0bd3ca1bb3555/roles/0999d3da63a545f18ff3252afdc90518 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 16:56:32.988767 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c98764-7ccd-450b-b058-f5773a024407 None None] REQUEST_METHOD: `POST` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:56:32.989267 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c98764-7ccd-450b-b058-f5773a024407 None None] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:56:32.989362 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c98764-7ccd-450b-b058-f5773a024407 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:56:33.007367 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG dbcounter [-] [87057] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87057) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:56:33.008824 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d15102e-2df4-4aab-ad6b-6118b9463ba4 admin admin] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:56:33.011253 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.fernet_utils [None req-4d15102e-2df4-4aab-ad6b-6118b9463ba4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:56:33.029052 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.auth.core [None req-31c98764-7ccd-450b-b058-f5773a024407 None None] MFA Rules not processed for user `5126b0c08930428ab0e0bd3ca1bb3555`. Rule list: `[]` (Enabled: `True`). {{(pid=87057) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:56:33.030353 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.fernet_utils [None req-31c98764-7ccd-450b-b058-f5773a024407 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87057) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:56:33.076226 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG dbcounter [-] [87058] Writing DB stats keystone:SELECT=94,keystone:DELETE=3,keystone:INSERT=3 {{(pid=87058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:56:33.096925 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d15102e-2df4-4aab-ad6b-6118b9463ba4 admin admin] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:56:33.101162 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 1069/2153] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 1 16:56:32 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:56:33.102128 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e1ff52-5ec2-460c-b7c2-4b835cca04c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:56:33.102880 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e1ff52-5ec2-460c-b7c2-4b835cca04c2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:56:33.103166 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e1ff52-5ec2-460c-b7c2-4b835cca04c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:56:33.103446 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e1ff52-5ec2-460c-b7c2-4b835cca04c2 admin admin] PATH_INFO: `/v3/projects/4b65f22ad0cd43f6a29ac52e5b93a2ae` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:56:33.104740 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90e1ff52-5ec2-460c-b7c2-4b835cca04c2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4b65f22ad0cd43f6a29ac52e5b93a2ae)` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:56:33.105984 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31c98764-7ccd-450b-b058-f5773a024407 None None] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:56:33.106710 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90e1ff52-5ec2-460c-b7c2-4b835cca04c2 admin admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:56:33.118496 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31c98764-7ccd-450b-b058-f5773a024407 None None] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:56:33.123299 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd3dc81b-f4ba-42fe-b823-dfe6ac508300 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:56:33.126537 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3dc81b-f4ba-42fe-b823-dfe6ac508300 admin admin] REQUEST_METHOD: `POST` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:56:33.127114 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3dc81b-f4ba-42fe-b823-dfe6ac508300 admin admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:56:33.127602 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3dc81b-f4ba-42fe-b823-dfe6ac508300 admin admin] PATH_INFO: `/v3/projects` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:56:33.128552 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd3dc81b-f4ba-42fe-b823-dfe6ac508300 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:56:33.130731 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd3dc81b-f4ba-42fe-b823-dfe6ac508300 admin admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:56:33.252556 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 1070/2154] 10.222.0.20 () {60 vars in 1153 bytes} [Mon Aug 1 16:56:33 2022] POST /identity/v3/projects => generated 377 bytes in 147 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 16:56:33.257138 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd3dc81b-f4ba-42fe-b823-dfe6ac508300 admin admin] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:56:33.269051 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.notifications [None req-90e1ff52-5ec2-460c-b7c2-4b835cca04c2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4b65f22ad0cd43f6a29ac52e5b93a2ae', 'request_id': 'req-90e1ff52-5ec2-460c-b7c2-4b835cca04c2'} {{(pid=87058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:56:33.287814 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.notifications [None req-90e1ff52-5ec2-460c-b7c2-4b835cca04c2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4b65f22ad0cd43f6a29ac52e5b93a2ae', 'request_id': 'req-90e1ff52-5ec2-460c-b7c2-4b835cca04c2'} {{(pid=87058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:56:33.308363 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.notifications [None req-90e1ff52-5ec2-460c-b7c2-4b835cca04c2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4b65f22ad0cd43f6a29ac52e5b93a2ae', 'request_id': 'req-90e1ff52-5ec2-460c-b7c2-4b835cca04c2'} {{(pid=87058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:56:33.354709 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd3dc81b-f4ba-42fe-b823-dfe6ac508300 admin admin] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:56:33.361299 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-725c5df3-299a-4ef1-9404-452513a697ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:56:33.362371 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-725c5df3-299a-4ef1-9404-452513a697ee admin admin] REQUEST_METHOD: `POST` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:56:33.362930 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-725c5df3-299a-4ef1-9404-452513a697ee admin admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:56:33.363408 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-725c5df3-299a-4ef1-9404-452513a697ee admin admin] PATH_INFO: `/v3/users` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:56:33.364286 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-725c5df3-299a-4ef1-9404-452513a697ee admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:56:33.366459 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-725c5df3-299a-4ef1-9404-452513a697ee admin admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:56:33.405905 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG dbcounter [-] [87057] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87057) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:56:33.410718 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.notifications [None req-90e1ff52-5ec2-460c-b7c2-4b835cca04c2 admin admin] The token cache is being invalidate because project 4b65f22ad0cd43f6a29ac52e5b93a2ae was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87058) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 16:56:33.411021 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.notifications [None req-90e1ff52-5ec2-460c-b7c2-4b835cca04c2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:56:33.411677 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.notifications [None req-90e1ff52-5ec2-460c-b7c2-4b835cca04c2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:56:33.413274 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 1085/2155] 10.222.0.20 () {58 vars in 1233 bytes} [Mon Aug 1 16:56:33 2022] DELETE /identity/v3/projects/4b65f22ad0cd43f6a29ac52e5b93a2ae => generated 0 bytes in 405 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 16:56:33.417037 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e1ff52-5ec2-460c-b7c2-4b835cca04c2 admin admin] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:56:33.417037 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.fernet_utils [None req-90e1ff52-5ec2-460c-b7c2-4b835cca04c2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:56:33.481169 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e1ff52-5ec2-460c-b7c2-4b835cca04c2 admin admin] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:56:33.486877 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-732bd70f-de96-46bf-8e71-1308be271223 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:56:33.490819 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 1071/2156] 10.222.0.20 () {60 vars in 1144 bytes} [Mon Aug 1 16:56:33 2022] POST /identity/v3/users => generated 295 bytes in 230 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 16:56:33.490819 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d997ca-8882-40bf-95ef-4436a725b441 None None] REQUEST_METHOD: `POST` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:56:33.490819 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d997ca-8882-40bf-95ef-4436a725b441 None None] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:56:33.490819 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d997ca-8882-40bf-95ef-4436a725b441 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:56:33.491263 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732bd70f-de96-46bf-8e71-1308be271223 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:56:33.491263 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732bd70f-de96-46bf-8e71-1308be271223 admin admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:56:33.491263 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732bd70f-de96-46bf-8e71-1308be271223 admin admin] PATH_INFO: `/v3/users/30ef28ff292a4c2598083f3c8ff0de9e` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:56:33.492631 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-732bd70f-de96-46bf-8e71-1308be271223 admin admin] RBAC: Authorizing `identity:delete_user(user_id=30ef28ff292a4c2598083f3c8ff0de9e)` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:56:33.494654 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-732bd70f-de96-46bf-8e71-1308be271223 admin admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:56:33.508823 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.auth.core [None req-53d997ca-8882-40bf-95ef-4436a725b441 None None] MFA Rules not processed for user `1007f8e379b74e169b98fb7c3c0ba5b4`. Rule list: `[]` (Enabled: `True`). {{(pid=87057) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:56:33.510155 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.fernet_utils [None req-53d997ca-8882-40bf-95ef-4436a725b441 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87057) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:56:33.519012 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 1072/2157] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 1 16:56:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:56:33.519320 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53d997ca-8882-40bf-95ef-4436a725b441 None None] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:56:33.520645 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.fernet_utils [None req-53d997ca-8882-40bf-95ef-4436a725b441 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87057) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:56:33.534588 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53d997ca-8882-40bf-95ef-4436a725b441 None None] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:56:33.539367 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4146c591-c76c-4890-afc4-b84fcc2810b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:56:33.539986 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4146c591-c76c-4890-afc4-b84fcc2810b5 admin admin] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:56:33.540265 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4146c591-c76c-4890-afc4-b84fcc2810b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:56:33.540539 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4146c591-c76c-4890-afc4-b84fcc2810b5 admin admin] PATH_INFO: `/v3/roles` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:56:33.541270 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4146c591-c76c-4890-afc4-b84fcc2810b5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:56:33.543503 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4146c591-c76c-4890-afc4-b84fcc2810b5 admin admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:56:33.553512 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 1073/2158] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Aug 1 16:56:33 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 16:56:33.554814 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ac9877-baf2-4d1a-a805-5c543341d270 None None] REQUEST_METHOD: `POST` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:56:33.555099 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ac9877-baf2-4d1a-a805-5c543341d270 None None] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:56:33.555373 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ac9877-baf2-4d1a-a805-5c543341d270 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:56:33.574304 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.auth.core [None req-c8ac9877-baf2-4d1a-a805-5c543341d270 None None] MFA Rules not processed for user `1007f8e379b74e169b98fb7c3c0ba5b4`. Rule list: `[]` (Enabled: `True`). {{(pid=87057) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:56:33.575497 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.fernet_utils [None req-c8ac9877-baf2-4d1a-a805-5c543341d270 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87057) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:56:33.583315 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 1074/2159] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 1 16:56:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:56:33.584040 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8ac9877-baf2-4d1a-a805-5c543341d270 None None] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:56:33.593442 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8ac9877-baf2-4d1a-a805-5c543341d270 None None] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:56:33.598170 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d94c9bbf-2078-44d6-a1b3-3646edf5208c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:56:33.598806 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94c9bbf-2078-44d6-a1b3-3646edf5208c admin admin] REQUEST_METHOD: `PUT` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:56:33.599080 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94c9bbf-2078-44d6-a1b3-3646edf5208c admin admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:56:33.599348 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94c9bbf-2078-44d6-a1b3-3646edf5208c admin admin] PATH_INFO: `/v3/projects/835b98983652463f8a9e806bc98a0a6f/users/96f87d5e7ebd44c5a0c9bc5fc42b8eab/roles/440bb943dd5b46fea038b903b6eaaf81` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:56:33.615094 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d94c9bbf-2078-44d6-a1b3-3646edf5208c admin admin] RBAC: Authorizing `identity:create_grant(project_id=835b98983652463f8a9e806bc98a0a6f, user_id=96f87d5e7ebd44c5a0c9bc5fc42b8eab, role_id=440bb943dd5b46fea038b903b6eaaf81)` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:56:33.618263 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d94c9bbf-2078-44d6-a1b3-3646edf5208c admin admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:56:33.687492 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.notifications [None req-732bd70f-de96-46bf-8e71-1308be271223 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '30ef28ff292a4c2598083f3c8ff0de9e'} {{(pid=87058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:56:33.688153 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.notifications [None req-732bd70f-de96-46bf-8e71-1308be271223 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '30ef28ff292a4c2598083f3c8ff0de9e'} {{(pid=87058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:56:33.688766 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.notifications [None req-732bd70f-de96-46bf-8e71-1308be271223 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '30ef28ff292a4c2598083f3c8ff0de9e'} {{(pid=87058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:56:33.701192 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.notifications [None req-732bd70f-de96-46bf-8e71-1308be271223 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '30ef28ff292a4c2598083f3c8ff0de9e'} {{(pid=87058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:56:33.711555 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 1075/2160] 10.222.0.20 () {60 vars in 1483 bytes} [Mon Aug 1 16:56:33 2022] PUT /identity/v3/projects/835b98983652463f8a9e806bc98a0a6f/users/96f87d5e7ebd44c5a0c9bc5fc42b8eab/roles/440bb943dd5b46fea038b903b6eaaf81 => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 16:56:33.711702 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.notifications [None req-732bd70f-de96-46bf-8e71-1308be271223 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '30ef28ff292a4c2598083f3c8ff0de9e'} {{(pid=87058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:56:33.711824 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d94c9bbf-2078-44d6-a1b3-3646edf5208c admin admin] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:56:33.711824 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.fernet_utils [None req-d94c9bbf-2078-44d6-a1b3-3646edf5208c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87057) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:56:33.778563 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d94c9bbf-2078-44d6-a1b3-3646edf5208c admin admin] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:56:33.783452 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd62d029-c3ca-4fb0-8c1d-b76b5955c997 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:56:33.784070 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd62d029-c3ca-4fb0-8c1d-b76b5955c997 admin admin] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:56:33.784354 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd62d029-c3ca-4fb0-8c1d-b76b5955c997 admin admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:56:33.784628 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd62d029-c3ca-4fb0-8c1d-b76b5955c997 admin admin] PATH_INFO: `/v3/domains` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:56:33.785326 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd62d029-c3ca-4fb0-8c1d-b76b5955c997 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87057) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:56:33.785631 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd62d029-c3ca-4fb0-8c1d-b76b5955c997 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:56:33.787268 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd62d029-c3ca-4fb0-8c1d-b76b5955c997 admin admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:56:33.798932 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 1076/2161] 10.222.0.20 () {58 vars in 1153 bytes} [Mon Aug 1 16:56:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:56:33.799873 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd62d029-c3ca-4fb0-8c1d-b76b5955c997 admin admin] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:56:33.801505 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.fernet_utils [None req-dd62d029-c3ca-4fb0-8c1d-b76b5955c997 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87057) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:56:33.814588 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd62d029-c3ca-4fb0-8c1d-b76b5955c997 admin admin] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:56:33.819214 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec4a2db-2912-427e-801a-3280beaeeaf9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:56:33.819798 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec4a2db-2912-427e-801a-3280beaeeaf9 admin admin] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:56:33.820085 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec4a2db-2912-427e-801a-3280beaeeaf9 admin admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:56:33.820358 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec4a2db-2912-427e-801a-3280beaeeaf9 admin admin] PATH_INFO: `/v3/roles` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:56:33.821037 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ec4a2db-2912-427e-801a-3280beaeeaf9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:56:33.822709 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ec4a2db-2912-427e-801a-3280beaeeaf9 admin admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:56:33.828939 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 1086/2162] 10.222.0.20 () {58 vars in 1224 bytes} [Mon Aug 1 16:56:33 2022] DELETE /identity/v3/users/30ef28ff292a4c2598083f3c8ff0de9e => generated 0 bytes in 415 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 16:56:33.832547 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-732bd70f-de96-46bf-8e71-1308be271223 admin admin] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:56:33.834592 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 1077/2163] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Aug 1 16:56:33 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 16:56:33.835918 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3fc1cc-cdd3-47db-928c-72d36a482271 None None] REQUEST_METHOD: `POST` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:56:33.836284 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3fc1cc-cdd3-47db-928c-72d36a482271 None None] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:56:33.836617 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG dbcounter [-] [87058] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=87058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:56:33.836774 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3fc1cc-cdd3-47db-928c-72d36a482271 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:56:33.856216 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.auth.core [None req-da3fc1cc-cdd3-47db-928c-72d36a482271 None None] MFA Rules not processed for user `1007f8e379b74e169b98fb7c3c0ba5b4`. Rule list: `[]` (Enabled: `True`). {{(pid=87057) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:56:33.857430 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.fernet_utils [None req-da3fc1cc-cdd3-47db-928c-72d36a482271 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87057) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:56:33.899262 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-732bd70f-de96-46bf-8e71-1308be271223 admin admin] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:56:33.905183 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e810b84-3713-4a44-8121-d5620124fc3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:56:33.906341 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e810b84-3713-4a44-8121-d5620124fc3c admin admin] REQUEST_METHOD: `POST` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:56:33.906730 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e810b84-3713-4a44-8121-d5620124fc3c admin admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:56:33.907107 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e810b84-3713-4a44-8121-d5620124fc3c admin admin] PATH_INFO: `/v3/projects` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:56:33.908032 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e810b84-3713-4a44-8121-d5620124fc3c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:56:33.910863 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e810b84-3713-4a44-8121-d5620124fc3c admin admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:56:33.931476 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 1078/2164] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 1 16:56:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:56:33.933104 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da3fc1cc-cdd3-47db-928c-72d36a482271 None None] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:56:33.943265 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da3fc1cc-cdd3-47db-928c-72d36a482271 None None] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:56:33.949351 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d789aa91-5268-4d6c-9e56-83e2dfa37d64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:56:33.950066 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d789aa91-5268-4d6c-9e56-83e2dfa37d64 admin admin] REQUEST_METHOD: `PUT` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:56:33.950362 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d789aa91-5268-4d6c-9e56-83e2dfa37d64 admin admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:56:33.950639 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d789aa91-5268-4d6c-9e56-83e2dfa37d64 admin admin] PATH_INFO: `/v3/projects/835b98983652463f8a9e806bc98a0a6f/users/96f87d5e7ebd44c5a0c9bc5fc42b8eab/roles/440bb943dd5b46fea038b903b6eaaf81` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:56:33.952814 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d789aa91-5268-4d6c-9e56-83e2dfa37d64 admin admin] RBAC: Authorizing `identity:create_grant(project_id=835b98983652463f8a9e806bc98a0a6f, user_id=96f87d5e7ebd44c5a0c9bc5fc42b8eab, role_id=440bb943dd5b46fea038b903b6eaaf81)` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:56:33.955541 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d789aa91-5268-4d6c-9e56-83e2dfa37d64 admin admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:56:33.966923 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 1079/2165] 10.222.0.20 () {60 vars in 1483 bytes} [Mon Aug 1 16:56:33 2022] PUT /identity/v3/projects/835b98983652463f8a9e806bc98a0a6f/users/96f87d5e7ebd44c5a0c9bc5fc42b8eab/roles/440bb943dd5b46fea038b903b6eaaf81 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 16:56:33.967761 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d789aa91-5268-4d6c-9e56-83e2dfa37d64 admin admin] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:56:33.976347 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.fernet_utils [None req-d789aa91-5268-4d6c-9e56-83e2dfa37d64 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87057) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:56:34.036342 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d789aa91-5268-4d6c-9e56-83e2dfa37d64 admin admin] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:56:34.037158 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 1087/2166] 10.222.0.20 () {60 vars in 1153 bytes} [Mon Aug 1 16:56:33 2022] POST /identity/v3/projects => generated 369 bytes in 207 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 16:56:34.038923 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da4cf88-fb2d-4cf2-82be-a5b46f34fd36 None None] REQUEST_METHOD: `POST` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:56:34.039226 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da4cf88-fb2d-4cf2-82be-a5b46f34fd36 None None] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:56:34.039587 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da4cf88-fb2d-4cf2-82be-a5b46f34fd36 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:56:34.043207 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG dbcounter [-] [87057] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=87057) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:56:34.077316 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.auth.core [None req-2da4cf88-fb2d-4cf2-82be-a5b46f34fd36 None None] MFA Rules not processed for user `96f87d5e7ebd44c5a0c9bc5fc42b8eab`. Rule list: `[]` (Enabled: `True`). {{(pid=87058) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:56:34.078655 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.fernet_utils [None req-2da4cf88-fb2d-4cf2-82be-a5b46f34fd36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:56:34.100156 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ab90b02-522c-414e-92e5-8cf279e27b38 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d9bd8d649154c979be3744bfd016f95', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b992d6a65df4bf9b27c13192d57c5f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:56:34.100967 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab90b02-522c-414e-92e5-8cf279e27b38 service neutron] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:56:34.101291 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab90b02-522c-414e-92e5-8cf279e27b38 service neutron] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:56:34.102165 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab90b02-522c-414e-92e5-8cf279e27b38 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:56:34.148557 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 1088/2167] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 1 16:56:34 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:56:34.149480 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da4cf88-fb2d-4cf2-82be-a5b46f34fd36 None None] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:56:34.169403 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ab90b02-522c-414e-92e5-8cf279e27b38 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:56:34.170961 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ab90b02-522c-414e-92e5-8cf279e27b38 service neutron] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:56:34.180759 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 1080/2168] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 1 16:56:33 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 213 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 16:56:34.181585 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ab90b02-522c-414e-92e5-8cf279e27b38 service neutron] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:56:34.183457 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.fernet_utils [None req-6ab90b02-522c-414e-92e5-8cf279e27b38 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87057) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:56:34.227382 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da4cf88-fb2d-4cf2-82be-a5b46f34fd36 None None] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:56:34.234477 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a1c254a-8132-49f2-8f58-40fcaa95dd5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:56:34.236126 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1c254a-8132-49f2-8f58-40fcaa95dd5c admin admin] REQUEST_METHOD: `POST` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:56:34.236533 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1c254a-8132-49f2-8f58-40fcaa95dd5c admin admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:56:34.237253 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1c254a-8132-49f2-8f58-40fcaa95dd5c admin admin] PATH_INFO: `/v3/users` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:56:34.250304 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a1c254a-8132-49f2-8f58-40fcaa95dd5c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:56:34.250304 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a1c254a-8132-49f2-8f58-40fcaa95dd5c admin admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:56:34.269514 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ab90b02-522c-414e-92e5-8cf279e27b38 service neutron] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:56:34.274137 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-134cb0b7-3fec-4466-ba0e-701002c3b65d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25970b52ca15448fa1abb1f974d64038', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b992d6a65df4bf9b27c13192d57c5f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:56:34.274767 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134cb0b7-3fec-4466-ba0e-701002c3b65d service cinder] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:56:34.278285 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134cb0b7-3fec-4466-ba0e-701002c3b65d service cinder] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:56:34.278285 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134cb0b7-3fec-4466-ba0e-701002c3b65d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:56:34.287505 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-134cb0b7-3fec-4466-ba0e-701002c3b65d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:56:34.288985 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-134cb0b7-3fec-4466-ba0e-701002c3b65d service cinder] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:56:34.300401 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 1081/2169] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 1 16:56:34 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 16:56:34.363602 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 1089/2170] 10.222.0.20 () {60 vars in 1144 bytes} [Mon Aug 1 16:56:34 2022] POST /identity/v3/users => generated 285 bytes in 211 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 16:56:34.364852 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-134cb0b7-3fec-4466-ba0e-701002c3b65d service cinder] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:56:34.376906 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-134cb0b7-3fec-4466-ba0e-701002c3b65d service cinder] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:56:34.381797 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e38159b-1356-4f94-8040-de33b8fba7d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:56:34.382449 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e38159b-1356-4f94-8040-de33b8fba7d4 admin admin] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:56:34.382738 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e38159b-1356-4f94-8040-de33b8fba7d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:56:34.383018 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e38159b-1356-4f94-8040-de33b8fba7d4 admin admin] PATH_INFO: `/v3/roles` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:56:34.383720 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e38159b-1356-4f94-8040-de33b8fba7d4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:56:34.385368 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e38159b-1356-4f94-8040-de33b8fba7d4 admin admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:56:34.401864 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 1082/2171] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Aug 1 16:56:34 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 16:56:34.404636 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a1c254a-8132-49f2-8f58-40fcaa95dd5c admin admin] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:56:34.417230 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a1c254a-8132-49f2-8f58-40fcaa95dd5c admin admin] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:56:34.428805 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91805d6a-b7cb-43b1-a0ec-47c401d96aa8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:56:34.429571 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91805d6a-b7cb-43b1-a0ec-47c401d96aa8 admin admin] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:56:34.432094 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91805d6a-b7cb-43b1-a0ec-47c401d96aa8 admin admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:56:34.432094 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91805d6a-b7cb-43b1-a0ec-47c401d96aa8 admin admin] PATH_INFO: `/v3/roles` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:56:34.432094 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91805d6a-b7cb-43b1-a0ec-47c401d96aa8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:56:34.432942 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91805d6a-b7cb-43b1-a0ec-47c401d96aa8 admin admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:56:34.450348 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e38159b-1356-4f94-8040-de33b8fba7d4 admin admin] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:56:34.461735 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e38159b-1356-4f94-8040-de33b8fba7d4 admin admin] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:56:34.463198 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 1090/2172] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Aug 1 16:56:34 2022] GET /identity/v3/roles => generated 1420 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 01 16:56:34.467758 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91805d6a-b7cb-43b1-a0ec-47c401d96aa8 admin admin] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:56:34.470311 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02097a34-44a5-4b24-bea2-a7d936349dfd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25970b52ca15448fa1abb1f974d64038', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b992d6a65df4bf9b27c13192d57c5f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:56:34.470952 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02097a34-44a5-4b24-bea2-a7d936349dfd service cinder] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:56:34.471289 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02097a34-44a5-4b24-bea2-a7d936349dfd service cinder] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:56:34.471575 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02097a34-44a5-4b24-bea2-a7d936349dfd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:56:34.475840 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.fernet_utils [None req-02097a34-44a5-4b24-bea2-a7d936349dfd service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87057) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:56:34.482226 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91805d6a-b7cb-43b1-a0ec-47c401d96aa8 admin admin] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:56:34.487551 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de119967-c7e2-439b-ab9f-18a3028ace81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:56:34.488304 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de119967-c7e2-439b-ab9f-18a3028ace81 admin admin] REQUEST_METHOD: `PUT` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:56:34.488662 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de119967-c7e2-439b-ab9f-18a3028ace81 admin admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:56:34.489012 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de119967-c7e2-439b-ab9f-18a3028ace81 admin admin] PATH_INFO: `/v3/projects/1f618f3c053d4a4d8d75a64e3ead4a61/users/69ee6584adbd480e8523a1043160f907/roles/0999d3da63a545f18ff3252afdc90518` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:56:34.506830 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de119967-c7e2-439b-ab9f-18a3028ace81 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1f618f3c053d4a4d8d75a64e3ead4a61, user_id=69ee6584adbd480e8523a1043160f907, role_id=0999d3da63a545f18ff3252afdc90518)` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:56:34.510856 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de119967-c7e2-439b-ab9f-18a3028ace81 admin admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:56:34.566049 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02097a34-44a5-4b24-bea2-a7d936349dfd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:56:34.576449 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02097a34-44a5-4b24-bea2-a7d936349dfd service cinder] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:56:34.588627 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 1083/2173] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 1 16:56:34 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 01 16:56:34.615162 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02097a34-44a5-4b24-bea2-a7d936349dfd service cinder] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:56:34.622485 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.fernet_utils [None req-02097a34-44a5-4b24-bea2-a7d936349dfd service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87057) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:56:34.624958 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 1091/2174] 10.222.0.20 () {60 vars in 1483 bytes} [Mon Aug 1 16:56:34 2022] PUT /identity/v3/projects/1f618f3c053d4a4d8d75a64e3ead4a61/users/69ee6584adbd480e8523a1043160f907/roles/0999d3da63a545f18ff3252afdc90518 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 16:56:34.634355 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02097a34-44a5-4b24-bea2-a7d936349dfd service cinder] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:56:34.635166 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f844de8b-a16a-4114-a165-1c2b19b575c3 None None] REQUEST_METHOD: `POST` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:56:34.635456 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f844de8b-a16a-4114-a165-1c2b19b575c3 None None] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:56:34.638946 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f844de8b-a16a-4114-a165-1c2b19b575c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:56:34.673403 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.auth.core [None req-f844de8b-a16a-4114-a165-1c2b19b575c3 None None] MFA Rules not processed for user `69ee6584adbd480e8523a1043160f907`. Rule list: `[]` (Enabled: `True`). {{(pid=87058) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:56:34.674389 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.fernet_utils [None req-f844de8b-a16a-4114-a165-1c2b19b575c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:56:34.699895 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30cf0d0c-2eba-43f3-998d-81f67696921c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25970b52ca15448fa1abb1f974d64038', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b992d6a65df4bf9b27c13192d57c5f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:56:34.700520 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30cf0d0c-2eba-43f3-998d-81f67696921c service cinder] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:56:34.700798 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30cf0d0c-2eba-43f3-998d-81f67696921c service cinder] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:56:34.701076 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30cf0d0c-2eba-43f3-998d-81f67696921c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:56:34.702901 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.fernet_utils [None req-30cf0d0c-2eba-43f3-998d-81f67696921c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87057) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:56:34.714787 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG dbcounter [-] [87058] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:56:34.741827 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 1092/2175] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 1 16:56:34 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 16:56:34.743115 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f844de8b-a16a-4114-a165-1c2b19b575c3 None None] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:56:34.745286 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.fernet_utils [None req-f844de8b-a16a-4114-a165-1c2b19b575c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:56:34.766618 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30cf0d0c-2eba-43f3-998d-81f67696921c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:56:34.769031 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30cf0d0c-2eba-43f3-998d-81f67696921c service cinder] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:56:34.777753 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 1084/2176] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 1 16:56:34 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 163 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 16:56:34.778598 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30cf0d0c-2eba-43f3-998d-81f67696921c service cinder] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:56:34.779985 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.fernet_utils [None req-30cf0d0c-2eba-43f3-998d-81f67696921c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87057) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:56:34.843095 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f844de8b-a16a-4114-a165-1c2b19b575c3 None None] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:56:34.851269 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f746f2-820b-459d-9365-847a3f006c48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:56:34.852210 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f746f2-820b-459d-9365-847a3f006c48 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:56:34.867075 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f746f2-820b-459d-9365-847a3f006c48 admin admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:56:34.867075 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f746f2-820b-459d-9365-847a3f006c48 admin admin] PATH_INFO: `/v3/projects/9efbcdf51b7346959297c5d0d67a9643` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:56:34.867075 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83f746f2-820b-459d-9365-847a3f006c48 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9efbcdf51b7346959297c5d0d67a9643)` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:56:34.867075 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83f746f2-820b-459d-9365-847a3f006c48 admin admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:56:34.894598 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30cf0d0c-2eba-43f3-998d-81f67696921c service cinder] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:56:34.902890 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33dcfdab-7ba3-4db2-8fa8-167c6cd1ad18 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd80694e439e2401d9491712666409016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b992d6a65df4bf9b27c13192d57c5f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:56:34.903561 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33dcfdab-7ba3-4db2-8fa8-167c6cd1ad18 service glance] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:56:34.903854 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33dcfdab-7ba3-4db2-8fa8-167c6cd1ad18 service glance] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:56:34.904446 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33dcfdab-7ba3-4db2-8fa8-167c6cd1ad18 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:56:34.915286 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33dcfdab-7ba3-4db2-8fa8-167c6cd1ad18 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:56:34.917329 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33dcfdab-7ba3-4db2-8fa8-167c6cd1ad18 service glance] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:56:34.926487 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 1085/2177] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 1 16:56:34 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 148 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 16:56:34.943085 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG dbcounter [-] [87057] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87057) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:56:35.019108 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.notifications [None req-83f746f2-820b-459d-9365-847a3f006c48 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9efbcdf51b7346959297c5d0d67a9643', 'request_id': 'req-83f746f2-820b-459d-9365-847a3f006c48'} {{(pid=87058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:56:35.023768 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG dbcounter [-] [87058] Writing DB stats keystone:SELECT=93,keystone:INSERT=2,keystone:DELETE=5 {{(pid=87058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:56:35.027810 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.notifications [None req-83f746f2-820b-459d-9365-847a3f006c48 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9efbcdf51b7346959297c5d0d67a9643', 'request_id': 'req-83f746f2-820b-459d-9365-847a3f006c48'} {{(pid=87058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:56:35.033884 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.notifications [None req-83f746f2-820b-459d-9365-847a3f006c48 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9efbcdf51b7346959297c5d0d67a9643', 'request_id': 'req-83f746f2-820b-459d-9365-847a3f006c48'} {{(pid=87058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:56:35.206950 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.notifications [None req-83f746f2-820b-459d-9365-847a3f006c48 admin admin] The token cache is being invalidate because project 9efbcdf51b7346959297c5d0d67a9643 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87058) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 16:56:35.207351 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.notifications [None req-83f746f2-820b-459d-9365-847a3f006c48 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:56:35.208061 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.notifications [None req-83f746f2-820b-459d-9365-847a3f006c48 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:56:35.210004 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 1093/2178] 10.222.0.20 () {58 vars in 1233 bytes} [Mon Aug 1 16:56:34 2022] DELETE /identity/v3/projects/9efbcdf51b7346959297c5d0d67a9643 => generated 0 bytes in 467 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 16:56:35.216203 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775a3383-02bf-42f2-b082-0c675c497acb None None] REQUEST_METHOD: `POST` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:56:35.216724 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775a3383-02bf-42f2-b082-0c675c497acb None None] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:56:35.217118 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775a3383-02bf-42f2-b082-0c675c497acb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:56:35.241237 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.auth.core [None req-775a3383-02bf-42f2-b082-0c675c497acb None None] MFA Rules not processed for user `1007f8e379b74e169b98fb7c3c0ba5b4`. Rule list: `[]` (Enabled: `True`). {{(pid=87057) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:56:35.242786 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.fernet_utils [None req-775a3383-02bf-42f2-b082-0c675c497acb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87057) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:56:35.303183 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 1086/2179] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 1 16:56:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:56:35.311206 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e08937-efd2-4d14-86fc-6c40efb64779 None None] REQUEST_METHOD: `POST` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:56:35.311492 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e08937-efd2-4d14-86fc-6c40efb64779 None None] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:56:35.311769 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e08937-efd2-4d14-86fc-6c40efb64779 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:56:35.333329 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.auth.core [None req-94e08937-efd2-4d14-86fc-6c40efb64779 None None] MFA Rules not processed for user `1007f8e379b74e169b98fb7c3c0ba5b4`. Rule list: `[]` (Enabled: `True`). {{(pid=87058) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:56:35.334596 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.fernet_utils [None req-94e08937-efd2-4d14-86fc-6c40efb64779 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:56:35.343747 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 1094/2180] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 1 16:56:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:56:35.348225 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-775a3383-02bf-42f2-b082-0c675c497acb None None] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:56:35.361963 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-775a3383-02bf-42f2-b082-0c675c497acb None None] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:56:35.369053 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9522356-d524-4375-a745-7cb9384c03b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:56:35.370090 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9522356-d524-4375-a745-7cb9384c03b8 admin admin] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:56:35.370373 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9522356-d524-4375-a745-7cb9384c03b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:56:35.370876 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9522356-d524-4375-a745-7cb9384c03b8 admin admin] PATH_INFO: `/v3/domains` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:56:35.371611 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9522356-d524-4375-a745-7cb9384c03b8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87057) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:56:35.371937 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9522356-d524-4375-a745-7cb9384c03b8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:56:35.373586 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9522356-d524-4375-a745-7cb9384c03b8 admin admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:56:35.386280 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 1087/2181] 10.222.0.20 () {58 vars in 1153 bytes} [Mon Aug 1 16:56:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:56:35.390582 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e08937-efd2-4d14-86fc-6c40efb64779 None None] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:56:35.399732 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e08937-efd2-4d14-86fc-6c40efb64779 None None] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:56:35.406011 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-586f4b2c-f2d0-4819-aead-2d09ac86099d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:56:35.406011 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586f4b2c-f2d0-4819-aead-2d09ac86099d admin admin] REQUEST_METHOD: `POST` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:56:35.406271 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586f4b2c-f2d0-4819-aead-2d09ac86099d admin admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:56:35.406547 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586f4b2c-f2d0-4819-aead-2d09ac86099d admin admin] PATH_INFO: `/v3/projects` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:56:35.407207 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-586f4b2c-f2d0-4819-aead-2d09ac86099d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:56:35.409192 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-586f4b2c-f2d0-4819-aead-2d09ac86099d admin admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:56:35.538933 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 1095/2182] 10.222.0.20 () {60 vars in 1153 bytes} [Mon Aug 1 16:56:35 2022] POST /identity/v3/projects => generated 383 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 16:56:35.543767 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9522356-d524-4375-a745-7cb9384c03b8 admin admin] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:56:35.553683 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9522356-d524-4375-a745-7cb9384c03b8 admin admin] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:56:35.721800 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG dbcounter [-] [87057] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=87057) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:56:35.778649 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17bc6f31-a982-4796-b35d-522ee42e46ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:56:35.779579 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17bc6f31-a982-4796-b35d-522ee42e46ef admin admin] REQUEST_METHOD: `POST` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:56:35.779698 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17bc6f31-a982-4796-b35d-522ee42e46ef admin admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:56:35.780046 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17bc6f31-a982-4796-b35d-522ee42e46ef admin admin] PATH_INFO: `/v3/users` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:56:35.780875 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17bc6f31-a982-4796-b35d-522ee42e46ef admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:56:35.783012 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17bc6f31-a982-4796-b35d-522ee42e46ef admin admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:56:35.857314 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 1088/2183] 10.222.0.20 () {60 vars in 1144 bytes} [Mon Aug 1 16:56:35 2022] POST /identity/v3/users => generated 292 bytes in 314 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 16:56:35.861462 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-586f4b2c-f2d0-4819-aead-2d09ac86099d admin admin] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:56:35.872772 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-586f4b2c-f2d0-4819-aead-2d09ac86099d admin admin] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:56:35.877202 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae0d9e3-a88b-49d6-9114-eb15eac9a336 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:56:35.877812 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae0d9e3-a88b-49d6-9114-eb15eac9a336 admin admin] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:56:35.878120 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae0d9e3-a88b-49d6-9114-eb15eac9a336 admin admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:56:35.878395 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae0d9e3-a88b-49d6-9114-eb15eac9a336 admin admin] PATH_INFO: `/v3/roles` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:56:35.879111 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cae0d9e3-a88b-49d6-9114-eb15eac9a336 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:56:35.880693 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cae0d9e3-a88b-49d6-9114-eb15eac9a336 admin admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:56:35.892013 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 1096/2184] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Aug 1 16:56:35 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 16:56:35.896290 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17bc6f31-a982-4796-b35d-522ee42e46ef admin admin] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:56:35.907151 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17bc6f31-a982-4796-b35d-522ee42e46ef admin admin] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:56:35.913295 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9eeb623-5313-4464-aa04-0ce4f7939edb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:56:35.914010 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9eeb623-5313-4464-aa04-0ce4f7939edb admin admin] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:56:35.914422 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9eeb623-5313-4464-aa04-0ce4f7939edb admin admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:56:35.914749 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9eeb623-5313-4464-aa04-0ce4f7939edb admin admin] PATH_INFO: `/v3/roles` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:56:35.915569 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9eeb623-5313-4464-aa04-0ce4f7939edb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:56:35.917977 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9eeb623-5313-4464-aa04-0ce4f7939edb admin admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:56:35.928322 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 1089/2185] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Aug 1 16:56:35 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 01 16:56:35.933311 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae0d9e3-a88b-49d6-9114-eb15eac9a336 admin admin] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:56:35.943229 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae0d9e3-a88b-49d6-9114-eb15eac9a336 admin admin] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:56:35.948274 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba4628e6-fcdb-4417-801d-65624359dc07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:56:35.948961 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4628e6-fcdb-4417-801d-65624359dc07 admin admin] REQUEST_METHOD: `PUT` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:56:35.949239 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4628e6-fcdb-4417-801d-65624359dc07 admin admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:56:35.949517 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4628e6-fcdb-4417-801d-65624359dc07 admin admin] PATH_INFO: `/v3/projects/e90a999b9d42435eb544f589329f53cf/users/7a72dec36306426cb9a8548948030ca0/roles/0999d3da63a545f18ff3252afdc90518` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:56:35.965296 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba4628e6-fcdb-4417-801d-65624359dc07 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e90a999b9d42435eb544f589329f53cf, user_id=7a72dec36306426cb9a8548948030ca0, role_id=0999d3da63a545f18ff3252afdc90518)` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:56:35.967998 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba4628e6-fcdb-4417-801d-65624359dc07 admin admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:56:36.028802 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG dbcounter [-] [87058] Writing DB stats keystone:DELETE=12,keystone:SELECT=86,keystone:INSERT=2 {{(pid=87058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:56:36.035378 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 1097/2186] 10.222.0.20 () {60 vars in 1483 bytes} [Mon Aug 1 16:56:35 2022] PUT /identity/v3/projects/e90a999b9d42435eb544f589329f53cf/users/7a72dec36306426cb9a8548948030ca0/roles/0999d3da63a545f18ff3252afdc90518 => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 16:56:36.045878 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99607284-529a-40bf-a6e1-80b914876f6f None None] REQUEST_METHOD: `POST` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:56:36.046206 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99607284-529a-40bf-a6e1-80b914876f6f None None] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:56:36.046480 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99607284-529a-40bf-a6e1-80b914876f6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:56:36.074235 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.auth.core [None req-99607284-529a-40bf-a6e1-80b914876f6f None None] MFA Rules not processed for user `7a72dec36306426cb9a8548948030ca0`. Rule list: `[]` (Enabled: `True`). {{(pid=87057) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:56:36.075369 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.fernet_utils [None req-99607284-529a-40bf-a6e1-80b914876f6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87057) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:56:36.115650 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG dbcounter [-] [87057] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87057) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:56:36.140931 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 1090/2187] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 1 16:56:36 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:56:36.145628 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba4628e6-fcdb-4417-801d-65624359dc07 admin admin] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:56:36.212933 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba4628e6-fcdb-4417-801d-65624359dc07 admin admin] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:56:36.215816 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d38360d-5d2e-4919-b7e3-13fc1ac684e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:56:36.216472 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d38360d-5d2e-4919-b7e3-13fc1ac684e9 admin admin] REQUEST_METHOD: `POST` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:56:36.216738 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d38360d-5d2e-4919-b7e3-13fc1ac684e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:56:36.217015 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d38360d-5d2e-4919-b7e3-13fc1ac684e9 admin admin] PATH_INFO: `/v3/projects` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:56:36.217679 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d38360d-5d2e-4919-b7e3-13fc1ac684e9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:56:36.219700 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d38360d-5d2e-4919-b7e3-13fc1ac684e9 admin admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:56:36.446510 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 1098/2188] 10.222.0.20 () {60 vars in 1153 bytes} [Mon Aug 1 16:56:36 2022] POST /identity/v3/projects => generated 385 bytes in 298 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 16:56:36.448391 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99607284-529a-40bf-a6e1-80b914876f6f None None] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:56:36.459997 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99607284-529a-40bf-a6e1-80b914876f6f None None] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:56:36.515437 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-985984b1-cf53-42fb-a9a7-a2d1857b78d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:56:36.516053 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985984b1-cf53-42fb-a9a7-a2d1857b78d2 admin admin] REQUEST_METHOD: `POST` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:56:36.516327 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985984b1-cf53-42fb-a9a7-a2d1857b78d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:56:36.516595 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985984b1-cf53-42fb-a9a7-a2d1857b78d2 admin admin] PATH_INFO: `/v3/users` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:56:36.517261 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-985984b1-cf53-42fb-a9a7-a2d1857b78d2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:56:36.519309 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-985984b1-cf53-42fb-a9a7-a2d1857b78d2 admin admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:56:36.600751 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 1091/2189] 10.222.0.20 () {60 vars in 1144 bytes} [Mon Aug 1 16:56:36 2022] POST /identity/v3/users => generated 293 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 16:56:36.616847 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d38360d-5d2e-4919-b7e3-13fc1ac684e9 admin admin] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:56:36.627837 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d38360d-5d2e-4919-b7e3-13fc1ac684e9 admin admin] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:56:36.632526 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60131183-53d4-4305-be30-d25fc0cca3f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:56:36.633159 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60131183-53d4-4305-be30-d25fc0cca3f5 admin admin] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:56:36.633419 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60131183-53d4-4305-be30-d25fc0cca3f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:56:36.633694 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60131183-53d4-4305-be30-d25fc0cca3f5 admin admin] PATH_INFO: `/v3/roles` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:56:36.634422 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60131183-53d4-4305-be30-d25fc0cca3f5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:56:36.636114 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60131183-53d4-4305-be30-d25fc0cca3f5 admin admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:56:36.647349 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 1099/2190] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Aug 1 16:56:36 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 01 16:56:36.652445 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-985984b1-cf53-42fb-a9a7-a2d1857b78d2 admin admin] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:56:36.664177 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-985984b1-cf53-42fb-a9a7-a2d1857b78d2 admin admin] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:56:36.676202 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57bba4b3-6e67-465a-9a62-8ddf1f53211e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:56:36.676848 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57bba4b3-6e67-465a-9a62-8ddf1f53211e admin admin] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:56:36.677130 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57bba4b3-6e67-465a-9a62-8ddf1f53211e admin admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:56:36.677404 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57bba4b3-6e67-465a-9a62-8ddf1f53211e admin admin] PATH_INFO: `/v3/roles` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:56:36.678175 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57bba4b3-6e67-465a-9a62-8ddf1f53211e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:56:36.679982 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57bba4b3-6e67-465a-9a62-8ddf1f53211e admin admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:56:36.692407 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 1092/2191] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Aug 1 16:56:36 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 16:56:36.696784 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60131183-53d4-4305-be30-d25fc0cca3f5 admin admin] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:56:36.708286 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60131183-53d4-4305-be30-d25fc0cca3f5 admin admin] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:56:36.713694 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ee31bf-ef06-4338-9bf1-0f5ad560f869 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:56:36.717835 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ee31bf-ef06-4338-9bf1-0f5ad560f869 admin admin] REQUEST_METHOD: `PUT` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:56:36.717835 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ee31bf-ef06-4338-9bf1-0f5ad560f869 admin admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:56:36.717835 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ee31bf-ef06-4338-9bf1-0f5ad560f869 admin admin] PATH_INFO: `/v3/projects/1ab357bf50d64943a65a7ce3c22ce714/users/5b57042db90047929c6b8a7cf36e286c/roles/0999d3da63a545f18ff3252afdc90518` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:56:36.730165 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33ee31bf-ef06-4338-9bf1-0f5ad560f869 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1ab357bf50d64943a65a7ce3c22ce714, user_id=5b57042db90047929c6b8a7cf36e286c, role_id=0999d3da63a545f18ff3252afdc90518)` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:56:36.732838 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33ee31bf-ef06-4338-9bf1-0f5ad560f869 admin admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:56:36.847178 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 1100/2192] 10.222.0.20 () {60 vars in 1483 bytes} [Mon Aug 1 16:56:36 2022] PUT /identity/v3/projects/1ab357bf50d64943a65a7ce3c22ce714/users/5b57042db90047929c6b8a7cf36e286c/roles/0999d3da63a545f18ff3252afdc90518 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 16:56:36.859322 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d07448f-d8c9-41c1-993c-d3781069f499 None None] REQUEST_METHOD: `POST` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:56:36.859535 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d07448f-d8c9-41c1-993c-d3781069f499 None None] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:56:36.859808 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d07448f-d8c9-41c1-993c-d3781069f499 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:56:36.889002 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.auth.core [None req-2d07448f-d8c9-41c1-993c-d3781069f499 None None] MFA Rules not processed for user `5b57042db90047929c6b8a7cf36e286c`. Rule list: `[]` (Enabled: `True`). {{(pid=87057) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:56:36.890313 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.fernet_utils [None req-2d07448f-d8c9-41c1-993c-d3781069f499 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87057) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:56:36.974874 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 1093/2193] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 1 16:56:36 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:56:36.979932 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ee31bf-ef06-4338-9bf1-0f5ad560f869 admin admin] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:56:37.012361 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG dbcounter [-] [87057] Writing DB stats keystone:SELECT=100 {{(pid=87057) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:56:37.031812 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d07448f-d8c9-41c1-993c-d3781069f499 None None] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:56:37.033333 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.fernet_utils [None req-2d07448f-d8c9-41c1-993c-d3781069f499 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87057) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:56:37.048830 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ee31bf-ef06-4338-9bf1-0f5ad560f869 admin admin] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:56:37.053666 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b927f7-6902-425a-bc49-7dd9912463ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:56:37.054319 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b927f7-6902-425a-bc49-7dd9912463ed admin admin] REQUEST_METHOD: `POST` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:56:37.054602 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b927f7-6902-425a-bc49-7dd9912463ed admin admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:56:37.054875 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b927f7-6902-425a-bc49-7dd9912463ed admin admin] PATH_INFO: `/v3/projects` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:56:37.055589 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29b927f7-6902-425a-bc49-7dd9912463ed admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:56:37.058236 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29b927f7-6902-425a-bc49-7dd9912463ed admin admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:56:37.076071 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG dbcounter [-] [87058] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=87058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:56:37.090346 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d07448f-d8c9-41c1-993c-d3781069f499 None None] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:56:37.093928 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77f8696d-12f8-45a0-9140-83c461303889 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25970b52ca15448fa1abb1f974d64038', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b992d6a65df4bf9b27c13192d57c5f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:56:37.094573 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f8696d-12f8-45a0-9140-83c461303889 service cinder] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:56:37.094854 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f8696d-12f8-45a0-9140-83c461303889 service cinder] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:56:37.095127 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f8696d-12f8-45a0-9140-83c461303889 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:56:37.096906 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.fernet_utils [None req-77f8696d-12f8-45a0-9140-83c461303889 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87057) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:56:37.104115 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG dbcounter [-] [87057] Writing DB stats keystone:SELECT=100 {{(pid=87057) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:56:37.160611 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 1101/2194] 10.222.0.20 () {60 vars in 1153 bytes} [Mon Aug 1 16:56:36 2022] POST /identity/v3/projects => generated 385 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 16:56:37.166834 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77f8696d-12f8-45a0-9140-83c461303889 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:56:37.166834 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77f8696d-12f8-45a0-9140-83c461303889 service cinder] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:56:37.167099 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b927f7-6902-425a-bc49-7dd9912463ed admin admin] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:56:37.173431 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 1094/2195] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 1 16:56:37 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 01 16:56:37.226883 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b927f7-6902-425a-bc49-7dd9912463ed admin admin] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:56:37.231583 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca28c2dc-e31a-477a-bc48-fc10ab6974d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:56:37.232204 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca28c2dc-e31a-477a-bc48-fc10ab6974d3 admin admin] REQUEST_METHOD: `POST` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:56:37.232476 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca28c2dc-e31a-477a-bc48-fc10ab6974d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:56:37.232750 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca28c2dc-e31a-477a-bc48-fc10ab6974d3 admin admin] PATH_INFO: `/v3/users` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:56:37.233409 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca28c2dc-e31a-477a-bc48-fc10ab6974d3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:56:37.235370 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca28c2dc-e31a-477a-bc48-fc10ab6974d3 admin admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:56:37.319685 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 1102/2196] 10.222.0.20 () {60 vars in 1144 bytes} [Mon Aug 1 16:56:37 2022] POST /identity/v3/users => generated 299 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 16:56:37.324368 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77f8696d-12f8-45a0-9140-83c461303889 service cinder] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:56:37.334751 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77f8696d-12f8-45a0-9140-83c461303889 service cinder] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:56:37.340361 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-304bf310-c76d-4101-bba4-ea99c666f1eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:56:37.340361 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304bf310-c76d-4101-bba4-ea99c666f1eb admin admin] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:56:37.340361 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304bf310-c76d-4101-bba4-ea99c666f1eb admin admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:56:37.340675 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304bf310-c76d-4101-bba4-ea99c666f1eb admin admin] PATH_INFO: `/v3/roles` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:56:37.341262 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-304bf310-c76d-4101-bba4-ea99c666f1eb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:56:37.342881 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-304bf310-c76d-4101-bba4-ea99c666f1eb admin admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:56:37.352641 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 1095/2197] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Aug 1 16:56:37 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 16:56:37.356940 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca28c2dc-e31a-477a-bc48-fc10ab6974d3 admin admin] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:56:37.369787 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca28c2dc-e31a-477a-bc48-fc10ab6974d3 admin admin] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:56:37.374699 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3582e644-25e2-4c91-80b3-b99df2da24fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:56:37.375371 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3582e644-25e2-4c91-80b3-b99df2da24fd admin admin] REQUEST_METHOD: `PUT` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:56:37.375653 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3582e644-25e2-4c91-80b3-b99df2da24fd admin admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:56:37.375927 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3582e644-25e2-4c91-80b3-b99df2da24fd admin admin] PATH_INFO: `/v3/projects/c5d07c2715be4095be84462837332666/users/b50ba2e22db7412895f7c137cc412b0d/roles/440bb943dd5b46fea038b903b6eaaf81` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:56:37.388854 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG dbcounter [-] [87057] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=87057) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:56:37.392463 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3582e644-25e2-4c91-80b3-b99df2da24fd admin admin] RBAC: Authorizing `identity:create_grant(project_id=c5d07c2715be4095be84462837332666, user_id=b50ba2e22db7412895f7c137cc412b0d, role_id=440bb943dd5b46fea038b903b6eaaf81)` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:56:37.395221 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3582e644-25e2-4c91-80b3-b99df2da24fd admin admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:56:37.474446 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 1103/2198] 10.222.0.20 () {60 vars in 1483 bytes} [Mon Aug 1 16:56:37 2022] PUT /identity/v3/projects/c5d07c2715be4095be84462837332666/users/b50ba2e22db7412895f7c137cc412b0d/roles/440bb943dd5b46fea038b903b6eaaf81 => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 16:56:37.478907 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-304bf310-c76d-4101-bba4-ea99c666f1eb admin admin] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:56:37.489076 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-304bf310-c76d-4101-bba4-ea99c666f1eb admin admin] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:56:37.515106 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG dbcounter [-] [87058] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=87058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:56:37.549485 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-556fba11-b5e3-421c-8270-f5d1e7619c5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:56:37.550281 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556fba11-b5e3-421c-8270-f5d1e7619c5d admin admin] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:56:37.550711 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556fba11-b5e3-421c-8270-f5d1e7619c5d admin admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:56:37.551097 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556fba11-b5e3-421c-8270-f5d1e7619c5d admin admin] PATH_INFO: `/v3/roles` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:56:37.551894 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-556fba11-b5e3-421c-8270-f5d1e7619c5d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:56:37.553760 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-556fba11-b5e3-421c-8270-f5d1e7619c5d admin admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:56:37.563918 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 1096/2199] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Aug 1 16:56:37 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 16:56:37.568166 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3582e644-25e2-4c91-80b3-b99df2da24fd admin admin] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:56:37.579981 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3582e644-25e2-4c91-80b3-b99df2da24fd admin admin] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:56:37.584590 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b4cfb38-4e50-48a9-8ef0-dcbdee5db1d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:56:37.585243 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b4cfb38-4e50-48a9-8ef0-dcbdee5db1d0 admin admin] REQUEST_METHOD: `PUT` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:56:37.585525 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b4cfb38-4e50-48a9-8ef0-dcbdee5db1d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:56:37.585792 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b4cfb38-4e50-48a9-8ef0-dcbdee5db1d0 admin admin] PATH_INFO: `/v3/projects/c5d07c2715be4095be84462837332666/users/b50ba2e22db7412895f7c137cc412b0d/roles/440bb943dd5b46fea038b903b6eaaf81` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:56:37.588621 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b4cfb38-4e50-48a9-8ef0-dcbdee5db1d0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c5d07c2715be4095be84462837332666, user_id=b50ba2e22db7412895f7c137cc412b0d, role_id=440bb943dd5b46fea038b903b6eaaf81)` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:56:37.592280 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b4cfb38-4e50-48a9-8ef0-dcbdee5db1d0 admin admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:56:37.602475 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 1104/2200] 10.222.0.20 () {60 vars in 1483 bytes} [Mon Aug 1 16:56:37 2022] PUT /identity/v3/projects/c5d07c2715be4095be84462837332666/users/b50ba2e22db7412895f7c137cc412b0d/roles/440bb943dd5b46fea038b903b6eaaf81 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 16:56:37.611631 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG dbcounter [-] [87057] Writing DB stats keystone:SELECT=100 {{(pid=87057) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:56:37.613415 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe72248-6cec-4d6e-a487-a11c57f959ec None None] REQUEST_METHOD: `POST` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:56:37.613737 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe72248-6cec-4d6e-a487-a11c57f959ec None None] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:56:37.614006 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe72248-6cec-4d6e-a487-a11c57f959ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:56:37.645799 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.auth.core [None req-8fe72248-6cec-4d6e-a487-a11c57f959ec None None] MFA Rules not processed for user `b50ba2e22db7412895f7c137cc412b0d`. Rule list: `[]` (Enabled: `True`). {{(pid=87057) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:56:37.647240 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.fernet_utils [None req-8fe72248-6cec-4d6e-a487-a11c57f959ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87057) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:56:37.720712 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 1097/2201] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 1 16:56:37 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:56:37.732502 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b4cfb38-4e50-48a9-8ef0-dcbdee5db1d0 admin admin] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:56:37.744023 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG dbcounter [-] [87058] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=87058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:56:37.790259 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b4cfb38-4e50-48a9-8ef0-dcbdee5db1d0 admin admin] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:56:37.794619 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d8a0649-07e5-4405-86d9-64172f514d18 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25970b52ca15448fa1abb1f974d64038', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b992d6a65df4bf9b27c13192d57c5f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:56:37.795258 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8a0649-07e5-4405-86d9-64172f514d18 service cinder] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:56:37.795554 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8a0649-07e5-4405-86d9-64172f514d18 service cinder] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:56:37.795850 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8a0649-07e5-4405-86d9-64172f514d18 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:56:37.852319 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d8a0649-07e5-4405-86d9-64172f514d18 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:56:37.854452 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d8a0649-07e5-4405-86d9-64172f514d18 service cinder] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:56:37.863759 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 1105/2202] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 1 16:56:37 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 16:56:42.378257 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fe72248-6cec-4d6e-a487-a11c57f959ec None None] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:56:42.382629 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.fernet_utils [None req-8fe72248-6cec-4d6e-a487-a11c57f959ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87057) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:56:42.410173 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fe72248-6cec-4d6e-a487-a11c57f959ec None None] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:56:42.414758 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1848998c-c582-4f4f-a2af-028a7f9e83bd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25970b52ca15448fa1abb1f974d64038', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b992d6a65df4bf9b27c13192d57c5f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:56:42.415384 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1848998c-c582-4f4f-a2af-028a7f9e83bd service cinder] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:56:42.415852 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1848998c-c582-4f4f-a2af-028a7f9e83bd service cinder] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:56:42.416273 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1848998c-c582-4f4f-a2af-028a7f9e83bd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:56:42.477188 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1848998c-c582-4f4f-a2af-028a7f9e83bd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:56:42.478789 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1848998c-c582-4f4f-a2af-028a7f9e83bd service cinder] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:56:42.490752 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 1098/2203] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 1 16:56:42 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 16:56:43.923543 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d8a0649-07e5-4405-86d9-64172f514d18 service cinder] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:56:43.927628 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.fernet_utils [None req-2d8a0649-07e5-4405-86d9-64172f514d18 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:56:43.966908 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG dbcounter [-] [87058] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=87058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:56:44.019334 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d8a0649-07e5-4405-86d9-64172f514d18 service cinder] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:56:44.025017 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa07536-9133-4b63-9826-5120a84f792c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:56:44.026264 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa07536-9133-4b63-9826-5120a84f792c admin admin] REQUEST_METHOD: `DELETE` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:56:44.026797 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa07536-9133-4b63-9826-5120a84f792c admin admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:56:44.027263 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa07536-9133-4b63-9826-5120a84f792c admin admin] PATH_INFO: `/v3/users/bea8963139a74858bfe6c4b3e0cfbf99` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:56:44.028909 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaa07536-9133-4b63-9826-5120a84f792c admin admin] RBAC: Authorizing `identity:delete_user(user_id=bea8963139a74858bfe6c4b3e0cfbf99)` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:56:44.031461 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaa07536-9133-4b63-9826-5120a84f792c admin admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:56:44.182865 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1848998c-c582-4f4f-a2af-028a7f9e83bd service cinder] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:56:44.194414 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1848998c-c582-4f4f-a2af-028a7f9e83bd service cinder] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:56:44.198242 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f71f90fa-4e92-4ff1-8e0d-550561a91326 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25970b52ca15448fa1abb1f974d64038', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b992d6a65df4bf9b27c13192d57c5f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:56:44.199120 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71f90fa-4e92-4ff1-8e0d-550561a91326 service cinder] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:56:44.201900 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71f90fa-4e92-4ff1-8e0d-550561a91326 service cinder] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:56:44.201900 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71f90fa-4e92-4ff1-8e0d-550561a91326 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:56:44.211875 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f71f90fa-4e92-4ff1-8e0d-550561a91326 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:56:44.213734 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f71f90fa-4e92-4ff1-8e0d-550561a91326 service cinder] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:56:44.223798 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 1099/2204] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 1 16:56:44 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 16:56:44.224598 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f71f90fa-4e92-4ff1-8e0d-550561a91326 service cinder] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:56:44.225926 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.fernet_utils [None req-f71f90fa-4e92-4ff1-8e0d-550561a91326 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87057) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:56:44.242474 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f71f90fa-4e92-4ff1-8e0d-550561a91326 service cinder] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:56:44.250219 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e710785-a7e6-467c-8e95-a0021da94ed1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:56:44.251209 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e710785-a7e6-467c-8e95-a0021da94ed1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:56:44.251793 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e710785-a7e6-467c-8e95-a0021da94ed1 admin admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:56:44.252314 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e710785-a7e6-467c-8e95-a0021da94ed1 admin admin] PATH_INFO: `/v3/users/7df0f7fce7854e4bb7a34c3040b0f9d0` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:56:44.254489 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e710785-a7e6-467c-8e95-a0021da94ed1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7df0f7fce7854e4bb7a34c3040b0f9d0)` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:56:44.256459 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e710785-a7e6-467c-8e95-a0021da94ed1 admin admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:56:44.257920 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.notifications [None req-aaa07536-9133-4b63-9826-5120a84f792c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bea8963139a74858bfe6c4b3e0cfbf99'} {{(pid=87058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:56:44.259415 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.notifications [None req-aaa07536-9133-4b63-9826-5120a84f792c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bea8963139a74858bfe6c4b3e0cfbf99'} {{(pid=87058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:56:44.260752 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.notifications [None req-aaa07536-9133-4b63-9826-5120a84f792c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bea8963139a74858bfe6c4b3e0cfbf99'} {{(pid=87058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:56:44.282271 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG dbcounter [-] [87057] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=87057) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:56:44.286279 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.notifications [None req-aaa07536-9133-4b63-9826-5120a84f792c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bea8963139a74858bfe6c4b3e0cfbf99'} {{(pid=87058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:56:44.294570 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.notifications [None req-aaa07536-9133-4b63-9826-5120a84f792c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bea8963139a74858bfe6c4b3e0cfbf99'} {{(pid=87058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:56:44.354690 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 1106/2205] 10.222.0.20 () {58 vars in 1224 bytes} [Mon Aug 1 16:56:43 2022] DELETE /identity/v3/users/bea8963139a74858bfe6c4b3e0cfbf99 => generated 0 bytes in 432 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 16:56:44.359438 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca72424-25fd-4e1d-97cb-30bb7bdb3b35 None None] REQUEST_METHOD: `POST` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:56:44.359839 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca72424-25fd-4e1d-97cb-30bb7bdb3b35 None None] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:56:44.360187 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca72424-25fd-4e1d-97cb-30bb7bdb3b35 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:56:44.380649 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.auth.core [None req-5ca72424-25fd-4e1d-97cb-30bb7bdb3b35 None None] MFA Rules not processed for user `1007f8e379b74e169b98fb7c3c0ba5b4`. Rule list: `[]` (Enabled: `True`). {{(pid=87058) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:56:44.381872 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.fernet_utils [None req-5ca72424-25fd-4e1d-97cb-30bb7bdb3b35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:56:44.417223 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.notifications [None req-6e710785-a7e6-467c-8e95-a0021da94ed1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7df0f7fce7854e4bb7a34c3040b0f9d0'} {{(pid=87057) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:56:44.422916 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.notifications [None req-6e710785-a7e6-467c-8e95-a0021da94ed1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7df0f7fce7854e4bb7a34c3040b0f9d0'} {{(pid=87057) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:56:44.462509 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 1107/2206] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 1 16:56:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 16:56:44.472990 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ca72424-25fd-4e1d-97cb-30bb7bdb3b35 None None] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:56:44.474356 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.fernet_utils [None req-5ca72424-25fd-4e1d-97cb-30bb7bdb3b35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:56:44.514430 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.notifications [None req-6e710785-a7e6-467c-8e95-a0021da94ed1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7df0f7fce7854e4bb7a34c3040b0f9d0'} {{(pid=87057) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:56:44.515274 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.notifications [None req-6e710785-a7e6-467c-8e95-a0021da94ed1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7df0f7fce7854e4bb7a34c3040b0f9d0'} {{(pid=87057) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:56:44.524015 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.notifications [None req-6e710785-a7e6-467c-8e95-a0021da94ed1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7df0f7fce7854e4bb7a34c3040b0f9d0'} {{(pid=87057) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:56:44.533469 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG dbcounter [-] [87058] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:56:44.537673 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ca72424-25fd-4e1d-97cb-30bb7bdb3b35 None None] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:56:44.542281 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e03e07-a970-4669-a37a-7da9f8c4625e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d9bd8d649154c979be3744bfd016f95', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b992d6a65df4bf9b27c13192d57c5f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:56:44.542900 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e03e07-a970-4669-a37a-7da9f8c4625e service neutron] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:56:44.543176 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e03e07-a970-4669-a37a-7da9f8c4625e service neutron] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:56:44.543461 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e03e07-a970-4669-a37a-7da9f8c4625e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:56:44.544944 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 1100/2207] 10.222.0.20 () {58 vars in 1224 bytes} [Mon Aug 1 16:56:44 2022] DELETE /identity/v3/users/7df0f7fce7854e4bb7a34c3040b0f9d0 => generated 0 bytes in 321 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 16:56:44.545279 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.fernet_utils [None req-c9e03e07-a970-4669-a37a-7da9f8c4625e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:56:44.549362 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54524f12-9d4f-4bd2-9377-0fcd17afed7b None None] REQUEST_METHOD: `POST` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:56:44.553369 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54524f12-9d4f-4bd2-9377-0fcd17afed7b None None] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:56:44.553369 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54524f12-9d4f-4bd2-9377-0fcd17afed7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:56:44.558182 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9e03e07-a970-4669-a37a-7da9f8c4625e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:56:44.559643 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9e03e07-a970-4669-a37a-7da9f8c4625e service neutron] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:56:44.569690 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 1108/2208] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 1 16:56:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 01 16:56:44.570145 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.auth.core [None req-54524f12-9d4f-4bd2-9377-0fcd17afed7b None None] MFA Rules not processed for user `1007f8e379b74e169b98fb7c3c0ba5b4`. Rule list: `[]` (Enabled: `True`). {{(pid=87057) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:56:44.571487 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.fernet_utils [None req-54524f12-9d4f-4bd2-9377-0fcd17afed7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87057) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:56:44.655137 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 1101/2209] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 1 16:56:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 16:56:44.665235 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e03e07-a970-4669-a37a-7da9f8c4625e service neutron] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:56:44.667649 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.fernet_utils [None req-c9e03e07-a970-4669-a37a-7da9f8c4625e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:56:44.681273 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e03e07-a970-4669-a37a-7da9f8c4625e service neutron] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:56:44.736769 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-378a8c12-aa10-4771-a132-24bebc189584 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d9bd8d649154c979be3744bfd016f95', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b992d6a65df4bf9b27c13192d57c5f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:56:44.737389 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378a8c12-aa10-4771-a132-24bebc189584 service neutron] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:56:44.737669 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378a8c12-aa10-4771-a132-24bebc189584 service neutron] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:56:44.737940 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378a8c12-aa10-4771-a132-24bebc189584 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:56:44.747992 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-378a8c12-aa10-4771-a132-24bebc189584 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:56:44.749427 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-378a8c12-aa10-4771-a132-24bebc189584 service neutron] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:56:44.766315 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 1109/2210] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 1 16:56:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 16:56:45.040084 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54524f12-9d4f-4bd2-9377-0fcd17afed7b None None] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:56:45.043257 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.fernet_utils [None req-54524f12-9d4f-4bd2-9377-0fcd17afed7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87057) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:56:45.062471 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54524f12-9d4f-4bd2-9377-0fcd17afed7b None None] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:56:45.062562 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12098897-92df-46d4-9c86-7207e1705b8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:56:45.063220 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12098897-92df-46d4-9c86-7207e1705b8c admin admin] REQUEST_METHOD: `DELETE` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:56:45.063646 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12098897-92df-46d4-9c86-7207e1705b8c admin admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:56:45.063750 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12098897-92df-46d4-9c86-7207e1705b8c admin admin] PATH_INFO: `/v3/projects/c6cb6a9491384a3f829fc713bb3bbcaa` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:56:45.064980 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12098897-92df-46d4-9c86-7207e1705b8c admin admin] RBAC: Authorizing `identity:delete_project(project_id=c6cb6a9491384a3f829fc713bb3bbcaa)` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:56:45.067050 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12098897-92df-46d4-9c86-7207e1705b8c admin admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:56:45.233575 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.notifications [None req-12098897-92df-46d4-9c86-7207e1705b8c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c6cb6a9491384a3f829fc713bb3bbcaa', 'request_id': 'req-12098897-92df-46d4-9c86-7207e1705b8c'} {{(pid=87057) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:56:45.241660 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.notifications [None req-12098897-92df-46d4-9c86-7207e1705b8c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c6cb6a9491384a3f829fc713bb3bbcaa', 'request_id': 'req-12098897-92df-46d4-9c86-7207e1705b8c'} {{(pid=87057) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:56:45.249691 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.notifications [None req-12098897-92df-46d4-9c86-7207e1705b8c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c6cb6a9491384a3f829fc713bb3bbcaa', 'request_id': 'req-12098897-92df-46d4-9c86-7207e1705b8c'} {{(pid=87057) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:56:45.300906 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-378a8c12-aa10-4771-a132-24bebc189584 service neutron] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:56:45.303072 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.fernet_utils [None req-378a8c12-aa10-4771-a132-24bebc189584 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:56:45.320706 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.notifications [None req-12098897-92df-46d4-9c86-7207e1705b8c admin admin] The token cache is being invalidate because project c6cb6a9491384a3f829fc713bb3bbcaa was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87057) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 16:56:45.321392 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.notifications [None req-12098897-92df-46d4-9c86-7207e1705b8c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87057) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:56:45.322851 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.notifications [None req-12098897-92df-46d4-9c86-7207e1705b8c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87057) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:56:45.325392 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 1102/2211] 10.222.0.20 () {58 vars in 1233 bytes} [Mon Aug 1 16:56:45 2022] DELETE /identity/v3/projects/c6cb6a9491384a3f829fc713bb3bbcaa => generated 0 bytes in 286 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 16:56:45.331040 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0052ce9-9f0d-4604-bf5d-da8ec099eacd None None] REQUEST_METHOD: `POST` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:56:45.331515 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0052ce9-9f0d-4604-bf5d-da8ec099eacd None None] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:56:45.331894 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-378a8c12-aa10-4771-a132-24bebc189584 service neutron] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:56:45.332245 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0052ce9-9f0d-4604-bf5d-da8ec099eacd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:56:45.370635 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.auth.core [None req-a0052ce9-9f0d-4604-bf5d-da8ec099eacd None None] MFA Rules not processed for user `1007f8e379b74e169b98fb7c3c0ba5b4`. Rule list: `[]` (Enabled: `True`). {{(pid=87057) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:56:45.371864 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.fernet_utils [None req-a0052ce9-9f0d-4604-bf5d-da8ec099eacd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87057) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:56:45.399713 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG dbcounter [-] [87057] Writing DB stats keystone:SELECT=82,keystone:DELETE=16,keystone:INSERT=2 {{(pid=87057) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:56:45.430296 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b56c2f-12df-4944-acd4-9272f6c9b07b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:56:45.431283 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b56c2f-12df-4944-acd4-9272f6c9b07b admin admin] REQUEST_METHOD: `DELETE` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:56:45.432137 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b56c2f-12df-4944-acd4-9272f6c9b07b admin admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:56:45.432416 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b56c2f-12df-4944-acd4-9272f6c9b07b admin admin] PATH_INFO: `/v3/projects/62ba61ebdbd14cea8d409bfb55b75e0a` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:56:45.433610 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28b56c2f-12df-4944-acd4-9272f6c9b07b admin admin] RBAC: Authorizing `identity:delete_project(project_id=62ba61ebdbd14cea8d409bfb55b75e0a)` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:56:45.435603 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28b56c2f-12df-4944-acd4-9272f6c9b07b admin admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:56:45.486540 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 1103/2212] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 1 16:56:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 16:56:45.495849 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29593271-2ae9-4688-ba7b-d42d94e75dfe None None] REQUEST_METHOD: `POST` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:56:45.498061 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29593271-2ae9-4688-ba7b-d42d94e75dfe None None] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:56:45.498359 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29593271-2ae9-4688-ba7b-d42d94e75dfe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:56:45.519895 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.auth.core [None req-29593271-2ae9-4688-ba7b-d42d94e75dfe None None] MFA Rules not processed for user `1007f8e379b74e169b98fb7c3c0ba5b4`. Rule list: `[]` (Enabled: `True`). {{(pid=87057) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:56:45.521395 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.fernet_utils [None req-29593271-2ae9-4688-ba7b-d42d94e75dfe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87057) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:56:45.534883 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 1104/2213] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 1 16:56:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 16:56:45.535999 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29593271-2ae9-4688-ba7b-d42d94e75dfe None None] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:56:45.546427 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29593271-2ae9-4688-ba7b-d42d94e75dfe None None] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:56:45.552230 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65fd4298-a62d-461d-a80a-e0a49178ea2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:56:45.552862 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65fd4298-a62d-461d-a80a-e0a49178ea2a admin admin] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:56:45.556378 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65fd4298-a62d-461d-a80a-e0a49178ea2a admin admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:56:45.556378 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65fd4298-a62d-461d-a80a-e0a49178ea2a admin admin] PATH_INFO: `/v3/domains` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:56:45.556378 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65fd4298-a62d-461d-a80a-e0a49178ea2a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87057) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:56:45.556378 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65fd4298-a62d-461d-a80a-e0a49178ea2a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:56:45.557392 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65fd4298-a62d-461d-a80a-e0a49178ea2a admin admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:56:45.570222 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 1105/2214] 10.222.0.20 () {58 vars in 1153 bytes} [Mon Aug 1 16:56:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:56:45.575295 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65fd4298-a62d-461d-a80a-e0a49178ea2a admin admin] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:56:45.582567 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.notifications [None req-28b56c2f-12df-4944-acd4-9272f6c9b07b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '62ba61ebdbd14cea8d409bfb55b75e0a', 'request_id': 'req-28b56c2f-12df-4944-acd4-9272f6c9b07b'} {{(pid=87058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:56:45.590091 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.notifications [None req-28b56c2f-12df-4944-acd4-9272f6c9b07b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '62ba61ebdbd14cea8d409bfb55b75e0a', 'request_id': 'req-28b56c2f-12df-4944-acd4-9272f6c9b07b'} {{(pid=87058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:56:45.590258 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65fd4298-a62d-461d-a80a-e0a49178ea2a admin admin] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:56:45.598443 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.notifications [None req-28b56c2f-12df-4944-acd4-9272f6c9b07b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '62ba61ebdbd14cea8d409bfb55b75e0a', 'request_id': 'req-28b56c2f-12df-4944-acd4-9272f6c9b07b'} {{(pid=87058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:56:45.673112 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57c906c7-3fdc-4349-8be7-caadd3f07bd8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:56:45.673741 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c906c7-3fdc-4349-8be7-caadd3f07bd8 admin admin] REQUEST_METHOD: `POST` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:56:45.674025 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c906c7-3fdc-4349-8be7-caadd3f07bd8 admin admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:56:45.674331 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c906c7-3fdc-4349-8be7-caadd3f07bd8 admin admin] PATH_INFO: `/v3/projects` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:56:45.675025 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57c906c7-3fdc-4349-8be7-caadd3f07bd8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:56:45.677023 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57c906c7-3fdc-4349-8be7-caadd3f07bd8 admin admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:56:45.722407 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.notifications [None req-28b56c2f-12df-4944-acd4-9272f6c9b07b admin admin] The token cache is being invalidate because project 62ba61ebdbd14cea8d409bfb55b75e0a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87058) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 16:56:45.722943 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.notifications [None req-28b56c2f-12df-4944-acd4-9272f6c9b07b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:56:45.723597 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.notifications [None req-28b56c2f-12df-4944-acd4-9272f6c9b07b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:56:45.725134 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 1110/2215] 10.222.0.20 () {58 vars in 1233 bytes} [Mon Aug 1 16:56:45 2022] DELETE /identity/v3/projects/62ba61ebdbd14cea8d409bfb55b75e0a => generated 0 bytes in 425 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 16:56:45.729226 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b56c2f-12df-4944-acd4-9272f6c9b07b admin admin] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:56:45.730840 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.fernet_utils [None req-28b56c2f-12df-4944-acd4-9272f6c9b07b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:56:45.744004 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b56c2f-12df-4944-acd4-9272f6c9b07b admin admin] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:56:45.749181 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a226c3c-5cbd-4711-86c1-44420840305a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:56:45.750765 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a226c3c-5cbd-4711-86c1-44420840305a admin admin] REQUEST_METHOD: `DELETE` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:56:45.750765 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a226c3c-5cbd-4711-86c1-44420840305a admin admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:56:45.750765 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a226c3c-5cbd-4711-86c1-44420840305a admin admin] PATH_INFO: `/v3/users/5126b0c08930428ab0e0bd3ca1bb3555` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:56:45.751826 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a226c3c-5cbd-4711-86c1-44420840305a admin admin] RBAC: Authorizing `identity:delete_user(user_id=5126b0c08930428ab0e0bd3ca1bb3555)` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:56:45.754207 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a226c3c-5cbd-4711-86c1-44420840305a admin admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:56:45.778064 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG dbcounter [-] [87058] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=87058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:56:45.801107 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 1106/2216] 10.222.0.20 () {60 vars in 1153 bytes} [Mon Aug 1 16:56:45 2022] POST /identity/v3/projects => generated 367 bytes in 227 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 16:56:45.805312 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57c906c7-3fdc-4349-8be7-caadd3f07bd8 admin admin] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:56:45.807005 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.fernet_utils [None req-57c906c7-3fdc-4349-8be7-caadd3f07bd8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87057) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:56:45.883864 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57c906c7-3fdc-4349-8be7-caadd3f07bd8 admin admin] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:56:45.889550 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e70cec4-37e3-4cc4-8121-6e84b69005dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:56:45.890232 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e70cec4-37e3-4cc4-8121-6e84b69005dc admin admin] REQUEST_METHOD: `POST` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:56:45.890511 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e70cec4-37e3-4cc4-8121-6e84b69005dc admin admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:56:45.890892 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e70cec4-37e3-4cc4-8121-6e84b69005dc admin admin] PATH_INFO: `/v3/users` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:56:45.891560 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e70cec4-37e3-4cc4-8121-6e84b69005dc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:56:45.893474 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e70cec4-37e3-4cc4-8121-6e84b69005dc admin admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:56:45.952740 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.notifications [None req-4a226c3c-5cbd-4711-86c1-44420840305a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5126b0c08930428ab0e0bd3ca1bb3555'} {{(pid=87058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:56:45.954224 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.notifications [None req-4a226c3c-5cbd-4711-86c1-44420840305a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5126b0c08930428ab0e0bd3ca1bb3555'} {{(pid=87058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:56:45.955170 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.notifications [None req-4a226c3c-5cbd-4711-86c1-44420840305a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5126b0c08930428ab0e0bd3ca1bb3555'} {{(pid=87058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:56:45.967705 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.notifications [None req-4a226c3c-5cbd-4711-86c1-44420840305a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5126b0c08930428ab0e0bd3ca1bb3555'} {{(pid=87058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:56:45.974570 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.notifications [None req-4a226c3c-5cbd-4711-86c1-44420840305a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5126b0c08930428ab0e0bd3ca1bb3555'} {{(pid=87058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:56:46.005818 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 1107/2217] 10.222.0.20 () {60 vars in 1144 bytes} [Mon Aug 1 16:56:45 2022] POST /identity/v3/users => generated 284 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 16:56:46.009971 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e70cec4-37e3-4cc4-8121-6e84b69005dc admin admin] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:56:46.011924 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.fernet_utils [None req-1e70cec4-37e3-4cc4-8121-6e84b69005dc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87057) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:56:46.024417 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e70cec4-37e3-4cc4-8121-6e84b69005dc admin admin] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:56:46.046750 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d50e392-478b-4478-ab8f-bacd008d7287 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:56:46.046750 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d50e392-478b-4478-ab8f-bacd008d7287 admin admin] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:56:46.046750 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d50e392-478b-4478-ab8f-bacd008d7287 admin admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:56:46.046750 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d50e392-478b-4478-ab8f-bacd008d7287 admin admin] PATH_INFO: `/v3/roles` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:56:46.047294 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d50e392-478b-4478-ab8f-bacd008d7287 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:56:46.047294 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d50e392-478b-4478-ab8f-bacd008d7287 admin admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:56:46.047294 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 1108/2218] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Aug 1 16:56:46 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 16:56:46.049446 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d50e392-478b-4478-ab8f-bacd008d7287 admin admin] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:56:46.060167 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d50e392-478b-4478-ab8f-bacd008d7287 admin admin] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:56:46.064713 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4406881f-3eee-4e63-a5eb-280e1c058791 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:56:46.069520 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4406881f-3eee-4e63-a5eb-280e1c058791 admin admin] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:56:46.069520 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4406881f-3eee-4e63-a5eb-280e1c058791 admin admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:56:46.069520 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4406881f-3eee-4e63-a5eb-280e1c058791 admin admin] PATH_INFO: `/v3/roles` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:56:46.069520 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4406881f-3eee-4e63-a5eb-280e1c058791 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:56:46.069520 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4406881f-3eee-4e63-a5eb-280e1c058791 admin admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:56:46.082231 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 1109/2219] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Aug 1 16:56:46 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 01 16:56:46.084293 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4406881f-3eee-4e63-a5eb-280e1c058791 admin admin] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:56:46.091911 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 1111/2220] 10.222.0.20 () {58 vars in 1224 bytes} [Mon Aug 1 16:56:45 2022] DELETE /identity/v3/users/5126b0c08930428ab0e0bd3ca1bb3555 => generated 0 bytes in 363 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 16:56:46.098791 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4406881f-3eee-4e63-a5eb-280e1c058791 admin admin] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:56:46.157629 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e24bf6c-ade9-4557-8928-237cb51958ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:56:46.158734 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e24bf6c-ade9-4557-8928-237cb51958ed admin admin] REQUEST_METHOD: `PUT` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:56:46.159129 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e24bf6c-ade9-4557-8928-237cb51958ed admin admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:56:46.159317 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e24bf6c-ade9-4557-8928-237cb51958ed admin admin] PATH_INFO: `/v3/projects/dca2fb3173af463392caaf34369ae99e/users/cdb443e00f0b420a8365c0975194f5e2/roles/0999d3da63a545f18ff3252afdc90518` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:56:46.179668 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e24bf6c-ade9-4557-8928-237cb51958ed admin admin] RBAC: Authorizing `identity:create_grant(project_id=dca2fb3173af463392caaf34369ae99e, user_id=cdb443e00f0b420a8365c0975194f5e2, role_id=0999d3da63a545f18ff3252afdc90518)` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:56:46.184714 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e24bf6c-ade9-4557-8928-237cb51958ed admin admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:56:46.266725 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 1110/2221] 10.222.0.20 () {60 vars in 1483 bytes} [Mon Aug 1 16:56:46 2022] PUT /identity/v3/projects/dca2fb3173af463392caaf34369ae99e/users/cdb443e00f0b420a8365c0975194f5e2/roles/0999d3da63a545f18ff3252afdc90518 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 16:56:46.267892 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9652482f-0828-46a9-ab57-4546fea21625 None None] REQUEST_METHOD: `POST` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:56:46.268188 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9652482f-0828-46a9-ab57-4546fea21625 None None] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:56:46.268468 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9652482f-0828-46a9-ab57-4546fea21625 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:56:46.310612 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.auth.core [None req-9652482f-0828-46a9-ab57-4546fea21625 None None] MFA Rules not processed for user `cdb443e00f0b420a8365c0975194f5e2`. Rule list: `[]` (Enabled: `True`). {{(pid=87058) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:56:46.311972 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.fernet_utils [None req-9652482f-0828-46a9-ab57-4546fea21625 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:56:46.343144 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e24bf6c-ade9-4557-8928-237cb51958ed admin admin] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:56:46.344959 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.fernet_utils [None req-4e24bf6c-ade9-4557-8928-237cb51958ed admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87057) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:56:46.402075 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 1112/2222] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 1 16:56:46 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 16:56:46.413262 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9652482f-0828-46a9-ab57-4546fea21625 None None] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:56:46.415588 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.fernet_utils [None req-9652482f-0828-46a9-ab57-4546fea21625 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:56:46.460462 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e24bf6c-ade9-4557-8928-237cb51958ed admin admin] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:56:46.467011 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d74f6781-c838-45c3-a7fd-904de0e79c51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:56:46.467823 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74f6781-c838-45c3-a7fd-904de0e79c51 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:56:46.468309 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74f6781-c838-45c3-a7fd-904de0e79c51 admin admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:56:46.468651 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74f6781-c838-45c3-a7fd-904de0e79c51 admin admin] PATH_INFO: `/v3/users/0ea895229f3f407f80314a8723d90e9d` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:56:46.476472 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d74f6781-c838-45c3-a7fd-904de0e79c51 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0ea895229f3f407f80314a8723d90e9d)` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:56:46.476472 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d74f6781-c838-45c3-a7fd-904de0e79c51 admin admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:56:46.539566 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9652482f-0828-46a9-ab57-4546fea21625 None None] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:56:46.540339 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e6bcde-a141-49c4-9e0b-c15bc975b12f service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd80694e439e2401d9491712666409016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b992d6a65df4bf9b27c13192d57c5f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:56:46.541719 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e6bcde-a141-49c4-9e0b-c15bc975b12f service glance] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:56:46.542026 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e6bcde-a141-49c4-9e0b-c15bc975b12f service glance] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:56:46.542335 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e6bcde-a141-49c4-9e0b-c15bc975b12f service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:56:46.553718 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21e6bcde-a141-49c4-9e0b-c15bc975b12f service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:56:46.555449 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21e6bcde-a141-49c4-9e0b-c15bc975b12f service glance] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:56:46.567369 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 1113/2223] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 1 16:56:46 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 154 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 16:56:46.685330 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.notifications [None req-d74f6781-c838-45c3-a7fd-904de0e79c51 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0ea895229f3f407f80314a8723d90e9d'} {{(pid=87057) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:56:46.687379 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.notifications [None req-d74f6781-c838-45c3-a7fd-904de0e79c51 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0ea895229f3f407f80314a8723d90e9d'} {{(pid=87057) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:56:46.720495 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG dbcounter [-] [87057] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=87057) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:56:46.761248 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.notifications [None req-d74f6781-c838-45c3-a7fd-904de0e79c51 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0ea895229f3f407f80314a8723d90e9d'} {{(pid=87057) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:56:46.761881 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.notifications [None req-d74f6781-c838-45c3-a7fd-904de0e79c51 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0ea895229f3f407f80314a8723d90e9d'} {{(pid=87057) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:56:46.779894 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.notifications [None req-d74f6781-c838-45c3-a7fd-904de0e79c51 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0ea895229f3f407f80314a8723d90e9d'} {{(pid=87057) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:56:46.782054 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e6bcde-a141-49c4-9e0b-c15bc975b12f service glance] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:56:46.784374 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.fernet_utils [None req-21e6bcde-a141-49c4-9e0b-c15bc975b12f service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:56:46.843340 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e6bcde-a141-49c4-9e0b-c15bc975b12f service glance] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:56:46.844841 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 1111/2224] 10.222.0.20 () {58 vars in 1224 bytes} [Mon Aug 1 16:56:46 2022] DELETE /identity/v3/users/0ea895229f3f407f80314a8723d90e9d => generated 0 bytes in 502 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 16:56:46.855248 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ff97f4-ef3a-4e27-87aa-aed3ba0337b6 None None] REQUEST_METHOD: `POST` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:56:46.855618 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ff97f4-ef3a-4e27-87aa-aed3ba0337b6 None None] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:56:46.856158 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ff97f4-ef3a-4e27-87aa-aed3ba0337b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:56:46.883263 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.auth.core [None req-24ff97f4-ef3a-4e27-87aa-aed3ba0337b6 None None] MFA Rules not processed for user `1007f8e379b74e169b98fb7c3c0ba5b4`. Rule list: `[]` (Enabled: `True`). {{(pid=87057) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:56:46.884322 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG dbcounter [-] [87058] Writing DB stats keystone:DELETE=8,keystone:SELECT=91,keystone:INSERT=1 {{(pid=87058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:56:46.886964 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.fernet_utils [None req-24ff97f4-ef3a-4e27-87aa-aed3ba0337b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87057) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:56:46.949615 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09474bed-6a0e-47d9-b813-11e40744bafb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:56:46.950321 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09474bed-6a0e-47d9-b813-11e40744bafb admin admin] REQUEST_METHOD: `DELETE` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:56:46.950600 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09474bed-6a0e-47d9-b813-11e40744bafb admin admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:56:46.950870 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09474bed-6a0e-47d9-b813-11e40744bafb admin admin] PATH_INFO: `/v3/projects/a1978a2f0e23448ba0ecd38d452287fa` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:56:46.952108 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09474bed-6a0e-47d9-b813-11e40744bafb admin admin] RBAC: Authorizing `identity:delete_project(project_id=a1978a2f0e23448ba0ecd38d452287fa)` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:56:46.954133 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09474bed-6a0e-47d9-b813-11e40744bafb admin admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:56:46.998945 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 1112/2225] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 1 16:56:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:56:47.008779 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ff97f4-ef3a-4e27-87aa-aed3ba0337b6 None None] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:56:47.010760 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.fernet_utils [None req-24ff97f4-ef3a-4e27-87aa-aed3ba0337b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87057) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:56:47.058858 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG dbcounter [-] [87057] Writing DB stats keystone:SELECT=89,keystone:INSERT=2,keystone:DELETE=9 {{(pid=87057) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:56:47.074154 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.notifications [None req-09474bed-6a0e-47d9-b813-11e40744bafb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a1978a2f0e23448ba0ecd38d452287fa', 'request_id': 'req-09474bed-6a0e-47d9-b813-11e40744bafb'} {{(pid=87058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:56:47.081341 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.notifications [None req-09474bed-6a0e-47d9-b813-11e40744bafb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a1978a2f0e23448ba0ecd38d452287fa', 'request_id': 'req-09474bed-6a0e-47d9-b813-11e40744bafb'} {{(pid=87058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:56:47.089294 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.notifications [None req-09474bed-6a0e-47d9-b813-11e40744bafb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a1978a2f0e23448ba0ecd38d452287fa', 'request_id': 'req-09474bed-6a0e-47d9-b813-11e40744bafb'} {{(pid=87058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:56:47.102425 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ff97f4-ef3a-4e27-87aa-aed3ba0337b6 None None] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:56:47.155680 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9f772c8-9f9f-4337-a61a-2c3f8b212c47 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d9bd8d649154c979be3744bfd016f95', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b992d6a65df4bf9b27c13192d57c5f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:56:47.156304 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f772c8-9f9f-4337-a61a-2c3f8b212c47 service neutron] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:56:47.156619 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f772c8-9f9f-4337-a61a-2c3f8b212c47 service neutron] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:56:47.156863 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f772c8-9f9f-4337-a61a-2c3f8b212c47 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:56:47.174057 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.notifications [None req-09474bed-6a0e-47d9-b813-11e40744bafb admin admin] The token cache is being invalidate because project a1978a2f0e23448ba0ecd38d452287fa was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87058) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 16:56:47.174454 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.notifications [None req-09474bed-6a0e-47d9-b813-11e40744bafb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:56:47.175579 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.notifications [None req-09474bed-6a0e-47d9-b813-11e40744bafb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:56:47.177509 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 1114/2226] 10.222.0.20 () {58 vars in 1233 bytes} [Mon Aug 1 16:56:46 2022] DELETE /identity/v3/projects/a1978a2f0e23448ba0ecd38d452287fa => generated 0 bytes in 397 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 16:56:47.191124 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09474bed-6a0e-47d9-b813-11e40744bafb admin admin] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:56:47.194310 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.fernet_utils [None req-09474bed-6a0e-47d9-b813-11e40744bafb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:56:47.256037 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9f772c8-9f9f-4337-a61a-2c3f8b212c47 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:56:47.256037 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9f772c8-9f9f-4337-a61a-2c3f8b212c47 service neutron] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:56:47.265900 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 1113/2227] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 1 16:56:47 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 258 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 01 16:56:47.283812 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09474bed-6a0e-47d9-b813-11e40744bafb admin admin] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:56:47.294563 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee639a57-3b46-44b8-a8a1-841fe2e9fd89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:56:47.295645 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee639a57-3b46-44b8-a8a1-841fe2e9fd89 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:56:47.296236 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee639a57-3b46-44b8-a8a1-841fe2e9fd89 admin admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:56:47.296884 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee639a57-3b46-44b8-a8a1-841fe2e9fd89 admin admin] PATH_INFO: `/v3/users/96f87d5e7ebd44c5a0c9bc5fc42b8eab` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:56:47.299954 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee639a57-3b46-44b8-a8a1-841fe2e9fd89 admin admin] RBAC: Authorizing `identity:delete_user(user_id=96f87d5e7ebd44c5a0c9bc5fc42b8eab)` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:56:47.306854 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee639a57-3b46-44b8-a8a1-841fe2e9fd89 admin admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:56:47.515382 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.notifications [None req-ee639a57-3b46-44b8-a8a1-841fe2e9fd89 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '96f87d5e7ebd44c5a0c9bc5fc42b8eab'} {{(pid=87058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:56:47.517965 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.notifications [None req-ee639a57-3b46-44b8-a8a1-841fe2e9fd89 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '96f87d5e7ebd44c5a0c9bc5fc42b8eab'} {{(pid=87058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:56:47.521120 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.notifications [None req-ee639a57-3b46-44b8-a8a1-841fe2e9fd89 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '96f87d5e7ebd44c5a0c9bc5fc42b8eab'} {{(pid=87058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:56:47.564725 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG dbcounter [-] [87058] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=87058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:56:47.569269 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.notifications [None req-ee639a57-3b46-44b8-a8a1-841fe2e9fd89 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '96f87d5e7ebd44c5a0c9bc5fc42b8eab'} {{(pid=87058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:56:47.582105 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.notifications [None req-ee639a57-3b46-44b8-a8a1-841fe2e9fd89 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '96f87d5e7ebd44c5a0c9bc5fc42b8eab'} {{(pid=87058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:56:47.694852 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 1115/2228] 10.222.0.20 () {58 vars in 1224 bytes} [Mon Aug 1 16:56:47 2022] DELETE /identity/v3/users/96f87d5e7ebd44c5a0c9bc5fc42b8eab => generated 0 bytes in 501 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 16:56:47.736584 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG dbcounter [-] [87057] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=87057) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:56:47.874157 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9f772c8-9f9f-4337-a61a-2c3f8b212c47 service neutron] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:56:47.877747 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.fernet_utils [None req-c9f772c8-9f9f-4337-a61a-2c3f8b212c47 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87057) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:56:47.939612 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9f772c8-9f9f-4337-a61a-2c3f8b212c47 service neutron] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:56:48.064427 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b40d6013-4699-4c02-89ab-a6d5a64a0410 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:56:48.065087 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40d6013-4699-4c02-89ab-a6d5a64a0410 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:56:48.065608 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40d6013-4699-4c02-89ab-a6d5a64a0410 admin admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:56:48.065913 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40d6013-4699-4c02-89ab-a6d5a64a0410 admin admin] PATH_INFO: `/v3/projects/f94c60f534f840c7a0459b24c85a1b93` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:56:48.067394 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b40d6013-4699-4c02-89ab-a6d5a64a0410 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f94c60f534f840c7a0459b24c85a1b93)` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:56:48.069411 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b40d6013-4699-4c02-89ab-a6d5a64a0410 admin admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:56:48.207980 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.notifications [None req-b40d6013-4699-4c02-89ab-a6d5a64a0410 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f94c60f534f840c7a0459b24c85a1b93', 'request_id': 'req-b40d6013-4699-4c02-89ab-a6d5a64a0410'} {{(pid=87057) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:56:48.215990 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.notifications [None req-b40d6013-4699-4c02-89ab-a6d5a64a0410 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f94c60f534f840c7a0459b24c85a1b93', 'request_id': 'req-b40d6013-4699-4c02-89ab-a6d5a64a0410'} {{(pid=87057) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:56:48.221765 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.notifications [None req-b40d6013-4699-4c02-89ab-a6d5a64a0410 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f94c60f534f840c7a0459b24c85a1b93', 'request_id': 'req-b40d6013-4699-4c02-89ab-a6d5a64a0410'} {{(pid=87057) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:56:48.284482 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.notifications [None req-b40d6013-4699-4c02-89ab-a6d5a64a0410 admin admin] The token cache is being invalidate because project f94c60f534f840c7a0459b24c85a1b93 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87057) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 16:56:48.284756 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.notifications [None req-b40d6013-4699-4c02-89ab-a6d5a64a0410 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87057) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:56:48.285415 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.notifications [None req-b40d6013-4699-4c02-89ab-a6d5a64a0410 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87057) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:56:48.287059 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 1114/2229] 10.222.0.20 () {58 vars in 1233 bytes} [Mon Aug 1 16:56:47 2022] DELETE /identity/v3/projects/f94c60f534f840c7a0459b24c85a1b93 => generated 0 bytes in 414 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 16:56:48.291705 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee639a57-3b46-44b8-a8a1-841fe2e9fd89 admin admin] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:56:48.294698 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.fernet_utils [None req-ee639a57-3b46-44b8-a8a1-841fe2e9fd89 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:56:48.316933 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee639a57-3b46-44b8-a8a1-841fe2e9fd89 admin admin] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:56:48.383278 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b40d6013-4699-4c02-89ab-a6d5a64a0410 admin admin] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:56:48.385137 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.fernet_utils [None req-b40d6013-4699-4c02-89ab-a6d5a64a0410 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87057) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:56:48.399625 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ef3d22-f0a1-4306-a8a0-ac797947fa06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:56:48.401099 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ef3d22-f0a1-4306-a8a0-ac797947fa06 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:56:48.401669 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ef3d22-f0a1-4306-a8a0-ac797947fa06 admin admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:56:48.402075 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ef3d22-f0a1-4306-a8a0-ac797947fa06 admin admin] PATH_INFO: `/v3/users/59641a4bc54243ac8632560ef6e24bbe` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:56:48.404396 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66ef3d22-f0a1-4306-a8a0-ac797947fa06 admin admin] RBAC: Authorizing `identity:delete_user(user_id=59641a4bc54243ac8632560ef6e24bbe)` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:56:48.407312 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66ef3d22-f0a1-4306-a8a0-ac797947fa06 admin admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:56:48.530742 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b40d6013-4699-4c02-89ab-a6d5a64a0410 admin admin] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:56:48.535712 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e0a59b-127c-4546-b022-2ac3936dcbff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:56:48.536375 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e0a59b-127c-4546-b022-2ac3936dcbff admin admin] REQUEST_METHOD: `DELETE` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:56:48.536655 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e0a59b-127c-4546-b022-2ac3936dcbff admin admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:56:48.536930 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e0a59b-127c-4546-b022-2ac3936dcbff admin admin] PATH_INFO: `/v3/projects/835b98983652463f8a9e806bc98a0a6f` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:56:48.538533 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5e0a59b-127c-4546-b022-2ac3936dcbff admin admin] RBAC: Authorizing `identity:delete_project(project_id=835b98983652463f8a9e806bc98a0a6f)` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:56:48.540404 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5e0a59b-127c-4546-b022-2ac3936dcbff admin admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:56:48.638268 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.notifications [None req-66ef3d22-f0a1-4306-a8a0-ac797947fa06 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '59641a4bc54243ac8632560ef6e24bbe'} {{(pid=87058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:56:48.639485 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.notifications [None req-66ef3d22-f0a1-4306-a8a0-ac797947fa06 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '59641a4bc54243ac8632560ef6e24bbe'} {{(pid=87058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:56:48.640218 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.notifications [None req-66ef3d22-f0a1-4306-a8a0-ac797947fa06 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '59641a4bc54243ac8632560ef6e24bbe'} {{(pid=87058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:56:48.662996 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.notifications [None req-66ef3d22-f0a1-4306-a8a0-ac797947fa06 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '59641a4bc54243ac8632560ef6e24bbe'} {{(pid=87058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:56:48.672596 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.notifications [None req-66ef3d22-f0a1-4306-a8a0-ac797947fa06 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '59641a4bc54243ac8632560ef6e24bbe'} {{(pid=87058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:56:48.690765 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG dbcounter [-] [87057] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=87057) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:56:48.703545 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.notifications [None req-f5e0a59b-127c-4546-b022-2ac3936dcbff admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '835b98983652463f8a9e806bc98a0a6f', 'request_id': 'req-f5e0a59b-127c-4546-b022-2ac3936dcbff'} {{(pid=87057) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:56:48.710308 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.notifications [None req-f5e0a59b-127c-4546-b022-2ac3936dcbff admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '835b98983652463f8a9e806bc98a0a6f', 'request_id': 'req-f5e0a59b-127c-4546-b022-2ac3936dcbff'} {{(pid=87057) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:56:48.715883 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.notifications [None req-f5e0a59b-127c-4546-b022-2ac3936dcbff admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '835b98983652463f8a9e806bc98a0a6f', 'request_id': 'req-f5e0a59b-127c-4546-b022-2ac3936dcbff'} {{(pid=87057) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:56:48.794938 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 1116/2230] 10.222.0.20 () {58 vars in 1224 bytes} [Mon Aug 1 16:56:48 2022] DELETE /identity/v3/users/59641a4bc54243ac8632560ef6e24bbe => generated 0 bytes in 504 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 16:56:48.879434 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.notifications [None req-f5e0a59b-127c-4546-b022-2ac3936dcbff admin admin] The token cache is being invalidate because project 835b98983652463f8a9e806bc98a0a6f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87057) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 16:56:48.880297 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.notifications [None req-f5e0a59b-127c-4546-b022-2ac3936dcbff admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87057) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:56:48.881091 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.notifications [None req-f5e0a59b-127c-4546-b022-2ac3936dcbff admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87057) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:56:48.884244 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 1115/2231] 10.222.0.20 () {58 vars in 1233 bytes} [Mon Aug 1 16:56:48 2022] DELETE /identity/v3/projects/835b98983652463f8a9e806bc98a0a6f => generated 0 bytes in 500 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 16:56:48.893632 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228abcbf-d55b-4688-82ba-dbd396037624 None None] REQUEST_METHOD: `POST` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:56:48.893936 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228abcbf-d55b-4688-82ba-dbd396037624 None None] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:56:48.894246 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228abcbf-d55b-4688-82ba-dbd396037624 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:56:48.931218 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.auth.core [None req-228abcbf-d55b-4688-82ba-dbd396037624 None None] MFA Rules not processed for user `1007f8e379b74e169b98fb7c3c0ba5b4`. Rule list: `[]` (Enabled: `True`). {{(pid=87058) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:56:48.932703 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.fernet_utils [None req-228abcbf-d55b-4688-82ba-dbd396037624 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:56:48.996977 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG dbcounter [-] [87058] Writing DB stats keystone:DELETE=12,keystone:SELECT=86,keystone:INSERT=2 {{(pid=87058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:56:49.007029 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 1117/2232] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 1 16:56:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:56:49.015726 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2f90cc-03da-4377-8c02-b567e350f65b None None] REQUEST_METHOD: `POST` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:56:49.016000 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2f90cc-03da-4377-8c02-b567e350f65b None None] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:56:49.019119 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2f90cc-03da-4377-8c02-b567e350f65b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:56:49.047665 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.auth.core [None req-9a2f90cc-03da-4377-8c02-b567e350f65b None None] MFA Rules not processed for user `1007f8e379b74e169b98fb7c3c0ba5b4`. Rule list: `[]` (Enabled: `True`). {{(pid=87057) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:56:49.047665 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.fernet_utils [None req-9a2f90cc-03da-4377-8c02-b567e350f65b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87057) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:56:49.057677 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 1116/2233] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 1 16:56:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:56:49.063368 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-228abcbf-d55b-4688-82ba-dbd396037624 None None] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:56:49.076634 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-228abcbf-d55b-4688-82ba-dbd396037624 None None] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:56:49.083361 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ade84b3-558d-4595-a43c-f0560f1edcd8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:56:49.083962 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ade84b3-558d-4595-a43c-f0560f1edcd8 admin admin] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:56:49.084272 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ade84b3-558d-4595-a43c-f0560f1edcd8 admin admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:56:49.084613 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ade84b3-558d-4595-a43c-f0560f1edcd8 admin admin] PATH_INFO: `/v3/domains` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:56:49.085585 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ade84b3-558d-4595-a43c-f0560f1edcd8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87058) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:56:49.086604 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ade84b3-558d-4595-a43c-f0560f1edcd8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:56:49.091318 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ade84b3-558d-4595-a43c-f0560f1edcd8 admin admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:56:49.118316 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 1118/2234] 10.222.0.20 () {58 vars in 1153 bytes} [Mon Aug 1 16:56:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:56:49.119316 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a2f90cc-03da-4377-8c02-b567e350f65b None None] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:56:49.131882 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a2f90cc-03da-4377-8c02-b567e350f65b None None] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:56:49.146271 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba74963-83bb-4140-af2d-e210f3d80746 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:56:49.146795 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba74963-83bb-4140-af2d-e210f3d80746 admin admin] REQUEST_METHOD: `POST` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:56:49.148762 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba74963-83bb-4140-af2d-e210f3d80746 admin admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:56:49.148762 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba74963-83bb-4140-af2d-e210f3d80746 admin admin] PATH_INFO: `/v3/projects` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:56:49.149491 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ba74963-83bb-4140-af2d-e210f3d80746 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:56:49.152159 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ba74963-83bb-4140-af2d-e210f3d80746 admin admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:56:49.331298 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 1117/2235] 10.222.0.20 () {60 vars in 1153 bytes} [Mon Aug 1 16:56:49 2022] POST /identity/v3/projects => generated 375 bytes in 213 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 16:56:49.336093 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ade84b3-558d-4595-a43c-f0560f1edcd8 admin admin] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:56:49.350313 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ade84b3-558d-4595-a43c-f0560f1edcd8 admin admin] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:56:49.351337 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba74963-83bb-4140-af2d-e210f3d80746 admin admin] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:56:49.354133 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.fernet_utils [None req-0ba74963-83bb-4140-af2d-e210f3d80746 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87057) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:56:49.460231 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba74963-83bb-4140-af2d-e210f3d80746 admin admin] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:56:49.465699 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2c356ab-04f0-4285-b298-5a712de866a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:56:49.466437 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c356ab-04f0-4285-b298-5a712de866a4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:56:49.466718 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c356ab-04f0-4285-b298-5a712de866a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:56:49.469627 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c356ab-04f0-4285-b298-5a712de866a4 admin admin] PATH_INFO: `/v3/projects/80ce9192be4849e1ae25548e57454fc9` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:56:49.469627 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2c356ab-04f0-4285-b298-5a712de866a4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=80ce9192be4849e1ae25548e57454fc9)` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:56:49.471669 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2c356ab-04f0-4285-b298-5a712de866a4 admin admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:56:49.480309 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bf4284c-b48d-4ba8-b2b5-41ab529386b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:56:49.481451 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf4284c-b48d-4ba8-b2b5-41ab529386b0 admin admin] REQUEST_METHOD: `POST` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:56:49.481904 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf4284c-b48d-4ba8-b2b5-41ab529386b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:56:49.482584 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf4284c-b48d-4ba8-b2b5-41ab529386b0 admin admin] PATH_INFO: `/v3/users` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:56:49.483941 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bf4284c-b48d-4ba8-b2b5-41ab529386b0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:56:49.486442 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bf4284c-b48d-4ba8-b2b5-41ab529386b0 admin admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:56:49.594335 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 1119/2236] 10.222.0.20 () {60 vars in 1144 bytes} [Mon Aug 1 16:56:49 2022] POST /identity/v3/users => generated 288 bytes in 259 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 16:56:49.595389 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bf4284c-b48d-4ba8-b2b5-41ab529386b0 admin admin] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:56:49.610321 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.notifications [None req-b2c356ab-04f0-4285-b298-5a712de866a4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '80ce9192be4849e1ae25548e57454fc9', 'request_id': 'req-b2c356ab-04f0-4285-b298-5a712de866a4'} {{(pid=87057) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:56:49.610561 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bf4284c-b48d-4ba8-b2b5-41ab529386b0 admin admin] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:56:49.621953 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.notifications [None req-b2c356ab-04f0-4285-b298-5a712de866a4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '80ce9192be4849e1ae25548e57454fc9', 'request_id': 'req-b2c356ab-04f0-4285-b298-5a712de866a4'} {{(pid=87057) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:56:49.637851 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.notifications [None req-b2c356ab-04f0-4285-b298-5a712de866a4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '80ce9192be4849e1ae25548e57454fc9', 'request_id': 'req-b2c356ab-04f0-4285-b298-5a712de866a4'} {{(pid=87057) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:56:49.708910 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ec220c1-8f01-481e-a38c-4b151e074b76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:56:49.709327 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec220c1-8f01-481e-a38c-4b151e074b76 admin admin] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:56:49.709841 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec220c1-8f01-481e-a38c-4b151e074b76 admin admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:56:49.710150 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec220c1-8f01-481e-a38c-4b151e074b76 admin admin] PATH_INFO: `/v3/roles` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:56:49.710876 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ec220c1-8f01-481e-a38c-4b151e074b76 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:56:49.712561 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ec220c1-8f01-481e-a38c-4b151e074b76 admin admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:56:49.733490 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.notifications [None req-b2c356ab-04f0-4285-b298-5a712de866a4 admin admin] The token cache is being invalidate because project 80ce9192be4849e1ae25548e57454fc9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87057) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 16:56:49.733490 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.notifications [None req-b2c356ab-04f0-4285-b298-5a712de866a4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87057) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:56:49.733813 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.notifications [None req-b2c356ab-04f0-4285-b298-5a712de866a4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87057) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:56:49.735465 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 1118/2237] 10.222.0.20 () {58 vars in 1233 bytes} [Mon Aug 1 16:56:49 2022] DELETE /identity/v3/projects/80ce9192be4849e1ae25548e57454fc9 => generated 0 bytes in 385 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 16:56:49.739721 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2c356ab-04f0-4285-b298-5a712de866a4 admin admin] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:56:49.743504 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 1120/2238] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Aug 1 16:56:49 2022] GET /identity/v3/roles => generated 1420 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 16:56:49.745022 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.fernet_utils [None req-b2c356ab-04f0-4285-b298-5a712de866a4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87057) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:56:49.747199 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ec220c1-8f01-481e-a38c-4b151e074b76 admin admin] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:56:49.748987 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.fernet_utils [None req-6ec220c1-8f01-481e-a38c-4b151e074b76 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:56:49.768438 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2c356ab-04f0-4285-b298-5a712de866a4 admin admin] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:56:49.773729 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ec220c1-8f01-481e-a38c-4b151e074b76 admin admin] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:56:49.779234 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac0d6d5-e3d1-40cb-a0f8-7b926fe4abc9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:56:49.780038 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac0d6d5-e3d1-40cb-a0f8-7b926fe4abc9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:56:49.780382 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac0d6d5-e3d1-40cb-a0f8-7b926fe4abc9 admin admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:56:49.780712 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac0d6d5-e3d1-40cb-a0f8-7b926fe4abc9 admin admin] PATH_INFO: `/v3/users/151b09136bba4bf6bdc0772a2df571e7` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:56:49.781976 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ac0d6d5-e3d1-40cb-a0f8-7b926fe4abc9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=151b09136bba4bf6bdc0772a2df571e7)` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:56:49.783932 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ac0d6d5-e3d1-40cb-a0f8-7b926fe4abc9 admin admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:56:49.788199 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d969f421-c70e-40ab-8f4e-9a742207245c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:56:49.789402 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d969f421-c70e-40ab-8f4e-9a742207245c admin admin] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:56:49.789947 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d969f421-c70e-40ab-8f4e-9a742207245c admin admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:56:49.791908 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d969f421-c70e-40ab-8f4e-9a742207245c admin admin] PATH_INFO: `/v3/roles` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:56:49.792822 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d969f421-c70e-40ab-8f4e-9a742207245c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:56:49.794997 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d969f421-c70e-40ab-8f4e-9a742207245c admin admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:56:49.802524 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG dbcounter [-] [87057] Writing DB stats keystone:SELECT=84,keystone:DELETE=13,keystone:INSERT=3 {{(pid=87057) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:56:49.810389 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 1121/2239] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Aug 1 16:56:49 2022] GET /identity/v3/roles => generated 1420 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 16:56:49.814910 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d969f421-c70e-40ab-8f4e-9a742207245c admin admin] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:56:49.826274 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d969f421-c70e-40ab-8f4e-9a742207245c admin admin] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:56:49.831153 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f35a153d-6187-4612-adcd-a799875a5061 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:56:49.831856 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35a153d-6187-4612-adcd-a799875a5061 admin admin] REQUEST_METHOD: `PUT` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:56:49.832164 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35a153d-6187-4612-adcd-a799875a5061 admin admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:56:49.832437 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35a153d-6187-4612-adcd-a799875a5061 admin admin] PATH_INFO: `/v3/projects/c78792a8e5624576a1d5ecde8c2e0767/users/f4c5dccbaef740549fa5e56a7032aa8e/roles/0999d3da63a545f18ff3252afdc90518` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:56:49.847740 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f35a153d-6187-4612-adcd-a799875a5061 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c78792a8e5624576a1d5ecde8c2e0767, user_id=f4c5dccbaef740549fa5e56a7032aa8e, role_id=0999d3da63a545f18ff3252afdc90518)` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:56:49.850423 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f35a153d-6187-4612-adcd-a799875a5061 admin admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:56:49.962080 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 1122/2240] 10.222.0.20 () {60 vars in 1483 bytes} [Mon Aug 1 16:56:49 2022] PUT /identity/v3/projects/c78792a8e5624576a1d5ecde8c2e0767/users/f4c5dccbaef740549fa5e56a7032aa8e/roles/0999d3da63a545f18ff3252afdc90518 => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 16:56:49.973771 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479770c2-b4c2-46ac-b37c-ecae52478c10 None None] REQUEST_METHOD: `POST` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:56:49.974120 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479770c2-b4c2-46ac-b37c-ecae52478c10 None None] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:56:49.974333 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479770c2-b4c2-46ac-b37c-ecae52478c10 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:56:49.985444 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.notifications [None req-8ac0d6d5-e3d1-40cb-a0f8-7b926fe4abc9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '151b09136bba4bf6bdc0772a2df571e7'} {{(pid=87057) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:56:49.987384 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.notifications [None req-8ac0d6d5-e3d1-40cb-a0f8-7b926fe4abc9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '151b09136bba4bf6bdc0772a2df571e7'} {{(pid=87057) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:56:50.029118 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.auth.core [None req-479770c2-b4c2-46ac-b37c-ecae52478c10 None None] MFA Rules not processed for user `f4c5dccbaef740549fa5e56a7032aa8e`. Rule list: `[]` (Enabled: `True`). {{(pid=87058) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:56:50.030622 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.fernet_utils [None req-479770c2-b4c2-46ac-b37c-ecae52478c10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:56:50.059395 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.notifications [None req-8ac0d6d5-e3d1-40cb-a0f8-7b926fe4abc9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '151b09136bba4bf6bdc0772a2df571e7'} {{(pid=87057) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:56:50.061827 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.notifications [None req-8ac0d6d5-e3d1-40cb-a0f8-7b926fe4abc9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '151b09136bba4bf6bdc0772a2df571e7'} {{(pid=87057) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:56:50.079379 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.notifications [None req-8ac0d6d5-e3d1-40cb-a0f8-7b926fe4abc9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '151b09136bba4bf6bdc0772a2df571e7'} {{(pid=87057) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:56:50.102194 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 1119/2241] 10.222.0.20 () {58 vars in 1224 bytes} [Mon Aug 1 16:56:49 2022] DELETE /identity/v3/users/151b09136bba4bf6bdc0772a2df571e7 => generated 0 bytes in 363 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 16:56:50.129857 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 1123/2242] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 1 16:56:49 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 16:56:50.140846 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac0d6d5-e3d1-40cb-a0f8-7b926fe4abc9 admin admin] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:56:50.142661 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.fernet_utils [None req-8ac0d6d5-e3d1-40cb-a0f8-7b926fe4abc9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87057) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:56:50.154565 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG dbcounter [-] [87058] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=87058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:56:50.231507 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac0d6d5-e3d1-40cb-a0f8-7b926fe4abc9 admin admin] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:56:50.236482 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd990517-2cf0-4990-9b90-1ada1128dcda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:56:50.237107 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd990517-2cf0-4990-9b90-1ada1128dcda admin admin] REQUEST_METHOD: `POST` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:56:50.237391 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd990517-2cf0-4990-9b90-1ada1128dcda admin admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:56:50.238868 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd990517-2cf0-4990-9b90-1ada1128dcda admin admin] PATH_INFO: `/v3/projects` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:56:50.238868 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd990517-2cf0-4990-9b90-1ada1128dcda admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:56:50.240336 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd990517-2cf0-4990-9b90-1ada1128dcda admin admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:56:50.265690 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-479770c2-b4c2-46ac-b37c-ecae52478c10 None None] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:56:50.266995 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.fernet_utils [None req-479770c2-b4c2-46ac-b37c-ecae52478c10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:56:50.359443 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 1120/2243] 10.222.0.20 () {60 vars in 1153 bytes} [Mon Aug 1 16:56:50 2022] POST /identity/v3/projects => generated 375 bytes in 213 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 16:56:50.359443 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd990517-2cf0-4990-9b90-1ada1128dcda admin admin] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:56:50.359443 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.fernet_utils [None req-cd990517-2cf0-4990-9b90-1ada1128dcda admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87057) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:56:50.359934 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-479770c2-b4c2-46ac-b37c-ecae52478c10 None None] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:56:50.431970 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd112b05-2d6d-4e42-bddf-5cf36889e7b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:56:50.436228 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd112b05-2d6d-4e42-bddf-5cf36889e7b1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:56:50.436228 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd112b05-2d6d-4e42-bddf-5cf36889e7b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:56:50.436228 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd112b05-2d6d-4e42-bddf-5cf36889e7b1 admin admin] PATH_INFO: `/v3/users/0d90db97f4334c41ba8e0e54fc14c317` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:56:50.438346 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd990517-2cf0-4990-9b90-1ada1128dcda admin admin] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:56:50.443767 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd112b05-2d6d-4e42-bddf-5cf36889e7b1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0d90db97f4334c41ba8e0e54fc14c317)` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:56:50.446524 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd112b05-2d6d-4e42-bddf-5cf36889e7b1 admin admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:56:50.450651 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae024350-3256-4330-b224-ecf8785ea1bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:56:50.452238 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae024350-3256-4330-b224-ecf8785ea1bf admin admin] REQUEST_METHOD: `DELETE` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:56:50.453664 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae024350-3256-4330-b224-ecf8785ea1bf admin admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:56:50.453952 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae024350-3256-4330-b224-ecf8785ea1bf admin admin] PATH_INFO: `/v3/users/7a72dec36306426cb9a8548948030ca0` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:56:50.455513 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae024350-3256-4330-b224-ecf8785ea1bf admin admin] RBAC: Authorizing `identity:delete_user(user_id=7a72dec36306426cb9a8548948030ca0)` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:56:50.458475 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae024350-3256-4330-b224-ecf8785ea1bf admin admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:56:50.648453 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.notifications [None req-fd112b05-2d6d-4e42-bddf-5cf36889e7b1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0d90db97f4334c41ba8e0e54fc14c317'} {{(pid=87058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:56:50.648874 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.notifications [None req-ae024350-3256-4330-b224-ecf8785ea1bf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7a72dec36306426cb9a8548948030ca0'} {{(pid=87057) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:56:50.649150 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.notifications [None req-fd112b05-2d6d-4e42-bddf-5cf36889e7b1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0d90db97f4334c41ba8e0e54fc14c317'} {{(pid=87058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:56:50.649702 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.notifications [None req-ae024350-3256-4330-b224-ecf8785ea1bf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7a72dec36306426cb9a8548948030ca0'} {{(pid=87057) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:56:50.649813 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.notifications [None req-fd112b05-2d6d-4e42-bddf-5cf36889e7b1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0d90db97f4334c41ba8e0e54fc14c317'} {{(pid=87058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:56:50.662304 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.notifications [None req-fd112b05-2d6d-4e42-bddf-5cf36889e7b1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0d90db97f4334c41ba8e0e54fc14c317'} {{(pid=87058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:56:50.669652 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.notifications [None req-fd112b05-2d6d-4e42-bddf-5cf36889e7b1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0d90db97f4334c41ba8e0e54fc14c317'} {{(pid=87058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:56:50.680043 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG dbcounter [-] [87058] Writing DB stats keystone:SELECT=91,keystone:DELETE=9 {{(pid=87058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:56:50.702420 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.notifications [None req-ae024350-3256-4330-b224-ecf8785ea1bf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7a72dec36306426cb9a8548948030ca0'} {{(pid=87057) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:56:50.703199 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.notifications [None req-ae024350-3256-4330-b224-ecf8785ea1bf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7a72dec36306426cb9a8548948030ca0'} {{(pid=87057) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:56:50.710645 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.notifications [None req-ae024350-3256-4330-b224-ecf8785ea1bf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7a72dec36306426cb9a8548948030ca0'} {{(pid=87057) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:56:50.719939 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG dbcounter [-] [87057] Writing DB stats keystone:SELECT=87,keystone:DELETE=11,keystone:INSERT=2 {{(pid=87057) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:56:50.729374 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 1121/2244] 10.222.0.20 () {58 vars in 1224 bytes} [Mon Aug 1 16:56:50 2022] DELETE /identity/v3/users/7a72dec36306426cb9a8548948030ca0 => generated 0 bytes in 375 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 16:56:50.730250 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae024350-3256-4330-b224-ecf8785ea1bf admin admin] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:56:50.731833 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.fernet_utils [None req-ae024350-3256-4330-b224-ecf8785ea1bf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87057) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:56:50.746233 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 1124/2245] 10.222.0.20 () {58 vars in 1224 bytes} [Mon Aug 1 16:56:50 2022] DELETE /identity/v3/users/0d90db97f4334c41ba8e0e54fc14c317 => generated 0 bytes in 480 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 16:56:50.746233 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd112b05-2d6d-4e42-bddf-5cf36889e7b1 admin admin] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:56:50.747452 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.fernet_utils [None req-fd112b05-2d6d-4e42-bddf-5cf36889e7b1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:56:50.813571 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae024350-3256-4330-b224-ecf8785ea1bf admin admin] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:56:50.848920 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd112b05-2d6d-4e42-bddf-5cf36889e7b1 admin admin] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:56:50.855852 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-168c4848-2a22-416a-a2cd-06cee08c82b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:56:50.856518 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168c4848-2a22-416a-a2cd-06cee08c82b9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:56:50.856856 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168c4848-2a22-416a-a2cd-06cee08c82b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:56:50.857131 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168c4848-2a22-416a-a2cd-06cee08c82b9 admin admin] PATH_INFO: `/v3/projects/74bafa2382ed4e988c211442bebec6ab` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:56:50.858985 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-168c4848-2a22-416a-a2cd-06cee08c82b9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=74bafa2382ed4e988c211442bebec6ab)` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:56:50.860997 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-168c4848-2a22-416a-a2cd-06cee08c82b9 admin admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:56:50.903938 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a456977-2697-4d41-8077-8cc5b6affdcf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:56:50.903938 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a456977-2697-4d41-8077-8cc5b6affdcf admin admin] REQUEST_METHOD: `POST` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:56:50.903938 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a456977-2697-4d41-8077-8cc5b6affdcf admin admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:56:50.903938 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a456977-2697-4d41-8077-8cc5b6affdcf admin admin] PATH_INFO: `/v3/users` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:56:50.904425 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a456977-2697-4d41-8077-8cc5b6affdcf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:56:50.904425 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a456977-2697-4d41-8077-8cc5b6affdcf admin admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:56:50.989261 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.notifications [None req-168c4848-2a22-416a-a2cd-06cee08c82b9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '74bafa2382ed4e988c211442bebec6ab', 'request_id': 'req-168c4848-2a22-416a-a2cd-06cee08c82b9'} {{(pid=87058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:56:50.996480 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 1122/2246] 10.222.0.20 () {60 vars in 1144 bytes} [Mon Aug 1 16:56:50 2022] POST /identity/v3/users => generated 294 bytes in 267 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 16:56:50.997915 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa9d79a-8b55-43bd-82a1-cd699ce3618a None None] REQUEST_METHOD: `POST` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:56:50.998177 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa9d79a-8b55-43bd-82a1-cd699ce3618a None None] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:56:50.998451 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa9d79a-8b55-43bd-82a1-cd699ce3618a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:56:50.999641 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.notifications [None req-168c4848-2a22-416a-a2cd-06cee08c82b9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '74bafa2382ed4e988c211442bebec6ab', 'request_id': 'req-168c4848-2a22-416a-a2cd-06cee08c82b9'} {{(pid=87058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:56:51.005527 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.notifications [None req-168c4848-2a22-416a-a2cd-06cee08c82b9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '74bafa2382ed4e988c211442bebec6ab', 'request_id': 'req-168c4848-2a22-416a-a2cd-06cee08c82b9'} {{(pid=87058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:56:51.018462 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.auth.core [None req-efa9d79a-8b55-43bd-82a1-cd699ce3618a None None] MFA Rules not processed for user `1007f8e379b74e169b98fb7c3c0ba5b4`. Rule list: `[]` (Enabled: `True`). {{(pid=87057) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:56:51.019706 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.fernet_utils [None req-efa9d79a-8b55-43bd-82a1-cd699ce3618a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87057) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:56:51.076268 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.notifications [None req-168c4848-2a22-416a-a2cd-06cee08c82b9 admin admin] The token cache is being invalidate because project 74bafa2382ed4e988c211442bebec6ab was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87058) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 16:56:51.077746 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.notifications [None req-168c4848-2a22-416a-a2cd-06cee08c82b9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:56:51.078246 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.notifications [None req-168c4848-2a22-416a-a2cd-06cee08c82b9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:56:51.080020 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 1125/2247] 10.222.0.20 () {58 vars in 1233 bytes} [Mon Aug 1 16:56:50 2022] DELETE /identity/v3/projects/74bafa2382ed4e988c211442bebec6ab => generated 0 bytes in 334 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 16:56:51.081326 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-168c4848-2a22-416a-a2cd-06cee08c82b9 admin admin] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:56:51.090678 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.fernet_utils [None req-168c4848-2a22-416a-a2cd-06cee08c82b9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:56:51.098288 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 1123/2248] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 1 16:56:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:56:51.099483 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efa9d79a-8b55-43bd-82a1-cd699ce3618a None None] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:56:51.102224 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.fernet_utils [None req-efa9d79a-8b55-43bd-82a1-cd699ce3618a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87057) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:56:51.107275 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-168c4848-2a22-416a-a2cd-06cee08c82b9 admin admin] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:56:51.113361 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e55b0cb-efeb-4ef3-8052-54230cf63785 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:56:51.113889 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e55b0cb-efeb-4ef3-8052-54230cf63785 admin admin] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:56:51.114206 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e55b0cb-efeb-4ef3-8052-54230cf63785 admin admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:56:51.114478 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e55b0cb-efeb-4ef3-8052-54230cf63785 admin admin] PATH_INFO: `/v3/roles` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:56:51.115184 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e55b0cb-efeb-4ef3-8052-54230cf63785 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:56:51.116788 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e55b0cb-efeb-4ef3-8052-54230cf63785 admin admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:56:51.118112 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efa9d79a-8b55-43bd-82a1-cd699ce3618a None None] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:56:51.129924 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 1126/2249] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Aug 1 16:56:51 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 16:56:51.130744 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2953ec1-378d-4109-b8f3-cf6068a39d23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:56:51.131219 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4469c26-49f8-4cfe-86f0-51cadc0d867c None None] REQUEST_METHOD: `POST` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:56:51.131514 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4469c26-49f8-4cfe-86f0-51cadc0d867c None None] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:56:51.131728 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2953ec1-378d-4109-b8f3-cf6068a39d23 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:56:51.131848 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4469c26-49f8-4cfe-86f0-51cadc0d867c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:56:51.132168 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2953ec1-378d-4109-b8f3-cf6068a39d23 admin admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:56:51.132798 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2953ec1-378d-4109-b8f3-cf6068a39d23 admin admin] PATH_INFO: `/v3/projects/704a245a8de14350b5ffe3d1d13ca1b2` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:56:51.134948 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2953ec1-378d-4109-b8f3-cf6068a39d23 admin admin] RBAC: Authorizing `identity:delete_project(project_id=704a245a8de14350b5ffe3d1d13ca1b2)` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:56:51.137126 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2953ec1-378d-4109-b8f3-cf6068a39d23 admin admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:56:51.154993 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.auth.core [None req-e4469c26-49f8-4cfe-86f0-51cadc0d867c None None] MFA Rules not processed for user `1007f8e379b74e169b98fb7c3c0ba5b4`. Rule list: `[]` (Enabled: `True`). {{(pid=87058) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:56:51.156226 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.fernet_utils [None req-e4469c26-49f8-4cfe-86f0-51cadc0d867c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:56:51.165438 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 1127/2250] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 1 16:56:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:56:51.167164 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4469c26-49f8-4cfe-86f0-51cadc0d867c None None] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:56:51.168736 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.fernet_utils [None req-e4469c26-49f8-4cfe-86f0-51cadc0d867c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:56:51.230460 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4469c26-49f8-4cfe-86f0-51cadc0d867c None None] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:56:51.234179 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65e92960-f724-4456-9a71-b55434b3b9eb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d9bd8d649154c979be3744bfd016f95', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b992d6a65df4bf9b27c13192d57c5f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:56:51.234800 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e92960-f724-4456-9a71-b55434b3b9eb service neutron] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:56:51.235080 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e92960-f724-4456-9a71-b55434b3b9eb service neutron] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:56:51.235352 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e92960-f724-4456-9a71-b55434b3b9eb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:56:51.245662 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65e92960-f724-4456-9a71-b55434b3b9eb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:56:51.247128 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65e92960-f724-4456-9a71-b55434b3b9eb service neutron] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:56:51.256710 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 1128/2251] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 1 16:56:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 16:56:51.257483 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65e92960-f724-4456-9a71-b55434b3b9eb service neutron] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:56:51.258376 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.notifications [None req-b2953ec1-378d-4109-b8f3-cf6068a39d23 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '704a245a8de14350b5ffe3d1d13ca1b2', 'request_id': 'req-b2953ec1-378d-4109-b8f3-cf6068a39d23'} {{(pid=87057) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:56:51.267818 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.notifications [None req-b2953ec1-378d-4109-b8f3-cf6068a39d23 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '704a245a8de14350b5ffe3d1d13ca1b2', 'request_id': 'req-b2953ec1-378d-4109-b8f3-cf6068a39d23'} {{(pid=87057) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:56:51.269243 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65e92960-f724-4456-9a71-b55434b3b9eb service neutron] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:56:51.273272 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG dbcounter [-] [87058] Writing DB stats keystone:INSERT=2,keystone:DELETE=8,keystone:SELECT=90 {{(pid=87058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:56:51.275160 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.notifications [None req-b2953ec1-378d-4109-b8f3-cf6068a39d23 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '704a245a8de14350b5ffe3d1d13ca1b2', 'request_id': 'req-b2953ec1-378d-4109-b8f3-cf6068a39d23'} {{(pid=87057) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:56:51.329754 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d444fd30-4886-4f69-a91b-8e60f4f63ae9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:56:51.330543 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d444fd30-4886-4f69-a91b-8e60f4f63ae9 admin admin] REQUEST_METHOD: `PUT` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:56:51.330820 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d444fd30-4886-4f69-a91b-8e60f4f63ae9 admin admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:56:51.331089 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d444fd30-4886-4f69-a91b-8e60f4f63ae9 admin admin] PATH_INFO: `/v3/projects/94ba30eccee547f8a240dd4644631343/users/587eb18dd944444e872bfa25759f8d78/roles/440bb943dd5b46fea038b903b6eaaf81` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:56:51.344012 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.notifications [None req-b2953ec1-378d-4109-b8f3-cf6068a39d23 admin admin] The token cache is being invalidate because project 704a245a8de14350b5ffe3d1d13ca1b2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87057) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 16:56:51.344529 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.notifications [None req-b2953ec1-378d-4109-b8f3-cf6068a39d23 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87057) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:56:51.345432 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.notifications [None req-b2953ec1-378d-4109-b8f3-cf6068a39d23 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87057) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:56:51.347329 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 1124/2252] 10.222.0.20 () {58 vars in 1233 bytes} [Mon Aug 1 16:56:51 2022] DELETE /identity/v3/projects/704a245a8de14350b5ffe3d1d13ca1b2 => generated 0 bytes in 248 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 16:56:51.348712 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da69bf4a-dad5-4d5a-9a69-be884c59a9e3 None None] REQUEST_METHOD: `POST` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:56:51.349201 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da69bf4a-dad5-4d5a-9a69-be884c59a9e3 None None] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:56:51.349477 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da69bf4a-dad5-4d5a-9a69-be884c59a9e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:56:51.351864 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d444fd30-4886-4f69-a91b-8e60f4f63ae9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=94ba30eccee547f8a240dd4644631343, user_id=587eb18dd944444e872bfa25759f8d78, role_id=440bb943dd5b46fea038b903b6eaaf81)` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:56:51.356359 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d444fd30-4886-4f69-a91b-8e60f4f63ae9 admin admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:56:51.371702 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.auth.core [None req-da69bf4a-dad5-4d5a-9a69-be884c59a9e3 None None] MFA Rules not processed for user `1007f8e379b74e169b98fb7c3c0ba5b4`. Rule list: `[]` (Enabled: `True`). {{(pid=87057) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:56:51.372936 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.fernet_utils [None req-da69bf4a-dad5-4d5a-9a69-be884c59a9e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87057) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:56:51.381600 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 1125/2253] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 1 16:56:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:56:51.383080 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c5f6e3-6402-48c7-a54b-ce6582ab5867 None None] REQUEST_METHOD: `POST` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:56:51.383793 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c5f6e3-6402-48c7-a54b-ce6582ab5867 None None] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:56:51.385635 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG dbcounter [-] [87057] Writing DB stats keystone:DELETE=8,keystone:SELECT=88,keystone:INSERT=4 {{(pid=87057) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:56:51.386323 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c5f6e3-6402-48c7-a54b-ce6582ab5867 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:56:51.407286 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.auth.core [None req-95c5f6e3-6402-48c7-a54b-ce6582ab5867 None None] MFA Rules not processed for user `1007f8e379b74e169b98fb7c3c0ba5b4`. Rule list: `[]` (Enabled: `True`). {{(pid=87057) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:56:51.408497 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.fernet_utils [None req-95c5f6e3-6402-48c7-a54b-ce6582ab5867 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87057) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:56:51.417794 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 1126/2254] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 1 16:56:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:56:51.418757 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95c5f6e3-6402-48c7-a54b-ce6582ab5867 None None] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:56:51.430216 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95c5f6e3-6402-48c7-a54b-ce6582ab5867 None None] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:56:51.432108 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 1129/2255] 10.222.0.20 () {60 vars in 1483 bytes} [Mon Aug 1 16:56:51 2022] PUT /identity/v3/projects/94ba30eccee547f8a240dd4644631343/users/587eb18dd944444e872bfa25759f8d78/roles/440bb943dd5b46fea038b903b6eaaf81 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 16:56:51.433807 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93bbfb2-8245-4274-bae7-e46be2fa65bf None None] REQUEST_METHOD: `POST` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:56:51.434121 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93bbfb2-8245-4274-bae7-e46be2fa65bf None None] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:56:51.434667 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93bbfb2-8245-4274-bae7-e46be2fa65bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:56:51.457646 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.auth.core [None req-d93bbfb2-8245-4274-bae7-e46be2fa65bf None None] MFA Rules not processed for user `1007f8e379b74e169b98fb7c3c0ba5b4`. Rule list: `[]` (Enabled: `True`). {{(pid=87058) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:56:51.459596 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.fernet_utils [None req-d93bbfb2-8245-4274-bae7-e46be2fa65bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:56:51.499515 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e81e4e44-cbaa-4b92-9385-61d799c52c44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:56:51.500141 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81e4e44-cbaa-4b92-9385-61d799c52c44 admin admin] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:56:51.500417 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81e4e44-cbaa-4b92-9385-61d799c52c44 admin admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:56:51.500718 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81e4e44-cbaa-4b92-9385-61d799c52c44 admin admin] PATH_INFO: `/v3/domains` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:56:51.501438 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e81e4e44-cbaa-4b92-9385-61d799c52c44 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87057) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:56:51.501741 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e81e4e44-cbaa-4b92-9385-61d799c52c44 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:56:51.503403 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e81e4e44-cbaa-4b92-9385-61d799c52c44 admin admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:56:51.524348 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 1127/2256] 10.222.0.20 () {58 vars in 1153 bytes} [Mon Aug 1 16:56:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 106 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:56:51.525996 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e81e4e44-cbaa-4b92-9385-61d799c52c44 admin admin] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:56:51.531541 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.fernet_utils [None req-e81e4e44-cbaa-4b92-9385-61d799c52c44 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87057) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:56:51.544381 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e81e4e44-cbaa-4b92-9385-61d799c52c44 admin admin] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:56:51.553482 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5482c368-36d9-41e3-96db-59e0226ebb3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:56:51.554258 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5482c368-36d9-41e3-96db-59e0226ebb3e admin admin] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:56:51.554609 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5482c368-36d9-41e3-96db-59e0226ebb3e admin admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:56:51.554953 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5482c368-36d9-41e3-96db-59e0226ebb3e admin admin] PATH_INFO: `/v3/roles` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:56:51.555734 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5482c368-36d9-41e3-96db-59e0226ebb3e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:56:51.558709 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5482c368-36d9-41e3-96db-59e0226ebb3e admin admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:56:51.568655 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 1130/2257] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 1 16:56:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:56:51.568655 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d93bbfb2-8245-4274-bae7-e46be2fa65bf None None] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:56:51.571737 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 1128/2258] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Aug 1 16:56:51 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 16:56:51.571737 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5482c368-36d9-41e3-96db-59e0226ebb3e admin admin] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:56:51.579864 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d93bbfb2-8245-4274-bae7-e46be2fa65bf None None] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:56:51.584381 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5482c368-36d9-41e3-96db-59e0226ebb3e admin admin] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:56:51.584506 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7986acb6-f613-4ea4-9482-c751c636b116 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:56:51.585535 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7986acb6-f613-4ea4-9482-c751c636b116 admin admin] REQUEST_METHOD: `POST` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:56:51.585819 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7986acb6-f613-4ea4-9482-c751c636b116 admin admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:56:51.586125 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7986acb6-f613-4ea4-9482-c751c636b116 admin admin] PATH_INFO: `/v3/projects` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:56:51.586838 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7986acb6-f613-4ea4-9482-c751c636b116 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:56:51.589040 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7986acb6-f613-4ea4-9482-c751c636b116 admin admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:56:51.590979 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e804c16-17ed-4572-ac20-07502a9ff848 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:56:51.592210 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e804c16-17ed-4572-ac20-07502a9ff848 admin admin] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:56:51.592825 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e804c16-17ed-4572-ac20-07502a9ff848 admin admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:56:51.593469 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e804c16-17ed-4572-ac20-07502a9ff848 admin admin] PATH_INFO: `/v3/domains` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:56:51.594755 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e804c16-17ed-4572-ac20-07502a9ff848 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87057) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:56:51.595223 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e804c16-17ed-4572-ac20-07502a9ff848 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:56:51.597421 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e804c16-17ed-4572-ac20-07502a9ff848 admin admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:56:51.611345 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 1129/2259] 10.222.0.20 () {58 vars in 1153 bytes} [Mon Aug 1 16:56:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:56:51.612250 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e804c16-17ed-4572-ac20-07502a9ff848 admin admin] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:56:51.626475 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e804c16-17ed-4572-ac20-07502a9ff848 admin admin] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:56:51.631244 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af2b9b23-1c15-4995-9260-f9e874cec5a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:56:51.631958 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2b9b23-1c15-4995-9260-f9e874cec5a7 admin admin] REQUEST_METHOD: `PUT` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:56:51.632235 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2b9b23-1c15-4995-9260-f9e874cec5a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:56:51.632514 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2b9b23-1c15-4995-9260-f9e874cec5a7 admin admin] PATH_INFO: `/v3/projects/94ba30eccee547f8a240dd4644631343/users/587eb18dd944444e872bfa25759f8d78/roles/440bb943dd5b46fea038b903b6eaaf81` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:56:51.635080 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af2b9b23-1c15-4995-9260-f9e874cec5a7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=94ba30eccee547f8a240dd4644631343, user_id=587eb18dd944444e872bfa25759f8d78, role_id=440bb943dd5b46fea038b903b6eaaf81)` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:56:51.637763 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af2b9b23-1c15-4995-9260-f9e874cec5a7 admin admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:56:51.649457 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 1130/2260] 10.222.0.20 () {60 vars in 1483 bytes} [Mon Aug 1 16:56:51 2022] PUT /identity/v3/projects/94ba30eccee547f8a240dd4644631343/users/587eb18dd944444e872bfa25759f8d78/roles/440bb943dd5b46fea038b903b6eaaf81 => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 16:56:51.650334 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af2b9b23-1c15-4995-9260-f9e874cec5a7 admin admin] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:56:51.716524 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 1131/2261] 10.222.0.20 () {60 vars in 1153 bytes} [Mon Aug 1 16:56:51 2022] POST /identity/v3/projects => generated 397 bytes in 147 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 16:56:51.716524 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482842fd-b520-4e4b-bacd-2e4179cda352 None None] REQUEST_METHOD: `POST` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:56:51.716524 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482842fd-b520-4e4b-bacd-2e4179cda352 None None] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:56:51.716524 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482842fd-b520-4e4b-bacd-2e4179cda352 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:56:51.736402 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af2b9b23-1c15-4995-9260-f9e874cec5a7 admin admin] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:56:51.769203 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.auth.core [None req-482842fd-b520-4e4b-bacd-2e4179cda352 None None] MFA Rules not processed for user `587eb18dd944444e872bfa25759f8d78`. Rule list: `[]` (Enabled: `True`). {{(pid=87058) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:56:51.770956 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.fernet_utils [None req-482842fd-b520-4e4b-bacd-2e4179cda352 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:56:51.789509 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG dbcounter [-] [87057] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=87057) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:56:51.831185 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3148c8a9-d530-4907-a9f9-50a1ceae3266 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:56:51.831185 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3148c8a9-d530-4907-a9f9-50a1ceae3266 admin admin] REQUEST_METHOD: `POST` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:56:51.831185 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3148c8a9-d530-4907-a9f9-50a1ceae3266 admin admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:56:51.831185 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3148c8a9-d530-4907-a9f9-50a1ceae3266 admin admin] PATH_INFO: `/v3/projects` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:56:51.831632 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3148c8a9-d530-4907-a9f9-50a1ceae3266 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:56:51.834213 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3148c8a9-d530-4907-a9f9-50a1ceae3266 admin admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:56:51.856650 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG dbcounter [-] [87058] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=87058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:56:51.869517 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 1132/2262] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 1 16:56:51 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:56:51.870844 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-482842fd-b520-4e4b-bacd-2e4179cda352 None None] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:56:51.874352 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.fernet_utils [None req-482842fd-b520-4e4b-bacd-2e4179cda352 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:56:51.889994 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-482842fd-b520-4e4b-bacd-2e4179cda352 None None] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:56:51.894729 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab5d3c7-215e-49fd-98c3-0aedc41d3690 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:56:51.895975 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab5d3c7-215e-49fd-98c3-0aedc41d3690 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:56:51.895975 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab5d3c7-215e-49fd-98c3-0aedc41d3690 admin admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:56:51.895975 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab5d3c7-215e-49fd-98c3-0aedc41d3690 admin admin] PATH_INFO: `/v3/projects/e90a999b9d42435eb544f589329f53cf` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:56:51.897099 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ab5d3c7-215e-49fd-98c3-0aedc41d3690 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e90a999b9d42435eb544f589329f53cf)` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:56:51.899058 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ab5d3c7-215e-49fd-98c3-0aedc41d3690 admin admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:56:51.967732 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 1131/2263] 10.222.0.20 () {60 vars in 1153 bytes} [Mon Aug 1 16:56:51 2022] POST /identity/v3/projects => generated 387 bytes in 318 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 16:56:51.968564 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3148c8a9-d530-4907-a9f9-50a1ceae3266 admin admin] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:56:52.011779 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.notifications [None req-5ab5d3c7-215e-49fd-98c3-0aedc41d3690 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e90a999b9d42435eb544f589329f53cf', 'request_id': 'req-5ab5d3c7-215e-49fd-98c3-0aedc41d3690'} {{(pid=87058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:56:52.017990 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.notifications [None req-5ab5d3c7-215e-49fd-98c3-0aedc41d3690 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e90a999b9d42435eb544f589329f53cf', 'request_id': 'req-5ab5d3c7-215e-49fd-98c3-0aedc41d3690'} {{(pid=87058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:56:52.023467 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.notifications [None req-5ab5d3c7-215e-49fd-98c3-0aedc41d3690 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e90a999b9d42435eb544f589329f53cf', 'request_id': 'req-5ab5d3c7-215e-49fd-98c3-0aedc41d3690'} {{(pid=87058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:56:52.042827 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3148c8a9-d530-4907-a9f9-50a1ceae3266 admin admin] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:56:52.093081 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.notifications [None req-5ab5d3c7-215e-49fd-98c3-0aedc41d3690 admin admin] The token cache is being invalidate because project e90a999b9d42435eb544f589329f53cf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87058) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 16:56:52.093378 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.notifications [None req-5ab5d3c7-215e-49fd-98c3-0aedc41d3690 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:56:52.094057 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.notifications [None req-5ab5d3c7-215e-49fd-98c3-0aedc41d3690 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:56:52.095953 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 1133/2264] 10.222.0.20 () {58 vars in 1233 bytes} [Mon Aug 1 16:56:51 2022] DELETE /identity/v3/projects/e90a999b9d42435eb544f589329f53cf => generated 0 bytes in 226 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 16:56:52.099105 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab5d3c7-215e-49fd-98c3-0aedc41d3690 admin admin] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:56:52.099105 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.fernet_utils [None req-5ab5d3c7-215e-49fd-98c3-0aedc41d3690 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:56:52.101738 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e5c7a80-4566-44a1-b178-18968d75867e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:56:52.102361 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5c7a80-4566-44a1-b178-18968d75867e admin admin] REQUEST_METHOD: `POST` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:56:52.102642 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5c7a80-4566-44a1-b178-18968d75867e admin admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:56:52.102915 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5c7a80-4566-44a1-b178-18968d75867e admin admin] PATH_INFO: `/v3/users` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:56:52.103765 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e5c7a80-4566-44a1-b178-18968d75867e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:56:52.105746 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e5c7a80-4566-44a1-b178-18968d75867e admin admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:56:52.156072 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab5d3c7-215e-49fd-98c3-0aedc41d3690 admin admin] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:56:52.159995 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-864af69c-14bc-4c6b-9eac-dc62fd41dfc0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25970b52ca15448fa1abb1f974d64038', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b992d6a65df4bf9b27c13192d57c5f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:56:52.160633 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864af69c-14bc-4c6b-9eac-dc62fd41dfc0 service cinder] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:56:52.160921 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864af69c-14bc-4c6b-9eac-dc62fd41dfc0 service cinder] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:56:52.161189 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864af69c-14bc-4c6b-9eac-dc62fd41dfc0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:56:52.163034 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.fernet_utils [None req-864af69c-14bc-4c6b-9eac-dc62fd41dfc0 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:56:52.223061 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-864af69c-14bc-4c6b-9eac-dc62fd41dfc0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:56:52.224544 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-864af69c-14bc-4c6b-9eac-dc62fd41dfc0 service cinder] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:56:52.235681 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 1134/2265] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 1 16:56:52 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 16:56:52.236706 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-864af69c-14bc-4c6b-9eac-dc62fd41dfc0 service cinder] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:56:52.238014 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.fernet_utils [None req-864af69c-14bc-4c6b-9eac-dc62fd41dfc0 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:56:52.252832 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-864af69c-14bc-4c6b-9eac-dc62fd41dfc0 service cinder] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:56:52.258360 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b0a8860-63c5-40a2-91a4-c12146aaa2bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:56:52.258994 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b0a8860-63c5-40a2-91a4-c12146aaa2bf admin admin] REQUEST_METHOD: `POST` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:56:52.259274 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b0a8860-63c5-40a2-91a4-c12146aaa2bf admin admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:56:52.260229 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b0a8860-63c5-40a2-91a4-c12146aaa2bf admin admin] PATH_INFO: `/v3/users` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:56:52.260895 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b0a8860-63c5-40a2-91a4-c12146aaa2bf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:56:52.262841 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b0a8860-63c5-40a2-91a4-c12146aaa2bf admin admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:56:52.267477 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 1132/2266] 10.222.0.20 () {60 vars in 1144 bytes} [Mon Aug 1 16:56:51 2022] POST /identity/v3/users => generated 299 bytes in 296 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 16:56:52.267477 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e5c7a80-4566-44a1-b178-18968d75867e admin admin] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:56:52.269138 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.fernet_utils [None req-9e5c7a80-4566-44a1-b178-18968d75867e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87057) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:56:52.282759 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e5c7a80-4566-44a1-b178-18968d75867e admin admin] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:56:52.289025 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d88241-e709-4c65-a8dc-9fe5df6a3313 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:56:52.289025 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d88241-e709-4c65-a8dc-9fe5df6a3313 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:56:52.289243 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d88241-e709-4c65-a8dc-9fe5df6a3313 admin admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:56:52.289439 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d88241-e709-4c65-a8dc-9fe5df6a3313 admin admin] PATH_INFO: `/v3/users/5b57042db90047929c6b8a7cf36e286c` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:56:52.290770 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29d88241-e709-4c65-a8dc-9fe5df6a3313 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5b57042db90047929c6b8a7cf36e286c)` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:56:52.292919 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29d88241-e709-4c65-a8dc-9fe5df6a3313 admin admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:56:52.355212 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 1135/2267] 10.222.0.20 () {60 vars in 1144 bytes} [Mon Aug 1 16:56:52 2022] POST /identity/v3/users => generated 294 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 16:56:52.355994 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b0a8860-63c5-40a2-91a4-c12146aaa2bf admin admin] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:56:52.359389 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.fernet_utils [None req-1b0a8860-63c5-40a2-91a4-c12146aaa2bf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:56:52.371418 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG dbcounter [-] [87057] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87057) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:56:52.373173 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b0a8860-63c5-40a2-91a4-c12146aaa2bf admin admin] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:56:52.377899 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-043777b6-8f73-4e6b-a3cb-bbe346b7c391 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:56:52.379634 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043777b6-8f73-4e6b-a3cb-bbe346b7c391 admin admin] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:56:52.379634 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043777b6-8f73-4e6b-a3cb-bbe346b7c391 admin admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:56:52.379634 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043777b6-8f73-4e6b-a3cb-bbe346b7c391 admin admin] PATH_INFO: `/v3/roles` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:56:52.379893 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-043777b6-8f73-4e6b-a3cb-bbe346b7c391 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:56:52.381422 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-043777b6-8f73-4e6b-a3cb-bbe346b7c391 admin admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:56:52.392146 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 1136/2268] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Aug 1 16:56:52 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 16:56:52.392933 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-043777b6-8f73-4e6b-a3cb-bbe346b7c391 admin admin] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:56:52.397688 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG dbcounter [-] [87058] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=87058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:56:52.402757 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-043777b6-8f73-4e6b-a3cb-bbe346b7c391 admin admin] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:56:52.407500 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8696268a-5761-47a3-a2c6-eccc094f8b6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:56:52.408087 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8696268a-5761-47a3-a2c6-eccc094f8b6e admin admin] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:56:52.408369 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8696268a-5761-47a3-a2c6-eccc094f8b6e admin admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:56:52.408642 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8696268a-5761-47a3-a2c6-eccc094f8b6e admin admin] PATH_INFO: `/v3/roles` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:56:52.409333 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8696268a-5761-47a3-a2c6-eccc094f8b6e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:56:52.410936 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8696268a-5761-47a3-a2c6-eccc094f8b6e admin admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:56:52.426856 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 1137/2269] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Aug 1 16:56:52 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 16:56:52.426856 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8696268a-5761-47a3-a2c6-eccc094f8b6e admin admin] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:56:52.434715 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8696268a-5761-47a3-a2c6-eccc094f8b6e admin admin] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:56:52.439447 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-548626f4-52b6-4642-ab83-3dd885a670ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:56:52.440060 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548626f4-52b6-4642-ab83-3dd885a670ff admin admin] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:56:52.440341 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548626f4-52b6-4642-ab83-3dd885a670ff admin admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:56:52.440616 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548626f4-52b6-4642-ab83-3dd885a670ff admin admin] PATH_INFO: `/v3/roles` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:56:52.441320 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-548626f4-52b6-4642-ab83-3dd885a670ff admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:56:52.442966 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-548626f4-52b6-4642-ab83-3dd885a670ff admin admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:56:52.457502 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 1138/2270] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Aug 1 16:56:52 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 16:56:52.457502 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-548626f4-52b6-4642-ab83-3dd885a670ff admin admin] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:56:52.466810 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-548626f4-52b6-4642-ab83-3dd885a670ff admin admin] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:56:52.472605 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-438a8d76-562a-41be-872e-81b726866cc0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:56:52.473271 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-438a8d76-562a-41be-872e-81b726866cc0 admin admin] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:56:52.473569 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-438a8d76-562a-41be-872e-81b726866cc0 admin admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:56:52.473833 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-438a8d76-562a-41be-872e-81b726866cc0 admin admin] PATH_INFO: `/v3/roles` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:56:52.474774 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-438a8d76-562a-41be-872e-81b726866cc0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:56:52.476705 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-438a8d76-562a-41be-872e-81b726866cc0 admin admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:56:52.496074 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 1139/2271] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Aug 1 16:56:52 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 16:56:52.497167 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-438a8d76-562a-41be-872e-81b726866cc0 admin admin] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:56:52.501444 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.notifications [None req-29d88241-e709-4c65-a8dc-9fe5df6a3313 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5b57042db90047929c6b8a7cf36e286c'} {{(pid=87057) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:56:52.504908 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.notifications [None req-29d88241-e709-4c65-a8dc-9fe5df6a3313 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5b57042db90047929c6b8a7cf36e286c'} {{(pid=87057) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:56:52.508033 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-438a8d76-562a-41be-872e-81b726866cc0 admin admin] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:56:52.513480 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4be3bb2-ea72-450a-bbc9-141bddab6b30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:56:52.514197 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4be3bb2-ea72-450a-bbc9-141bddab6b30 admin admin] REQUEST_METHOD: `PUT` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:56:52.514494 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4be3bb2-ea72-450a-bbc9-141bddab6b30 admin admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:56:52.514759 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4be3bb2-ea72-450a-bbc9-141bddab6b30 admin admin] PATH_INFO: `/v3/projects/bae76503dca34892b404b3f1db8fea83/users/dda13b874aab4c119d70a76aa43bf924/roles/0999d3da63a545f18ff3252afdc90518` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:56:52.529962 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4be3bb2-ea72-450a-bbc9-141bddab6b30 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bae76503dca34892b404b3f1db8fea83, user_id=dda13b874aab4c119d70a76aa43bf924, role_id=0999d3da63a545f18ff3252afdc90518)` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:56:52.532650 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4be3bb2-ea72-450a-bbc9-141bddab6b30 admin admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:56:52.587709 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.notifications [None req-29d88241-e709-4c65-a8dc-9fe5df6a3313 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5b57042db90047929c6b8a7cf36e286c'} {{(pid=87057) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:56:52.588592 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.notifications [None req-29d88241-e709-4c65-a8dc-9fe5df6a3313 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5b57042db90047929c6b8a7cf36e286c'} {{(pid=87057) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:56:52.596377 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.notifications [None req-29d88241-e709-4c65-a8dc-9fe5df6a3313 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5b57042db90047929c6b8a7cf36e286c'} {{(pid=87057) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:56:52.614437 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 1133/2272] 10.222.0.20 () {58 vars in 1224 bytes} [Mon Aug 1 16:56:52 2022] DELETE /identity/v3/users/5b57042db90047929c6b8a7cf36e286c => generated 0 bytes in 349 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 16:56:52.615248 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d88241-e709-4c65-a8dc-9fe5df6a3313 admin admin] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:56:52.616577 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.fernet_utils [None req-29d88241-e709-4c65-a8dc-9fe5df6a3313 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87057) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:56:52.625150 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 1140/2273] 10.222.0.20 () {60 vars in 1483 bytes} [Mon Aug 1 16:56:52 2022] PUT /identity/v3/projects/bae76503dca34892b404b3f1db8fea83/users/dda13b874aab4c119d70a76aa43bf924/roles/0999d3da63a545f18ff3252afdc90518 => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 16:56:52.638314 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27dbed01-91b0-4bab-a3b1-3bfcb3f23b30 None None] REQUEST_METHOD: `POST` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:56:52.638988 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27dbed01-91b0-4bab-a3b1-3bfcb3f23b30 None None] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:56:52.639315 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27dbed01-91b0-4bab-a3b1-3bfcb3f23b30 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:56:52.677658 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.auth.core [None req-27dbed01-91b0-4bab-a3b1-3bfcb3f23b30 None None] MFA Rules not processed for user `dda13b874aab4c119d70a76aa43bf924`. Rule list: `[]` (Enabled: `True`). {{(pid=87058) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:56:52.678931 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.fernet_utils [None req-27dbed01-91b0-4bab-a3b1-3bfcb3f23b30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:56:52.687914 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d88241-e709-4c65-a8dc-9fe5df6a3313 admin admin] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:56:52.750526 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7811529a-3b87-49b0-8346-7a8bd6848089 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:56:52.751397 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7811529a-3b87-49b0-8346-7a8bd6848089 admin admin] REQUEST_METHOD: `PUT` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:56:52.751800 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7811529a-3b87-49b0-8346-7a8bd6848089 admin admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:56:52.752076 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7811529a-3b87-49b0-8346-7a8bd6848089 admin admin] PATH_INFO: `/v3/projects/dfc1c3badb144d24a2b86341a155505a/users/48cc0715e7b04f8a8c9f0d8c455bd4c6/roles/0999d3da63a545f18ff3252afdc90518` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:56:52.757311 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG dbcounter [-] [87057] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=87057) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:56:52.770938 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 1141/2274] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 1 16:56:52 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:56:52.774207 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7811529a-3b87-49b0-8346-7a8bd6848089 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dfc1c3badb144d24a2b86341a155505a, user_id=48cc0715e7b04f8a8c9f0d8c455bd4c6, role_id=0999d3da63a545f18ff3252afdc90518)` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:56:52.777439 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7811529a-3b87-49b0-8346-7a8bd6848089 admin admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:56:52.782137 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27dbed01-91b0-4bab-a3b1-3bfcb3f23b30 None None] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:56:52.783576 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.fernet_utils [None req-27dbed01-91b0-4bab-a3b1-3bfcb3f23b30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:56:52.843396 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27dbed01-91b0-4bab-a3b1-3bfcb3f23b30 None None] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:56:52.847306 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb7c853-2be1-4703-9bc5-cf99ade82338 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25970b52ca15448fa1abb1f974d64038', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b992d6a65df4bf9b27c13192d57c5f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:56:52.849350 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb7c853-2be1-4703-9bc5-cf99ade82338 service cinder] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:56:52.849350 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb7c853-2be1-4703-9bc5-cf99ade82338 service cinder] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:56:52.849350 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb7c853-2be1-4703-9bc5-cf99ade82338 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:56:52.862498 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eb7c853-2be1-4703-9bc5-cf99ade82338 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:56:52.864048 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eb7c853-2be1-4703-9bc5-cf99ade82338 service cinder] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:56:52.878978 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 1142/2276] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 1 16:56:52 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 01 16:56:52.879162 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 1134/2276] 10.222.0.20 () {60 vars in 1483 bytes} [Mon Aug 1 16:56:52 2022] PUT /identity/v3/projects/dfc1c3badb144d24a2b86341a155505a/users/48cc0715e7b04f8a8c9f0d8c455bd4c6/roles/0999d3da63a545f18ff3252afdc90518 => generated 0 bytes in 264 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 16:56:52.886070 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7e7bcc-b024-434f-bc32-fba53a5c4206 None None] REQUEST_METHOD: `POST` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:56:52.886202 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7e7bcc-b024-434f-bc32-fba53a5c4206 None None] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:56:52.886460 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7e7bcc-b024-434f-bc32-fba53a5c4206 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:56:52.932515 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.auth.core [None req-ba7e7bcc-b024-434f-bc32-fba53a5c4206 None None] MFA Rules not processed for user `1007f8e379b74e169b98fb7c3c0ba5b4`. Rule list: `[]` (Enabled: `True`). {{(pid=87058) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:56:52.934002 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.fernet_utils [None req-ba7e7bcc-b024-434f-bc32-fba53a5c4206 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:56:52.982652 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG dbcounter [-] [87058] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:56:53.062594 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 1143/2277] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 1 16:56:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:56:53.077135 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7811529a-3b87-49b0-8346-7a8bd6848089 admin admin] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:56:53.080469 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.fernet_utils [None req-7811529a-3b87-49b0-8346-7a8bd6848089 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87057) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:56:53.182347 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7811529a-3b87-49b0-8346-7a8bd6848089 admin admin] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:56:53.186326 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f7c62da-a188-4cde-aa09-6507801c8ef4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d9bd8d649154c979be3744bfd016f95', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b992d6a65df4bf9b27c13192d57c5f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:56:53.187025 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7c62da-a188-4cde-aa09-6507801c8ef4 service neutron] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:56:53.187373 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7c62da-a188-4cde-aa09-6507801c8ef4 service neutron] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:56:53.187642 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7c62da-a188-4cde-aa09-6507801c8ef4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:56:53.199247 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f7c62da-a188-4cde-aa09-6507801c8ef4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:56:53.205718 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f7c62da-a188-4cde-aa09-6507801c8ef4 service neutron] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:56:53.211219 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 1135/2278] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 1 16:56:53 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 16:56:53.262943 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba7e7bcc-b024-434f-bc32-fba53a5c4206 None None] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:56:53.264437 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.fernet_utils [None req-ba7e7bcc-b024-434f-bc32-fba53a5c4206 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:56:53.289473 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba7e7bcc-b024-434f-bc32-fba53a5c4206 None None] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:56:53.297505 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b5681e-05cb-49a1-9e87-bf2b5b1089a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:56:53.298196 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b5681e-05cb-49a1-9e87-bf2b5b1089a6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:56:53.298482 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b5681e-05cb-49a1-9e87-bf2b5b1089a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:56:53.298780 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b5681e-05cb-49a1-9e87-bf2b5b1089a6 admin admin] PATH_INFO: `/v3/projects/1ab357bf50d64943a65a7ce3c22ce714` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:56:53.301543 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35b5681e-05cb-49a1-9e87-bf2b5b1089a6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1ab357bf50d64943a65a7ce3c22ce714)` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:56:53.303764 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35b5681e-05cb-49a1-9e87-bf2b5b1089a6 admin admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:56:53.436186 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.notifications [None req-35b5681e-05cb-49a1-9e87-bf2b5b1089a6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1ab357bf50d64943a65a7ce3c22ce714', 'request_id': 'req-35b5681e-05cb-49a1-9e87-bf2b5b1089a6'} {{(pid=87058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:56:53.442673 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.notifications [None req-35b5681e-05cb-49a1-9e87-bf2b5b1089a6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1ab357bf50d64943a65a7ce3c22ce714', 'request_id': 'req-35b5681e-05cb-49a1-9e87-bf2b5b1089a6'} {{(pid=87058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:56:53.448308 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.notifications [None req-35b5681e-05cb-49a1-9e87-bf2b5b1089a6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1ab357bf50d64943a65a7ce3c22ce714', 'request_id': 'req-35b5681e-05cb-49a1-9e87-bf2b5b1089a6'} {{(pid=87058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:56:53.563910 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.notifications [None req-35b5681e-05cb-49a1-9e87-bf2b5b1089a6 admin admin] The token cache is being invalidate because project 1ab357bf50d64943a65a7ce3c22ce714 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87058) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 16:56:53.564289 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.notifications [None req-35b5681e-05cb-49a1-9e87-bf2b5b1089a6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:56:53.564981 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.notifications [None req-35b5681e-05cb-49a1-9e87-bf2b5b1089a6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:56:53.566934 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 1144/2279] 10.222.0.20 () {58 vars in 1233 bytes} [Mon Aug 1 16:56:53 2022] DELETE /identity/v3/projects/1ab357bf50d64943a65a7ce3c22ce714 => generated 0 bytes in 305 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 16:56:53.571141 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f7c62da-a188-4cde-aa09-6507801c8ef4 service neutron] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:56:53.587153 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f7c62da-a188-4cde-aa09-6507801c8ef4 service neutron] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:56:53.648587 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a530b13c-4f75-4144-b9fa-f7af48dcc00d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:56:53.649263 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a530b13c-4f75-4144-b9fa-f7af48dcc00d admin admin] REQUEST_METHOD: `DELETE` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:56:53.649545 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a530b13c-4f75-4144-b9fa-f7af48dcc00d admin admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:56:53.649824 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a530b13c-4f75-4144-b9fa-f7af48dcc00d admin admin] PATH_INFO: `/v3/users/b50ba2e22db7412895f7c137cc412b0d` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:56:53.651728 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a530b13c-4f75-4144-b9fa-f7af48dcc00d admin admin] RBAC: Authorizing `identity:delete_user(user_id=b50ba2e22db7412895f7c137cc412b0d)` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:56:53.653666 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a530b13c-4f75-4144-b9fa-f7af48dcc00d admin admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:56:53.792294 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.notifications [None req-a530b13c-4f75-4144-b9fa-f7af48dcc00d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b50ba2e22db7412895f7c137cc412b0d'} {{(pid=87057) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:56:53.793696 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.notifications [None req-a530b13c-4f75-4144-b9fa-f7af48dcc00d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b50ba2e22db7412895f7c137cc412b0d'} {{(pid=87057) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:56:53.874568 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.notifications [None req-a530b13c-4f75-4144-b9fa-f7af48dcc00d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b50ba2e22db7412895f7c137cc412b0d'} {{(pid=87057) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:56:53.875334 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.notifications [None req-a530b13c-4f75-4144-b9fa-f7af48dcc00d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b50ba2e22db7412895f7c137cc412b0d'} {{(pid=87057) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:56:53.883990 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.notifications [None req-a530b13c-4f75-4144-b9fa-f7af48dcc00d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b50ba2e22db7412895f7c137cc412b0d'} {{(pid=87057) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:56:53.896782 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG dbcounter [-] [87057] Writing DB stats keystone:SELECT=89,keystone:INSERT=2,keystone:DELETE=9 {{(pid=87057) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:56:53.907493 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 1136/2280] 10.222.0.20 () {58 vars in 1224 bytes} [Mon Aug 1 16:56:53 2022] DELETE /identity/v3/users/b50ba2e22db7412895f7c137cc412b0d => generated 0 bytes in 337 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 16:56:54.485675 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b5681e-05cb-49a1-9e87-bf2b5b1089a6 admin admin] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:56:54.487195 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.fernet_utils [None req-35b5681e-05cb-49a1-9e87-bf2b5b1089a6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:56:54.501178 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b5681e-05cb-49a1-9e87-bf2b5b1089a6 admin admin] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:56:54.565008 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5abd3bef-1e8e-4c93-9810-43ba19371bd5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:56:54.565887 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5abd3bef-1e8e-4c93-9810-43ba19371bd5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:56:54.566257 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5abd3bef-1e8e-4c93-9810-43ba19371bd5 admin admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:56:54.566574 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5abd3bef-1e8e-4c93-9810-43ba19371bd5 admin admin] PATH_INFO: `/v3/projects/c5d07c2715be4095be84462837332666` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:56:54.567863 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5abd3bef-1e8e-4c93-9810-43ba19371bd5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c5d07c2715be4095be84462837332666)` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:56:54.569845 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5abd3bef-1e8e-4c93-9810-43ba19371bd5 admin admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:56:54.715362 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.notifications [None req-5abd3bef-1e8e-4c93-9810-43ba19371bd5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c5d07c2715be4095be84462837332666', 'request_id': 'req-5abd3bef-1e8e-4c93-9810-43ba19371bd5'} {{(pid=87058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:56:54.722820 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.notifications [None req-5abd3bef-1e8e-4c93-9810-43ba19371bd5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c5d07c2715be4095be84462837332666', 'request_id': 'req-5abd3bef-1e8e-4c93-9810-43ba19371bd5'} {{(pid=87058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:56:54.729801 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.notifications [None req-5abd3bef-1e8e-4c93-9810-43ba19371bd5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c5d07c2715be4095be84462837332666', 'request_id': 'req-5abd3bef-1e8e-4c93-9810-43ba19371bd5'} {{(pid=87058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:56:54.732402 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG dbcounter [-] [87058] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=87058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:56:54.813757 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.notifications [None req-5abd3bef-1e8e-4c93-9810-43ba19371bd5 admin admin] The token cache is being invalidate because project c5d07c2715be4095be84462837332666 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87058) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 16:56:54.814141 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.notifications [None req-5abd3bef-1e8e-4c93-9810-43ba19371bd5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:56:54.814992 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.notifications [None req-5abd3bef-1e8e-4c93-9810-43ba19371bd5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:56:54.821399 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 1145/2281] 10.222.0.20 () {58 vars in 1233 bytes} [Mon Aug 1 16:56:54 2022] DELETE /identity/v3/projects/c5d07c2715be4095be84462837332666 => generated 0 bytes in 332 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 16:56:54.823289 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab428fb-eb71-4117-bef9-8c408ba8e9b4 None None] REQUEST_METHOD: `POST` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:56:54.823757 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab428fb-eb71-4117-bef9-8c408ba8e9b4 None None] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:56:54.823837 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab428fb-eb71-4117-bef9-8c408ba8e9b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:56:54.859669 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.auth.core [None req-4ab428fb-eb71-4117-bef9-8c408ba8e9b4 None None] MFA Rules not processed for user `1007f8e379b74e169b98fb7c3c0ba5b4`. Rule list: `[]` (Enabled: `True`). {{(pid=87057) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:56:54.860995 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.fernet_utils [None req-4ab428fb-eb71-4117-bef9-8c408ba8e9b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87057) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:56:54.966506 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 1137/2282] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 1 16:56:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 16:56:54.973586 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996b8312-198c-4fe4-a35c-d6787d2d7b68 None None] REQUEST_METHOD: `POST` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:56:54.973876 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996b8312-198c-4fe4-a35c-d6787d2d7b68 None None] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:56:54.974337 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996b8312-198c-4fe4-a35c-d6787d2d7b68 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:56:55.004970 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.auth.core [None req-996b8312-198c-4fe4-a35c-d6787d2d7b68 None None] MFA Rules not processed for user `1007f8e379b74e169b98fb7c3c0ba5b4`. Rule list: `[]` (Enabled: `True`). {{(pid=87058) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:56:55.006180 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.fernet_utils [None req-996b8312-198c-4fe4-a35c-d6787d2d7b68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:56:55.021517 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 1146/2283] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 1 16:56:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:56:55.022282 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab428fb-eb71-4117-bef9-8c408ba8e9b4 None None] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:56:55.041148 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab428fb-eb71-4117-bef9-8c408ba8e9b4 None None] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:56:55.047551 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f2806fe-eaec-499d-a2b7-c92c99f2688e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:56:55.048185 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2806fe-eaec-499d-a2b7-c92c99f2688e admin admin] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:56:55.048465 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2806fe-eaec-499d-a2b7-c92c99f2688e admin admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:56:55.048976 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2806fe-eaec-499d-a2b7-c92c99f2688e admin admin] PATH_INFO: `/v3/domains` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:56:55.049693 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f2806fe-eaec-499d-a2b7-c92c99f2688e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87057) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:56:55.050086 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f2806fe-eaec-499d-a2b7-c92c99f2688e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:56:55.051779 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f2806fe-eaec-499d-a2b7-c92c99f2688e admin admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:56:55.069530 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 1138/2284] 10.222.0.20 () {58 vars in 1153 bytes} [Mon Aug 1 16:56:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:56:55.074797 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-996b8312-198c-4fe4-a35c-d6787d2d7b68 None None] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:56:55.085369 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-996b8312-198c-4fe4-a35c-d6787d2d7b68 None None] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:56:55.091245 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a3889f-078f-4e48-b47e-8a50bf7a43a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:56:55.092003 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a3889f-078f-4e48-b47e-8a50bf7a43a1 admin admin] REQUEST_METHOD: `POST` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:56:55.093357 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a3889f-078f-4e48-b47e-8a50bf7a43a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:56:55.093357 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a3889f-078f-4e48-b47e-8a50bf7a43a1 admin admin] PATH_INFO: `/v3/projects` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:56:55.093357 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3a3889f-078f-4e48-b47e-8a50bf7a43a1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:56:55.095180 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3a3889f-078f-4e48-b47e-8a50bf7a43a1 admin admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:56:55.225856 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 1147/2285] 10.222.0.20 () {60 vars in 1153 bytes} [Mon Aug 1 16:56:55 2022] POST /identity/v3/projects => generated 365 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 16:56:55.229998 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f2806fe-eaec-499d-a2b7-c92c99f2688e admin admin] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:56:55.242592 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f2806fe-eaec-499d-a2b7-c92c99f2688e admin admin] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:56:55.326560 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c699978-5151-4898-a0cb-3bccb61c167a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:56:55.328163 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c699978-5151-4898-a0cb-3bccb61c167a admin admin] REQUEST_METHOD: `POST` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:56:55.328163 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c699978-5151-4898-a0cb-3bccb61c167a admin admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:56:55.328163 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c699978-5151-4898-a0cb-3bccb61c167a admin admin] PATH_INFO: `/v3/users` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:56:55.328954 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c699978-5151-4898-a0cb-3bccb61c167a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:56:55.330952 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c699978-5151-4898-a0cb-3bccb61c167a admin admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:56:55.421176 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 1139/2286] 10.222.0.20 () {60 vars in 1144 bytes} [Mon Aug 1 16:56:55 2022] POST /identity/v3/users => generated 283 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 16:56:55.429271 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a3889f-078f-4e48-b47e-8a50bf7a43a1 admin admin] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:56:55.444905 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a3889f-078f-4e48-b47e-8a50bf7a43a1 admin admin] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:56:55.451573 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa0b0b3c-b9fb-4da8-8c43-a5b0a576c8ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:56:55.452376 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0b0b3c-b9fb-4da8-8c43-a5b0a576c8ee admin admin] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:56:55.453203 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0b0b3c-b9fb-4da8-8c43-a5b0a576c8ee admin admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:56:55.453577 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0b0b3c-b9fb-4da8-8c43-a5b0a576c8ee admin admin] PATH_INFO: `/v3/roles` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:56:55.454551 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa0b0b3c-b9fb-4da8-8c43-a5b0a576c8ee admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:56:55.456273 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa0b0b3c-b9fb-4da8-8c43-a5b0a576c8ee admin admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:56:55.469854 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 1148/2287] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Aug 1 16:56:55 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 16:56:55.474422 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c699978-5151-4898-a0cb-3bccb61c167a admin admin] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:56:55.487682 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c699978-5151-4898-a0cb-3bccb61c167a admin admin] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:56:55.498065 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19589872-33b2-4199-bc39-b89d92e22700 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:56:55.500331 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19589872-33b2-4199-bc39-b89d92e22700 admin admin] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:56:55.500761 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19589872-33b2-4199-bc39-b89d92e22700 admin admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:56:55.501074 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19589872-33b2-4199-bc39-b89d92e22700 admin admin] PATH_INFO: `/v3/roles` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:56:55.503595 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19589872-33b2-4199-bc39-b89d92e22700 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:56:55.505340 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19589872-33b2-4199-bc39-b89d92e22700 admin admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:56:55.524605 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 1140/2288] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Aug 1 16:56:55 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 16:56:55.529911 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa0b0b3c-b9fb-4da8-8c43-a5b0a576c8ee admin admin] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:56:55.547563 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa0b0b3c-b9fb-4da8-8c43-a5b0a576c8ee admin admin] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:56:55.554134 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a97d730e-6cf3-4715-a0fe-01b04b704859 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:56:55.555046 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97d730e-6cf3-4715-a0fe-01b04b704859 admin admin] REQUEST_METHOD: `PUT` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:56:55.555454 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97d730e-6cf3-4715-a0fe-01b04b704859 admin admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:56:55.555900 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97d730e-6cf3-4715-a0fe-01b04b704859 admin admin] PATH_INFO: `/v3/projects/1ddce3c8d30f4343a6c94f95caebcd97/users/f56ebe2d12ed45b9ba1f6dec45fda595/roles/0999d3da63a545f18ff3252afdc90518` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:56:55.575689 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a97d730e-6cf3-4715-a0fe-01b04b704859 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1ddce3c8d30f4343a6c94f95caebcd97, user_id=f56ebe2d12ed45b9ba1f6dec45fda595, role_id=0999d3da63a545f18ff3252afdc90518)` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:56:55.579316 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a97d730e-6cf3-4715-a0fe-01b04b704859 admin admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:56:55.698450 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 1149/2289] 10.222.0.20 () {60 vars in 1483 bytes} [Mon Aug 1 16:56:55 2022] PUT /identity/v3/projects/1ddce3c8d30f4343a6c94f95caebcd97/users/f56ebe2d12ed45b9ba1f6dec45fda595/roles/0999d3da63a545f18ff3252afdc90518 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 16:56:55.710631 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a65d2b-8fba-4eb0-bd86-69a59932b32f None None] REQUEST_METHOD: `POST` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:56:55.711719 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a65d2b-8fba-4eb0-bd86-69a59932b32f None None] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:56:55.712199 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a65d2b-8fba-4eb0-bd86-69a59932b32f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:56:55.760824 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.auth.core [None req-78a65d2b-8fba-4eb0-bd86-69a59932b32f None None] MFA Rules not processed for user `f56ebe2d12ed45b9ba1f6dec45fda595`. Rule list: `[]` (Enabled: `True`). {{(pid=87057) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:56:55.762224 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.fernet_utils [None req-78a65d2b-8fba-4eb0-bd86-69a59932b32f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87057) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:56:55.847834 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG dbcounter [-] [87057] Writing DB stats keystone:DELETE=1,keystone:SELECT=96,keystone:INSERT=3 {{(pid=87057) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:56:55.876405 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 1141/2290] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 1 16:56:55 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:56:55.913074 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a97d730e-6cf3-4715-a0fe-01b04b704859 admin admin] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:56:55.918822 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.fernet_utils [None req-a97d730e-6cf3-4715-a0fe-01b04b704859 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:56:55.993529 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a97d730e-6cf3-4715-a0fe-01b04b704859 admin admin] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:56:55.998904 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54aea405-9e1d-487c-bea0-ece4f38b577b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25970b52ca15448fa1abb1f974d64038', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b992d6a65df4bf9b27c13192d57c5f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:56:55.999513 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54aea405-9e1d-487c-bea0-ece4f38b577b service cinder] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:56:55.999789 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54aea405-9e1d-487c-bea0-ece4f38b577b service cinder] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:56:56.000066 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54aea405-9e1d-487c-bea0-ece4f38b577b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:56:56.009575 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54aea405-9e1d-487c-bea0-ece4f38b577b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:56:56.011126 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54aea405-9e1d-487c-bea0-ece4f38b577b service cinder] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:56:56.021360 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 1150/2291] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 1 16:56:55 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 16:56:56.749199 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78a65d2b-8fba-4eb0-bd86-69a59932b32f None None] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:56:56.750780 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.fernet_utils [None req-78a65d2b-8fba-4eb0-bd86-69a59932b32f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87057) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:56:56.858015 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78a65d2b-8fba-4eb0-bd86-69a59932b32f None None] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:56:56.865333 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f801d477-c5e7-4165-a547-be30ef834d84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:56:56.866251 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f801d477-c5e7-4165-a547-be30ef834d84 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:56:56.867512 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f801d477-c5e7-4165-a547-be30ef834d84 admin admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:56:56.867977 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f801d477-c5e7-4165-a547-be30ef834d84 admin admin] PATH_INFO: `/v3/users/f4c5dccbaef740549fa5e56a7032aa8e` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:56:56.869832 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f801d477-c5e7-4165-a547-be30ef834d84 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f4c5dccbaef740549fa5e56a7032aa8e)` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:56:56.872392 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f801d477-c5e7-4165-a547-be30ef834d84 admin admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:56:57.121619 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.notifications [None req-f801d477-c5e7-4165-a547-be30ef834d84 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f4c5dccbaef740549fa5e56a7032aa8e'} {{(pid=87057) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:56:57.122265 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.notifications [None req-f801d477-c5e7-4165-a547-be30ef834d84 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f4c5dccbaef740549fa5e56a7032aa8e'} {{(pid=87057) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:56:57.227663 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.notifications [None req-f801d477-c5e7-4165-a547-be30ef834d84 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f4c5dccbaef740549fa5e56a7032aa8e'} {{(pid=87057) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:56:57.228894 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.notifications [None req-f801d477-c5e7-4165-a547-be30ef834d84 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f4c5dccbaef740549fa5e56a7032aa8e'} {{(pid=87057) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:56:57.237128 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.notifications [None req-f801d477-c5e7-4165-a547-be30ef834d84 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f4c5dccbaef740549fa5e56a7032aa8e'} {{(pid=87057) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:56:57.259820 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 1142/2292] 10.222.0.20 () {58 vars in 1224 bytes} [Mon Aug 1 16:56:56 2022] DELETE /identity/v3/users/f4c5dccbaef740549fa5e56a7032aa8e => generated 0 bytes in 511 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 16:56:57.264776 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79bbf9c8-116b-4338-8cfd-cadbed286ff1 None None] REQUEST_METHOD: `POST` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:56:57.265282 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79bbf9c8-116b-4338-8cfd-cadbed286ff1 None None] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:56:57.265628 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79bbf9c8-116b-4338-8cfd-cadbed286ff1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:56:57.285516 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.auth.core [None req-79bbf9c8-116b-4338-8cfd-cadbed286ff1 None None] MFA Rules not processed for user `1007f8e379b74e169b98fb7c3c0ba5b4`. Rule list: `[]` (Enabled: `True`). {{(pid=87058) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:56:57.286857 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.fernet_utils [None req-79bbf9c8-116b-4338-8cfd-cadbed286ff1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:56:57.362713 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 1151/2293] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 1 16:56:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:56:57.373376 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f801d477-c5e7-4165-a547-be30ef834d84 admin admin] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:56:57.374768 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.fernet_utils [None req-f801d477-c5e7-4165-a547-be30ef834d84 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87057) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:56:57.439267 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f801d477-c5e7-4165-a547-be30ef834d84 admin admin] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:56:57.443096 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-670d87b9-3f6d-4373-b090-55551e5782b9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d9bd8d649154c979be3744bfd016f95', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b992d6a65df4bf9b27c13192d57c5f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:56:57.443807 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670d87b9-3f6d-4373-b090-55551e5782b9 service neutron] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:56:57.444175 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670d87b9-3f6d-4373-b090-55551e5782b9 service neutron] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:56:57.444514 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670d87b9-3f6d-4373-b090-55551e5782b9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:56:57.454656 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-670d87b9-3f6d-4373-b090-55551e5782b9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:56:57.456177 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-670d87b9-3f6d-4373-b090-55551e5782b9 service neutron] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:56:57.467027 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 1143/2294] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 1 16:56:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 16:56:57.995461 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79bbf9c8-116b-4338-8cfd-cadbed286ff1 None None] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:56:57.998424 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.fernet_utils [None req-79bbf9c8-116b-4338-8cfd-cadbed286ff1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:56:58.020645 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79bbf9c8-116b-4338-8cfd-cadbed286ff1 None None] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:56:58.029594 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ec91bf5-57d9-4146-9235-7ef10845c818 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:56:58.031790 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec91bf5-57d9-4146-9235-7ef10845c818 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:56:58.034125 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec91bf5-57d9-4146-9235-7ef10845c818 admin admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:56:58.034661 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec91bf5-57d9-4146-9235-7ef10845c818 admin admin] PATH_INFO: `/v3/projects/c78792a8e5624576a1d5ecde8c2e0767` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:56:58.038326 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ec91bf5-57d9-4146-9235-7ef10845c818 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c78792a8e5624576a1d5ecde8c2e0767)` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:56:58.043797 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ec91bf5-57d9-4146-9235-7ef10845c818 admin admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:56:58.144363 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG dbcounter [-] [87058] Writing DB stats keystone:INSERT=3,keystone:DELETE=2,keystone:SELECT=95 {{(pid=87058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:56:58.162798 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.notifications [None req-9ec91bf5-57d9-4146-9235-7ef10845c818 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c78792a8e5624576a1d5ecde8c2e0767', 'request_id': 'req-9ec91bf5-57d9-4146-9235-7ef10845c818'} {{(pid=87058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:56:58.175141 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.notifications [None req-9ec91bf5-57d9-4146-9235-7ef10845c818 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c78792a8e5624576a1d5ecde8c2e0767', 'request_id': 'req-9ec91bf5-57d9-4146-9235-7ef10845c818'} {{(pid=87058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:56:58.181315 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.notifications [None req-9ec91bf5-57d9-4146-9235-7ef10845c818 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c78792a8e5624576a1d5ecde8c2e0767', 'request_id': 'req-9ec91bf5-57d9-4146-9235-7ef10845c818'} {{(pid=87058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:56:58.257415 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.notifications [None req-9ec91bf5-57d9-4146-9235-7ef10845c818 admin admin] The token cache is being invalidate because project c78792a8e5624576a1d5ecde8c2e0767 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87058) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 16:56:58.257779 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.notifications [None req-9ec91bf5-57d9-4146-9235-7ef10845c818 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:56:58.258510 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.notifications [None req-9ec91bf5-57d9-4146-9235-7ef10845c818 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:56:58.264220 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 1152/2295] 10.222.0.20 () {58 vars in 1233 bytes} [Mon Aug 1 16:56:57 2022] DELETE /identity/v3/projects/c78792a8e5624576a1d5ecde8c2e0767 => generated 0 bytes in 266 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 16:56:58.264811 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-670d87b9-3f6d-4373-b090-55551e5782b9 service neutron] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:56:58.276675 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-670d87b9-3f6d-4373-b090-55551e5782b9 service neutron] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:56:58.286312 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG dbcounter [-] [87057] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87057) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:56:58.354705 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3916a712-6f35-4281-be4b-4dab7c75b283 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:56:58.355372 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3916a712-6f35-4281-be4b-4dab7c75b283 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:56:58.355682 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3916a712-6f35-4281-be4b-4dab7c75b283 admin admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:56:58.355957 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3916a712-6f35-4281-be4b-4dab7c75b283 admin admin] PATH_INFO: `/v3/users/587eb18dd944444e872bfa25759f8d78` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:56:58.357684 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3916a712-6f35-4281-be4b-4dab7c75b283 admin admin] RBAC: Authorizing `identity:delete_user(user_id=587eb18dd944444e872bfa25759f8d78)` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:56:58.360387 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3916a712-6f35-4281-be4b-4dab7c75b283 admin admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:56:58.549058 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.notifications [None req-3916a712-6f35-4281-be4b-4dab7c75b283 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '587eb18dd944444e872bfa25759f8d78'} {{(pid=87057) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:56:58.550221 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.notifications [None req-3916a712-6f35-4281-be4b-4dab7c75b283 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '587eb18dd944444e872bfa25759f8d78'} {{(pid=87057) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:56:58.633515 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.notifications [None req-3916a712-6f35-4281-be4b-4dab7c75b283 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '587eb18dd944444e872bfa25759f8d78'} {{(pid=87057) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:56:58.634761 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.notifications [None req-3916a712-6f35-4281-be4b-4dab7c75b283 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '587eb18dd944444e872bfa25759f8d78'} {{(pid=87057) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:56:58.643030 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.notifications [None req-3916a712-6f35-4281-be4b-4dab7c75b283 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '587eb18dd944444e872bfa25759f8d78'} {{(pid=87057) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:56:58.663703 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 1144/2296] 10.222.0.20 () {58 vars in 1224 bytes} [Mon Aug 1 16:56:58 2022] DELETE /identity/v3/users/587eb18dd944444e872bfa25759f8d78 => generated 0 bytes in 399 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 16:56:59.330955 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ec91bf5-57d9-4146-9235-7ef10845c818 admin admin] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:56:59.332535 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.fernet_utils [None req-9ec91bf5-57d9-4146-9235-7ef10845c818 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:56:59.350720 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ec91bf5-57d9-4146-9235-7ef10845c818 admin admin] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:56:59.421884 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a041ec52-eee7-4b0a-a194-df27a76201e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:56:59.422871 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a041ec52-eee7-4b0a-a194-df27a76201e1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:56:59.423309 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a041ec52-eee7-4b0a-a194-df27a76201e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:56:59.423733 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a041ec52-eee7-4b0a-a194-df27a76201e1 admin admin] PATH_INFO: `/v3/projects/94ba30eccee547f8a240dd4644631343` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:56:59.425810 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a041ec52-eee7-4b0a-a194-df27a76201e1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=94ba30eccee547f8a240dd4644631343)` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:56:59.428312 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a041ec52-eee7-4b0a-a194-df27a76201e1 admin admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:56:59.552176 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.notifications [None req-a041ec52-eee7-4b0a-a194-df27a76201e1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '94ba30eccee547f8a240dd4644631343', 'request_id': 'req-a041ec52-eee7-4b0a-a194-df27a76201e1'} {{(pid=87058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:56:59.560069 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.notifications [None req-a041ec52-eee7-4b0a-a194-df27a76201e1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '94ba30eccee547f8a240dd4644631343', 'request_id': 'req-a041ec52-eee7-4b0a-a194-df27a76201e1'} {{(pid=87058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:56:59.566541 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.notifications [None req-a041ec52-eee7-4b0a-a194-df27a76201e1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '94ba30eccee547f8a240dd4644631343', 'request_id': 'req-a041ec52-eee7-4b0a-a194-df27a76201e1'} {{(pid=87058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:56:59.646301 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.notifications [None req-a041ec52-eee7-4b0a-a194-df27a76201e1 admin admin] The token cache is being invalidate because project 94ba30eccee547f8a240dd4644631343 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87058) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 16:56:59.646592 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.notifications [None req-a041ec52-eee7-4b0a-a194-df27a76201e1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:56:59.647261 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.notifications [None req-a041ec52-eee7-4b0a-a194-df27a76201e1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:56:59.649093 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 1153/2297] 10.222.0.20 () {58 vars in 1233 bytes} [Mon Aug 1 16:56:59 2022] DELETE /identity/v3/projects/94ba30eccee547f8a240dd4644631343 => generated 0 bytes in 319 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 16:56:59.655770 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6449e8-cf20-4dab-aefb-d23cedfe4b65 None None] REQUEST_METHOD: `POST` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:56:59.656619 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6449e8-cf20-4dab-aefb-d23cedfe4b65 None None] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:56:59.657044 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6449e8-cf20-4dab-aefb-d23cedfe4b65 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:56:59.684621 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.auth.core [None req-3c6449e8-cf20-4dab-aefb-d23cedfe4b65 None None] MFA Rules not processed for user `1007f8e379b74e169b98fb7c3c0ba5b4`. Rule list: `[]` (Enabled: `True`). {{(pid=87057) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:56:59.685926 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.fernet_utils [None req-3c6449e8-cf20-4dab-aefb-d23cedfe4b65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87057) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:56:59.797948 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 1145/2298] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 1 16:56:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:56:59.881940 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55176e72-956f-4557-81cc-eef31e841ef1 None None] REQUEST_METHOD: `POST` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:56:59.882301 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55176e72-956f-4557-81cc-eef31e841ef1 None None] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:56:59.882582 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55176e72-956f-4557-81cc-eef31e841ef1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:56:59.926535 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.auth.core [None req-55176e72-956f-4557-81cc-eef31e841ef1 None None] MFA Rules not processed for user `1007f8e379b74e169b98fb7c3c0ba5b4`. Rule list: `[]` (Enabled: `True`). {{(pid=87058) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:56:59.928089 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.fernet_utils [None req-55176e72-956f-4557-81cc-eef31e841ef1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:56:59.944631 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 1154/2299] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 1 16:56:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 63 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:56:59.946006 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c6449e8-cf20-4dab-aefb-d23cedfe4b65 None None] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:56:59.967529 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c6449e8-cf20-4dab-aefb-d23cedfe4b65 None None] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:56:59.979219 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac222e8-fb20-420b-9d33-4018fef9d913 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:56:59.986125 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac222e8-fb20-420b-9d33-4018fef9d913 admin admin] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:56:59.986125 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac222e8-fb20-420b-9d33-4018fef9d913 admin admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:56:59.986125 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac222e8-fb20-420b-9d33-4018fef9d913 admin admin] PATH_INFO: `/v3/domains` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:56:59.986125 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ac222e8-fb20-420b-9d33-4018fef9d913 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87057) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:56:59.986125 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ac222e8-fb20-420b-9d33-4018fef9d913 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:56:59.986125 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ac222e8-fb20-420b-9d33-4018fef9d913 admin admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:57:00.000828 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 1146/2300] 10.222.0.20 () {58 vars in 1153 bytes} [Mon Aug 1 16:56:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 16:57:00.011891 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55176e72-956f-4557-81cc-eef31e841ef1 None None] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:57:00.023510 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55176e72-956f-4557-81cc-eef31e841ef1 None None] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:57:00.031206 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c83134c5-5a17-4fcb-96b9-776784bbae50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:57:00.031839 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83134c5-5a17-4fcb-96b9-776784bbae50 admin admin] REQUEST_METHOD: `POST` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:57:00.032129 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83134c5-5a17-4fcb-96b9-776784bbae50 admin admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:57:00.032405 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83134c5-5a17-4fcb-96b9-776784bbae50 admin admin] PATH_INFO: `/v3/projects` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:57:00.033355 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c83134c5-5a17-4fcb-96b9-776784bbae50 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:57:00.035165 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c83134c5-5a17-4fcb-96b9-776784bbae50 admin admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:57:00.110593 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1035622a-60d6-405b-a492-2bdf5d821440 None None] REQUEST_METHOD: `POST` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:57:00.110886 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1035622a-60d6-405b-a492-2bdf5d821440 None None] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:57:00.111163 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1035622a-60d6-405b-a492-2bdf5d821440 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:57:00.143301 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.auth.core [None req-1035622a-60d6-405b-a492-2bdf5d821440 None None] MFA Rules not processed for user `48cc0715e7b04f8a8c9f0d8c455bd4c6`. Rule list: `[]` (Enabled: `True`). {{(pid=87057) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:57:00.144584 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.fernet_utils [None req-1035622a-60d6-405b-a492-2bdf5d821440 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87057) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:57:00.149576 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG dbcounter [-] [87057] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87057) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:57:00.222843 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 1155/2301] 10.222.0.20 () {60 vars in 1153 bytes} [Mon Aug 1 16:57:00 2022] POST /identity/v3/projects => generated 385 bytes in 212 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 16:57:00.226674 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c83134c5-5a17-4fcb-96b9-776784bbae50 admin admin] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:57:00.234771 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 1147/2302] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 1 16:57:00 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 16:57:00.243180 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1035622a-60d6-405b-a492-2bdf5d821440 None None] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:57:00.246015 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.fernet_utils [None req-1035622a-60d6-405b-a492-2bdf5d821440 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87057) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:57:00.250243 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG dbcounter [-] [87058] Writing DB stats keystone:DELETE=13,keystone:SELECT=84,keystone:INSERT=3 {{(pid=87058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:57:00.296262 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c83134c5-5a17-4fcb-96b9-776784bbae50 admin admin] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:57:00.303143 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da70600-b002-4fec-8650-c8248494b9db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:57:00.303781 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da70600-b002-4fec-8650-c8248494b9db admin admin] REQUEST_METHOD: `POST` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:57:00.304063 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da70600-b002-4fec-8650-c8248494b9db admin admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:57:00.304339 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da70600-b002-4fec-8650-c8248494b9db admin admin] PATH_INFO: `/v3/users` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:57:00.305011 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5da70600-b002-4fec-8650-c8248494b9db admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:57:00.308010 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5da70600-b002-4fec-8650-c8248494b9db admin admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:57:00.327461 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1035622a-60d6-405b-a492-2bdf5d821440 None None] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:57:00.391861 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7967c17-7a52-4dce-b1c6-b98f4bd7a9a7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6533106d03eb4ca185a8839336d2aff9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b992d6a65df4bf9b27c13192d57c5f0', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:57:00.392492 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7967c17-7a52-4dce-b1c6-b98f4bd7a9a7 service nova] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:57:00.392776 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7967c17-7a52-4dce-b1c6-b98f4bd7a9a7 service nova] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:57:00.393054 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7967c17-7a52-4dce-b1c6-b98f4bd7a9a7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:57:00.406391 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 1156/2303] 10.222.0.20 () {60 vars in 1144 bytes} [Mon Aug 1 16:57:00 2022] POST /identity/v3/users => generated 293 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 16:57:00.410490 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da70600-b002-4fec-8650-c8248494b9db admin admin] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:57:00.421789 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da70600-b002-4fec-8650-c8248494b9db admin admin] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:57:00.429903 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f818febd-d0af-424c-b5c8-d5fcc9df4d7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:57:00.430742 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f818febd-d0af-424c-b5c8-d5fcc9df4d7b admin admin] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:57:00.432212 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f818febd-d0af-424c-b5c8-d5fcc9df4d7b admin admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:57:00.432212 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f818febd-d0af-424c-b5c8-d5fcc9df4d7b admin admin] PATH_INFO: `/v3/roles` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:57:00.432998 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f818febd-d0af-424c-b5c8-d5fcc9df4d7b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:57:00.434758 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f818febd-d0af-424c-b5c8-d5fcc9df4d7b admin admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:57:00.446191 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 1157/2304] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Aug 1 16:57:00 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 01 16:57:00.450963 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f818febd-d0af-424c-b5c8-d5fcc9df4d7b admin admin] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:57:00.460301 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7967c17-7a52-4dce-b1c6-b98f4bd7a9a7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:57:00.461984 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f818febd-d0af-424c-b5c8-d5fcc9df4d7b admin admin] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:57:00.462888 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7967c17-7a52-4dce-b1c6-b98f4bd7a9a7 service nova] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:57:00.467881 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29f8b566-7d4e-45b0-9424-78bf8eb01f6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:57:00.468217 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f8b566-7d4e-45b0-9424-78bf8eb01f6d admin admin] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:57:00.468500 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f8b566-7d4e-45b0-9424-78bf8eb01f6d admin admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:57:00.468778 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f8b566-7d4e-45b0-9424-78bf8eb01f6d admin admin] PATH_INFO: `/v3/roles` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:57:00.469492 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29f8b566-7d4e-45b0-9424-78bf8eb01f6d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:57:00.471131 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29f8b566-7d4e-45b0-9424-78bf8eb01f6d admin admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:57:00.473590 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 1148/2305] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 1 16:57:00 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 232 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 16:57:00.487516 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 1158/2306] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Aug 1 16:57:00 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 01 16:57:00.491975 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7967c17-7a52-4dce-b1c6-b98f4bd7a9a7 service nova] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:57:00.503984 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7967c17-7a52-4dce-b1c6-b98f4bd7a9a7 service nova] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:57:00.509849 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ce02ae2-ec7b-4033-85aa-7d3a2a0f9dde admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:57:00.510537 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce02ae2-ec7b-4033-85aa-7d3a2a0f9dde admin admin] REQUEST_METHOD: `PUT` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:57:00.510819 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce02ae2-ec7b-4033-85aa-7d3a2a0f9dde admin admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:57:00.511098 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce02ae2-ec7b-4033-85aa-7d3a2a0f9dde admin admin] PATH_INFO: `/v3/projects/34f2fca7676641288c2c7f85bdb8af12/users/414eaae65c4547a092023311a44aa261/roles/0999d3da63a545f18ff3252afdc90518` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:57:00.540620 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG dbcounter [-] [87057] Writing DB stats keystone:SELECT=100 {{(pid=87057) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:57:00.543659 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ce02ae2-ec7b-4033-85aa-7d3a2a0f9dde admin admin] RBAC: Authorizing `identity:create_grant(project_id=34f2fca7676641288c2c7f85bdb8af12, user_id=414eaae65c4547a092023311a44aa261, role_id=0999d3da63a545f18ff3252afdc90518)` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:57:00.545941 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ce02ae2-ec7b-4033-85aa-7d3a2a0f9dde admin admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:57:00.617692 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 1149/2307] 10.222.0.20 () {60 vars in 1483 bytes} [Mon Aug 1 16:57:00 2022] PUT /identity/v3/projects/34f2fca7676641288c2c7f85bdb8af12/users/414eaae65c4547a092023311a44aa261/roles/0999d3da63a545f18ff3252afdc90518 => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 16:57:00.628994 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29f75d0-ddc6-4ab6-9ac8-7094afa091ae None None] REQUEST_METHOD: `POST` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:57:00.629399 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29f75d0-ddc6-4ab6-9ac8-7094afa091ae None None] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:57:00.629983 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29f75d0-ddc6-4ab6-9ac8-7094afa091ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:57:00.674293 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.auth.core [None req-d29f75d0-ddc6-4ab6-9ac8-7094afa091ae None None] MFA Rules not processed for user `414eaae65c4547a092023311a44aa261`. Rule list: `[]` (Enabled: `True`). {{(pid=87058) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:57:00.675601 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.fernet_utils [None req-d29f75d0-ddc6-4ab6-9ac8-7094afa091ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:57:00.755602 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 1159/2308] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 1 16:57:00 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:57:00.767887 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ce02ae2-ec7b-4033-85aa-7d3a2a0f9dde admin admin] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:57:00.770158 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.fernet_utils [None req-9ce02ae2-ec7b-4033-85aa-7d3a2a0f9dde admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87057) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:57:00.865678 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ce02ae2-ec7b-4033-85aa-7d3a2a0f9dde admin admin] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:57:00.871912 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae5a84e-13e6-47a8-97e8-ba5e3b95d5d7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25970b52ca15448fa1abb1f974d64038', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b992d6a65df4bf9b27c13192d57c5f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:57:00.872579 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae5a84e-13e6-47a8-97e8-ba5e3b95d5d7 service cinder] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:57:00.872881 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae5a84e-13e6-47a8-97e8-ba5e3b95d5d7 service cinder] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:57:00.873158 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae5a84e-13e6-47a8-97e8-ba5e3b95d5d7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:57:00.883265 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ae5a84e-13e6-47a8-97e8-ba5e3b95d5d7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:57:00.885303 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ae5a84e-13e6-47a8-97e8-ba5e3b95d5d7 service cinder] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:57:00.894758 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 1150/2309] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 1 16:57:00 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 16:57:03.052341 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bbb825e-8753-4417-be8e-8bd46592bda3 None None] REQUEST_METHOD: `POST` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:57:03.052812 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bbb825e-8753-4417-be8e-8bd46592bda3 None None] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:57:03.052888 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bbb825e-8753-4417-be8e-8bd46592bda3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:57:03.084463 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.auth.core [None req-5bbb825e-8753-4417-be8e-8bd46592bda3 None None] MFA Rules not processed for user `48cc0715e7b04f8a8c9f0d8c455bd4c6`. Rule list: `[]` (Enabled: `True`). {{(pid=87058) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:57:03.089395 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.fernet_utils [None req-5bbb825e-8753-4417-be8e-8bd46592bda3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:57:03.198591 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 1160/2310] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 1 16:57:03 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:57:10.890820 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG dbcounter [-] [87057] Writing DB stats keystone:SELECT=44,keystone:INSERT=1 {{(pid=87057) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:57:13.182223 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG dbcounter [-] [87058] Writing DB stats keystone:SELECT=89,keystone:INSERT=3 {{(pid=87058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:57:16.806212 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae5a84e-13e6-47a8-97e8-ba5e3b95d5d7 service cinder] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:57:16.881108 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae5a84e-13e6-47a8-97e8-ba5e3b95d5d7 service cinder] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:57:16.886003 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9c3f7f8-861c-471e-9d02-4c311242076a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:57:16.886688 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c3f7f8-861c-471e-9d02-4c311242076a admin admin] REQUEST_METHOD: `DELETE` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:57:16.886953 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c3f7f8-861c-471e-9d02-4c311242076a admin admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:57:16.887229 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c3f7f8-861c-471e-9d02-4c311242076a admin admin] PATH_INFO: `/v3/users/414eaae65c4547a092023311a44aa261` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:57:16.888447 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9c3f7f8-861c-471e-9d02-4c311242076a admin admin] RBAC: Authorizing `identity:delete_user(user_id=414eaae65c4547a092023311a44aa261)` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:57:16.890423 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9c3f7f8-861c-471e-9d02-4c311242076a admin admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:57:17.050312 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bbb825e-8753-4417-be8e-8bd46592bda3 None None] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:57:17.051746 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.fernet_utils [None req-5bbb825e-8753-4417-be8e-8bd46592bda3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:57:17.065802 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bbb825e-8753-4417-be8e-8bd46592bda3 None None] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:57:17.071244 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb497b2c-b9f8-4e76-a014-8377e57f87f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:57:17.072053 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb497b2c-b9f8-4e76-a014-8377e57f87f8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:57:17.072332 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb497b2c-b9f8-4e76-a014-8377e57f87f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:57:17.072603 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb497b2c-b9f8-4e76-a014-8377e57f87f8 admin admin] PATH_INFO: `/v3/users/dda13b874aab4c119d70a76aa43bf924` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:57:17.074612 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb497b2c-b9f8-4e76-a014-8377e57f87f8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=dda13b874aab4c119d70a76aa43bf924)` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:57:17.077222 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb497b2c-b9f8-4e76-a014-8377e57f87f8 admin admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:57:17.132566 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.notifications [None req-e9c3f7f8-861c-471e-9d02-4c311242076a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '414eaae65c4547a092023311a44aa261'} {{(pid=87057) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:57:17.133229 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.notifications [None req-e9c3f7f8-861c-471e-9d02-4c311242076a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '414eaae65c4547a092023311a44aa261'} {{(pid=87057) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:57:17.215750 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.notifications [None req-e9c3f7f8-861c-471e-9d02-4c311242076a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '414eaae65c4547a092023311a44aa261'} {{(pid=87057) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:57:17.216360 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.notifications [None req-e9c3f7f8-861c-471e-9d02-4c311242076a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '414eaae65c4547a092023311a44aa261'} {{(pid=87057) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:57:17.224955 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.notifications [None req-e9c3f7f8-861c-471e-9d02-4c311242076a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '414eaae65c4547a092023311a44aa261'} {{(pid=87057) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:57:17.243265 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 1151/2311] 10.222.0.20 () {58 vars in 1224 bytes} [Mon Aug 1 16:57:16 2022] DELETE /identity/v3/users/414eaae65c4547a092023311a44aa261 => generated 0 bytes in 438 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 16:57:17.248475 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc59aef5-41d1-4f1c-8e89-b6d8dc876788 None None] REQUEST_METHOD: `POST` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:57:17.248762 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc59aef5-41d1-4f1c-8e89-b6d8dc876788 None None] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:57:17.249039 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc59aef5-41d1-4f1c-8e89-b6d8dc876788 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:57:17.270474 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.auth.core [None req-fc59aef5-41d1-4f1c-8e89-b6d8dc876788 None None] MFA Rules not processed for user `1007f8e379b74e169b98fb7c3c0ba5b4`. Rule list: `[]` (Enabled: `True`). {{(pid=87057) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:57:17.271710 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.fernet_utils [None req-fc59aef5-41d1-4f1c-8e89-b6d8dc876788 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87057) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:57:17.279060 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.notifications [None req-bb497b2c-b9f8-4e76-a014-8377e57f87f8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dda13b874aab4c119d70a76aa43bf924'} {{(pid=87058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:57:17.281523 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.notifications [None req-bb497b2c-b9f8-4e76-a014-8377e57f87f8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dda13b874aab4c119d70a76aa43bf924'} {{(pid=87058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:57:17.283110 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.notifications [None req-bb497b2c-b9f8-4e76-a014-8377e57f87f8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dda13b874aab4c119d70a76aa43bf924'} {{(pid=87058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:57:17.298875 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.notifications [None req-bb497b2c-b9f8-4e76-a014-8377e57f87f8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dda13b874aab4c119d70a76aa43bf924'} {{(pid=87058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:57:17.308973 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.notifications [None req-bb497b2c-b9f8-4e76-a014-8377e57f87f8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dda13b874aab4c119d70a76aa43bf924'} {{(pid=87058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:57:17.337157 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 1152/2312] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 1 16:57:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 16:57:17.352920 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc59aef5-41d1-4f1c-8e89-b6d8dc876788 None None] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:57:17.354986 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.fernet_utils [None req-fc59aef5-41d1-4f1c-8e89-b6d8dc876788 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87057) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:57:17.378107 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG dbcounter [-] [87057] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87057) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:57:17.410985 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 1161/2313] 10.222.0.20 () {58 vars in 1224 bytes} [Mon Aug 1 16:57:17 2022] DELETE /identity/v3/users/dda13b874aab4c119d70a76aa43bf924 => generated 0 bytes in 361 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 16:57:17.414793 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc59aef5-41d1-4f1c-8e89-b6d8dc876788 None None] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:57:17.419054 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91cfb5c8-4d6e-413c-a14e-3b4aa3d3042e None None] REQUEST_METHOD: `POST` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:57:17.419054 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91cfb5c8-4d6e-413c-a14e-3b4aa3d3042e None None] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:57:17.419054 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91cfb5c8-4d6e-413c-a14e-3b4aa3d3042e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:57:17.441428 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.auth.core [None req-91cfb5c8-4d6e-413c-a14e-3b4aa3d3042e None None] MFA Rules not processed for user `1007f8e379b74e169b98fb7c3c0ba5b4`. Rule list: `[]` (Enabled: `True`). {{(pid=87058) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:57:17.442726 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.fernet_utils [None req-91cfb5c8-4d6e-413c-a14e-3b4aa3d3042e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:57:17.475771 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23f87a5b-17e2-4ee9-b47d-ad7c70f4fb23 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d9bd8d649154c979be3744bfd016f95', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b992d6a65df4bf9b27c13192d57c5f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:57:17.476396 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f87a5b-17e2-4ee9-b47d-ad7c70f4fb23 service neutron] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:57:17.476689 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f87a5b-17e2-4ee9-b47d-ad7c70f4fb23 service neutron] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:57:17.476946 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f87a5b-17e2-4ee9-b47d-ad7c70f4fb23 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:57:17.513168 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 1162/2314] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 1 16:57:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:57:17.524338 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91cfb5c8-4d6e-413c-a14e-3b4aa3d3042e None None] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:57:17.525220 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.fernet_utils [None req-91cfb5c8-4d6e-413c-a14e-3b4aa3d3042e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:57:17.540595 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91cfb5c8-4d6e-413c-a14e-3b4aa3d3042e None None] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:57:17.546411 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39adb47c-6bb7-45ff-85d9-9119e44429af service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d9bd8d649154c979be3744bfd016f95', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b992d6a65df4bf9b27c13192d57c5f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:57:17.546411 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39adb47c-6bb7-45ff-85d9-9119e44429af service neutron] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:57:17.546411 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39adb47c-6bb7-45ff-85d9-9119e44429af service neutron] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:57:17.546843 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39adb47c-6bb7-45ff-85d9-9119e44429af service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:57:17.549697 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23f87a5b-17e2-4ee9-b47d-ad7c70f4fb23 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:57:17.551643 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23f87a5b-17e2-4ee9-b47d-ad7c70f4fb23 service neutron] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:57:17.558529 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39adb47c-6bb7-45ff-85d9-9119e44429af service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:57:17.559954 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39adb47c-6bb7-45ff-85d9-9119e44429af service neutron] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:57:17.563442 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 1153/2315] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 1 16:57:17 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 211 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 01 16:57:17.570890 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 1163/2316] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 1 16:57:17 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 16:57:18.068533 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23f87a5b-17e2-4ee9-b47d-ad7c70f4fb23 service neutron] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:57:18.070745 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.fernet_utils [None req-23f87a5b-17e2-4ee9-b47d-ad7c70f4fb23 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87057) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:57:18.092241 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23f87a5b-17e2-4ee9-b47d-ad7c70f4fb23 service neutron] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:57:18.103669 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff8424f4-3de7-41fe-907e-8533d57346cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:57:18.103669 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8424f4-3de7-41fe-907e-8533d57346cf admin admin] REQUEST_METHOD: `DELETE` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:57:18.103669 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8424f4-3de7-41fe-907e-8533d57346cf admin admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:57:18.103669 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8424f4-3de7-41fe-907e-8533d57346cf admin admin] PATH_INFO: `/v3/projects/bae76503dca34892b404b3f1db8fea83` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:57:18.105020 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff8424f4-3de7-41fe-907e-8533d57346cf admin admin] RBAC: Authorizing `identity:delete_project(project_id=bae76503dca34892b404b3f1db8fea83)` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:57:18.106801 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff8424f4-3de7-41fe-907e-8533d57346cf admin admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:57:18.162440 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39adb47c-6bb7-45ff-85d9-9119e44429af service neutron] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:57:18.163387 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.fernet_utils [None req-39adb47c-6bb7-45ff-85d9-9119e44429af service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:57:18.181387 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39adb47c-6bb7-45ff-85d9-9119e44429af service neutron] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:57:18.186512 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a4f8bc2-aabc-4c3c-a2a8-d53fbaa8cf63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:57:18.188262 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4f8bc2-aabc-4c3c-a2a8-d53fbaa8cf63 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:57:18.188262 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4f8bc2-aabc-4c3c-a2a8-d53fbaa8cf63 admin admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:57:18.188262 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4f8bc2-aabc-4c3c-a2a8-d53fbaa8cf63 admin admin] PATH_INFO: `/v3/projects/34f2fca7676641288c2c7f85bdb8af12` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:57:18.189016 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a4f8bc2-aabc-4c3c-a2a8-d53fbaa8cf63 admin admin] RBAC: Authorizing `identity:delete_project(project_id=34f2fca7676641288c2c7f85bdb8af12)` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:57:18.191075 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a4f8bc2-aabc-4c3c-a2a8-d53fbaa8cf63 admin admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:57:18.252393 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.notifications [None req-ff8424f4-3de7-41fe-907e-8533d57346cf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bae76503dca34892b404b3f1db8fea83', 'request_id': 'req-ff8424f4-3de7-41fe-907e-8533d57346cf'} {{(pid=87057) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:57:18.258827 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.notifications [None req-ff8424f4-3de7-41fe-907e-8533d57346cf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bae76503dca34892b404b3f1db8fea83', 'request_id': 'req-ff8424f4-3de7-41fe-907e-8533d57346cf'} {{(pid=87057) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:57:18.265132 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.notifications [None req-ff8424f4-3de7-41fe-907e-8533d57346cf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bae76503dca34892b404b3f1db8fea83', 'request_id': 'req-ff8424f4-3de7-41fe-907e-8533d57346cf'} {{(pid=87057) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:57:18.296093 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.notifications [None req-2a4f8bc2-aabc-4c3c-a2a8-d53fbaa8cf63 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '34f2fca7676641288c2c7f85bdb8af12', 'request_id': 'req-2a4f8bc2-aabc-4c3c-a2a8-d53fbaa8cf63'} {{(pid=87058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:57:18.302371 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.notifications [None req-2a4f8bc2-aabc-4c3c-a2a8-d53fbaa8cf63 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '34f2fca7676641288c2c7f85bdb8af12', 'request_id': 'req-2a4f8bc2-aabc-4c3c-a2a8-d53fbaa8cf63'} {{(pid=87058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:57:18.303534 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG dbcounter [-] [87058] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=87058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:57:18.308902 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.notifications [None req-2a4f8bc2-aabc-4c3c-a2a8-d53fbaa8cf63 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '34f2fca7676641288c2c7f85bdb8af12', 'request_id': 'req-2a4f8bc2-aabc-4c3c-a2a8-d53fbaa8cf63'} {{(pid=87058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:57:18.340758 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.notifications [None req-ff8424f4-3de7-41fe-907e-8533d57346cf admin admin] The token cache is being invalidate because project bae76503dca34892b404b3f1db8fea83 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87057) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 16:57:18.341048 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.notifications [None req-ff8424f4-3de7-41fe-907e-8533d57346cf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87057) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:57:18.341716 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.notifications [None req-ff8424f4-3de7-41fe-907e-8533d57346cf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87057) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:57:18.343395 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 1154/2317] 10.222.0.20 () {58 vars in 1233 bytes} [Mon Aug 1 16:57:18 2022] DELETE /identity/v3/projects/bae76503dca34892b404b3f1db8fea83 => generated 0 bytes in 275 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 16:57:18.382773 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.notifications [None req-2a4f8bc2-aabc-4c3c-a2a8-d53fbaa8cf63 admin admin] The token cache is being invalidate because project 34f2fca7676641288c2c7f85bdb8af12 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87058) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 16:57:18.383068 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.notifications [None req-2a4f8bc2-aabc-4c3c-a2a8-d53fbaa8cf63 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:57:18.383757 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.notifications [None req-2a4f8bc2-aabc-4c3c-a2a8-d53fbaa8cf63 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:57:18.386393 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 1164/2318] 10.222.0.20 () {58 vars in 1233 bytes} [Mon Aug 1 16:57:18 2022] DELETE /identity/v3/projects/34f2fca7676641288c2c7f85bdb8af12 => generated 0 bytes in 224 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 16:57:18.391980 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115d3991-8b07-4efe-b169-bb958c120089 None None] REQUEST_METHOD: `POST` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:57:18.392263 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115d3991-8b07-4efe-b169-bb958c120089 None None] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:57:18.392564 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115d3991-8b07-4efe-b169-bb958c120089 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:57:18.414319 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.auth.core [None req-115d3991-8b07-4efe-b169-bb958c120089 None None] MFA Rules not processed for user `1007f8e379b74e169b98fb7c3c0ba5b4`. Rule list: `[]` (Enabled: `True`). {{(pid=87057) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:57:18.415613 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.fernet_utils [None req-115d3991-8b07-4efe-b169-bb958c120089 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87057) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:57:18.420581 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG dbcounter [-] [87057] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=87057) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:57:18.484358 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 1155/2319] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 1 16:57:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:57:18.493835 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33dcc660-9d85-4767-a01a-34496cc06262 None None] REQUEST_METHOD: `POST` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:57:18.494303 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33dcc660-9d85-4767-a01a-34496cc06262 None None] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:57:18.494668 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33dcc660-9d85-4767-a01a-34496cc06262 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:57:18.542596 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.auth.core [None req-33dcc660-9d85-4767-a01a-34496cc06262 None None] MFA Rules not processed for user `1007f8e379b74e169b98fb7c3c0ba5b4`. Rule list: `[]` (Enabled: `True`). {{(pid=87058) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:57:18.543657 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.fernet_utils [None req-33dcc660-9d85-4767-a01a-34496cc06262 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:57:18.559699 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 1165/2320] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 1 16:57:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 67 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 16:57:18.563762 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-115d3991-8b07-4efe-b169-bb958c120089 None None] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:57:18.579385 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-115d3991-8b07-4efe-b169-bb958c120089 None None] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:57:18.584301 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a4be7ee-500c-4f3f-a3fc-59f6bad4b284 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:57:18.585571 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4be7ee-500c-4f3f-a3fc-59f6bad4b284 admin admin] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:57:18.585571 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4be7ee-500c-4f3f-a3fc-59f6bad4b284 admin admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:57:18.585823 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4be7ee-500c-4f3f-a3fc-59f6bad4b284 admin admin] PATH_INFO: `/v3/domains` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:57:18.586475 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a4be7ee-500c-4f3f-a3fc-59f6bad4b284 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87057) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:57:18.586780 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a4be7ee-500c-4f3f-a3fc-59f6bad4b284 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:57:18.588381 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a4be7ee-500c-4f3f-a3fc-59f6bad4b284 admin admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:57:18.603246 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 1156/2321] 10.222.0.20 () {58 vars in 1153 bytes} [Mon Aug 1 16:57:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:57:18.607032 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33dcc660-9d85-4767-a01a-34496cc06262 None None] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:57:18.619112 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33dcc660-9d85-4767-a01a-34496cc06262 None None] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:57:18.623936 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dffd3c3c-2670-468e-83a4-9a3f97baa9f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:57:18.624547 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dffd3c3c-2670-468e-83a4-9a3f97baa9f5 admin admin] REQUEST_METHOD: `POST` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:57:18.624821 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dffd3c3c-2670-468e-83a4-9a3f97baa9f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:57:18.625087 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dffd3c3c-2670-468e-83a4-9a3f97baa9f5 admin admin] PATH_INFO: `/v3/projects` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:57:18.625751 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dffd3c3c-2670-468e-83a4-9a3f97baa9f5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:57:18.627779 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dffd3c3c-2670-468e-83a4-9a3f97baa9f5 admin admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:57:18.704037 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 1166/2322] 10.222.0.20 () {60 vars in 1153 bytes} [Mon Aug 1 16:57:18 2022] POST /identity/v3/projects => generated 383 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 16:57:18.708843 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a4be7ee-500c-4f3f-a3fc-59f6bad4b284 admin admin] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:57:18.720975 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a4be7ee-500c-4f3f-a3fc-59f6bad4b284 admin admin] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:57:18.791620 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a208883-41b7-476e-96eb-70daae3f0cff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:57:18.792673 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a208883-41b7-476e-96eb-70daae3f0cff admin admin] REQUEST_METHOD: `POST` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:57:18.792950 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a208883-41b7-476e-96eb-70daae3f0cff admin admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:57:18.793222 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a208883-41b7-476e-96eb-70daae3f0cff admin admin] PATH_INFO: `/v3/users` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:57:18.793889 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a208883-41b7-476e-96eb-70daae3f0cff admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:57:18.795964 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a208883-41b7-476e-96eb-70daae3f0cff admin admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:57:18.892790 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 1157/2323] 10.222.0.20 () {60 vars in 1144 bytes} [Mon Aug 1 16:57:18 2022] POST /identity/v3/users => generated 292 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 16:57:18.894427 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dffd3c3c-2670-468e-83a4-9a3f97baa9f5 admin admin] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:57:18.911700 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dffd3c3c-2670-468e-83a4-9a3f97baa9f5 admin admin] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:57:18.923590 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb45d34-6a25-46d8-a394-ab4f127465a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:57:18.924270 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb45d34-6a25-46d8-a394-ab4f127465a5 admin admin] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:57:18.924485 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb45d34-6a25-46d8-a394-ab4f127465a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:57:18.924756 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb45d34-6a25-46d8-a394-ab4f127465a5 admin admin] PATH_INFO: `/v3/roles` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:57:18.925573 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adb45d34-6a25-46d8-a394-ab4f127465a5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:57:18.927301 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adb45d34-6a25-46d8-a394-ab4f127465a5 admin admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:57:18.943566 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 1167/2324] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Aug 1 16:57:18 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 16:57:18.948493 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a208883-41b7-476e-96eb-70daae3f0cff admin admin] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:57:18.967130 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a208883-41b7-476e-96eb-70daae3f0cff admin admin] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:57:18.972853 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b26ae8d-cf71-4c78-8e3f-0f0f4e0ffc7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:57:18.973746 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b26ae8d-cf71-4c78-8e3f-0f0f4e0ffc7d admin admin] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:57:18.973868 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b26ae8d-cf71-4c78-8e3f-0f0f4e0ffc7d admin admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:57:18.974282 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b26ae8d-cf71-4c78-8e3f-0f0f4e0ffc7d admin admin] PATH_INFO: `/v3/roles` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:57:18.975004 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b26ae8d-cf71-4c78-8e3f-0f0f4e0ffc7d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:57:18.976676 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b26ae8d-cf71-4c78-8e3f-0f0f4e0ffc7d admin admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:57:18.993995 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 1158/2325] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Aug 1 16:57:18 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 16:57:18.997174 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb45d34-6a25-46d8-a394-ab4f127465a5 admin admin] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:57:19.015797 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb45d34-6a25-46d8-a394-ab4f127465a5 admin admin] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:57:19.045699 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be8e437-d6ee-4678-ae48-2491e9a9a0fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:57:19.050528 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be8e437-d6ee-4678-ae48-2491e9a9a0fb admin admin] REQUEST_METHOD: `PUT` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:57:19.050820 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be8e437-d6ee-4678-ae48-2491e9a9a0fb admin admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:57:19.051100 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be8e437-d6ee-4678-ae48-2491e9a9a0fb admin admin] PATH_INFO: `/v3/projects/d9fb92139d3643b2bb7ed4bfed2eae67/users/b2d83a4ae8684f3183756b2dcfc4d514/roles/0999d3da63a545f18ff3252afdc90518` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:57:19.068400 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1be8e437-d6ee-4678-ae48-2491e9a9a0fb admin admin] RBAC: Authorizing `identity:create_grant(project_id=d9fb92139d3643b2bb7ed4bfed2eae67, user_id=b2d83a4ae8684f3183756b2dcfc4d514, role_id=0999d3da63a545f18ff3252afdc90518)` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:57:19.071240 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1be8e437-d6ee-4678-ae48-2491e9a9a0fb admin admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:57:19.175173 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 1168/2326] 10.222.0.20 () {60 vars in 1483 bytes} [Mon Aug 1 16:57:18 2022] PUT /identity/v3/projects/d9fb92139d3643b2bb7ed4bfed2eae67/users/b2d83a4ae8684f3183756b2dcfc4d514/roles/0999d3da63a545f18ff3252afdc90518 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 16:57:19.175920 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed88a09f-88d4-4025-8048-4d8444be2320 None None] REQUEST_METHOD: `POST` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:57:19.175920 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed88a09f-88d4-4025-8048-4d8444be2320 None None] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:57:19.175920 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed88a09f-88d4-4025-8048-4d8444be2320 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:57:19.194401 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.auth.core [None req-ed88a09f-88d4-4025-8048-4d8444be2320 None None] MFA Rules not processed for user `1007f8e379b74e169b98fb7c3c0ba5b4`. Rule list: `[]` (Enabled: `True`). {{(pid=87057) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:57:19.195691 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.fernet_utils [None req-ed88a09f-88d4-4025-8048-4d8444be2320 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87057) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:57:19.267953 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 1159/2327] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 1 16:57:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 16:57:19.278219 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be8e437-d6ee-4678-ae48-2491e9a9a0fb admin admin] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:57:19.289420 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.fernet_utils [None req-1be8e437-d6ee-4678-ae48-2491e9a9a0fb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:57:19.370570 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be8e437-d6ee-4678-ae48-2491e9a9a0fb admin admin] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:57:19.375317 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fe1b78d-c3d8-4f59-a079-f7b554118e48 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d9bd8d649154c979be3744bfd016f95', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b992d6a65df4bf9b27c13192d57c5f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:57:19.376246 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe1b78d-c3d8-4f59-a079-f7b554118e48 service neutron] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:57:19.376246 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe1b78d-c3d8-4f59-a079-f7b554118e48 service neutron] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:57:19.376529 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe1b78d-c3d8-4f59-a079-f7b554118e48 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:57:19.386329 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fe1b78d-c3d8-4f59-a079-f7b554118e48 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:57:19.387822 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fe1b78d-c3d8-4f59-a079-f7b554118e48 service neutron] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:57:19.398287 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 1169/2328] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 1 16:57:19 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 16:57:25.505214 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1d2b80-cb0e-42a7-a0d5-1570849c52c3 None None] REQUEST_METHOD: `POST` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:57:25.505706 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1d2b80-cb0e-42a7-a0d5-1570849c52c3 None None] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:57:25.505788 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1d2b80-cb0e-42a7-a0d5-1570849c52c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:57:25.571665 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.auth.core [None req-ac1d2b80-cb0e-42a7-a0d5-1570849c52c3 None None] MFA Rules not processed for user `b2d83a4ae8684f3183756b2dcfc4d514`. Rule list: `[]` (Enabled: `True`). {{(pid=87057) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:57:25.572913 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.fernet_utils [None req-ac1d2b80-cb0e-42a7-a0d5-1570849c52c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87057) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:57:25.578355 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG dbcounter [-] [87057] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87057) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:57:25.653878 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 1160/2329] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 1 16:57:25 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:57:25.665689 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fe1b78d-c3d8-4f59-a079-f7b554118e48 service neutron] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:57:25.676253 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fe1b78d-c3d8-4f59-a079-f7b554118e48 service neutron] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:57:25.681302 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bed47e9-598c-43f0-99c8-686b85443b1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:57:25.682043 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bed47e9-598c-43f0-99c8-686b85443b1a admin admin] REQUEST_METHOD: `POST` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:57:25.682407 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bed47e9-598c-43f0-99c8-686b85443b1a admin admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:57:25.682753 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bed47e9-598c-43f0-99c8-686b85443b1a admin admin] PATH_INFO: `/v3/projects` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:57:25.683552 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bed47e9-598c-43f0-99c8-686b85443b1a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:57:25.685564 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bed47e9-598c-43f0-99c8-686b85443b1a admin admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:57:25.782811 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 1170/2330] 10.222.0.20 () {60 vars in 1153 bytes} [Mon Aug 1 16:57:25 2022] POST /identity/v3/projects => generated 383 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 16:57:25.786490 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac1d2b80-cb0e-42a7-a0d5-1570849c52c3 None None] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:57:25.797243 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac1d2b80-cb0e-42a7-a0d5-1570849c52c3 None None] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:57:25.863107 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58933b62-46b2-4018-908c-2463886cacca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:57:25.863818 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58933b62-46b2-4018-908c-2463886cacca admin admin] REQUEST_METHOD: `POST` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:57:25.864013 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58933b62-46b2-4018-908c-2463886cacca admin admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:57:25.864294 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58933b62-46b2-4018-908c-2463886cacca admin admin] PATH_INFO: `/v3/users` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:57:25.864978 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58933b62-46b2-4018-908c-2463886cacca admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:57:25.867235 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58933b62-46b2-4018-908c-2463886cacca admin admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:57:25.966103 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 1161/2331] 10.222.0.20 () {60 vars in 1144 bytes} [Mon Aug 1 16:57:25 2022] POST /identity/v3/users => generated 298 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 16:57:25.968496 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bed47e9-598c-43f0-99c8-686b85443b1a admin admin] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:57:25.981047 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bed47e9-598c-43f0-99c8-686b85443b1a admin admin] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:57:25.986265 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22f65a6d-5ee3-4509-8e00-494941668649 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:57:25.986887 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f65a6d-5ee3-4509-8e00-494941668649 admin admin] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:57:25.987168 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f65a6d-5ee3-4509-8e00-494941668649 admin admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:57:25.987446 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f65a6d-5ee3-4509-8e00-494941668649 admin admin] PATH_INFO: `/v3/roles` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:57:25.988156 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22f65a6d-5ee3-4509-8e00-494941668649 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:57:25.991488 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22f65a6d-5ee3-4509-8e00-494941668649 admin admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:57:26.004557 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 1171/2332] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Aug 1 16:57:25 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 16:57:26.008319 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58933b62-46b2-4018-908c-2463886cacca admin admin] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:57:26.019931 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58933b62-46b2-4018-908c-2463886cacca admin admin] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:57:26.024994 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f7cf76-d290-476b-9522-4e30962c4012 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:57:26.025677 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f7cf76-d290-476b-9522-4e30962c4012 admin admin] REQUEST_METHOD: `PUT` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:57:26.026159 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f7cf76-d290-476b-9522-4e30962c4012 admin admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:57:26.026448 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f7cf76-d290-476b-9522-4e30962c4012 admin admin] PATH_INFO: `/v3/projects/58fb1e54fe2545ce9120812fea4b9a46/users/5ffd0f4aeef741b1b1a84d61005cd79f/roles/440bb943dd5b46fea038b903b6eaaf81` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:57:26.049030 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8f7cf76-d290-476b-9522-4e30962c4012 admin admin] RBAC: Authorizing `identity:create_grant(project_id=58fb1e54fe2545ce9120812fea4b9a46, user_id=5ffd0f4aeef741b1b1a84d61005cd79f, role_id=440bb943dd5b46fea038b903b6eaaf81)` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:57:26.049030 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8f7cf76-d290-476b-9522-4e30962c4012 admin admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:57:26.102109 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 1162/2333] 10.222.0.20 () {60 vars in 1483 bytes} [Mon Aug 1 16:57:26 2022] PUT /identity/v3/projects/58fb1e54fe2545ce9120812fea4b9a46/users/5ffd0f4aeef741b1b1a84d61005cd79f/roles/440bb943dd5b46fea038b903b6eaaf81 => generated 0 bytes in 94 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 16:57:26.109400 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22f65a6d-5ee3-4509-8e00-494941668649 admin admin] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:57:26.119120 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22f65a6d-5ee3-4509-8e00-494941668649 admin admin] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:57:26.165754 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG dbcounter [-] [87058] Writing DB stats keystone:DELETE=3,keystone:SELECT=93,keystone:INSERT=4 {{(pid=87058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:57:26.178690 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee73be5-a132-4feb-acf1-0b14d06d9a77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:57:26.179412 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee73be5-a132-4feb-acf1-0b14d06d9a77 admin admin] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:57:26.179752 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee73be5-a132-4feb-acf1-0b14d06d9a77 admin admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:57:26.180033 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee73be5-a132-4feb-acf1-0b14d06d9a77 admin admin] PATH_INFO: `/v3/roles` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:57:26.180737 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fee73be5-a132-4feb-acf1-0b14d06d9a77 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:57:26.182430 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fee73be5-a132-4feb-acf1-0b14d06d9a77 admin admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:57:26.193574 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 1172/2334] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Aug 1 16:57:26 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 16:57:26.197311 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f7cf76-d290-476b-9522-4e30962c4012 admin admin] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:57:26.207984 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f7cf76-d290-476b-9522-4e30962c4012 admin admin] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:57:26.212671 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee576ea-10f3-42df-bc3d-817e0a95b227 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:57:26.213429 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee576ea-10f3-42df-bc3d-817e0a95b227 admin admin] REQUEST_METHOD: `PUT` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:57:26.213995 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee576ea-10f3-42df-bc3d-817e0a95b227 admin admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:57:26.214311 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee576ea-10f3-42df-bc3d-817e0a95b227 admin admin] PATH_INFO: `/v3/projects/58fb1e54fe2545ce9120812fea4b9a46/users/5ffd0f4aeef741b1b1a84d61005cd79f/roles/440bb943dd5b46fea038b903b6eaaf81` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:57:26.216501 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aee576ea-10f3-42df-bc3d-817e0a95b227 admin admin] RBAC: Authorizing `identity:create_grant(project_id=58fb1e54fe2545ce9120812fea4b9a46, user_id=5ffd0f4aeef741b1b1a84d61005cd79f, role_id=440bb943dd5b46fea038b903b6eaaf81)` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:57:26.219260 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aee576ea-10f3-42df-bc3d-817e0a95b227 admin admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:57:26.232858 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 1163/2335] 10.222.0.20 () {60 vars in 1483 bytes} [Mon Aug 1 16:57:26 2022] PUT /identity/v3/projects/58fb1e54fe2545ce9120812fea4b9a46/users/5ffd0f4aeef741b1b1a84d61005cd79f/roles/440bb943dd5b46fea038b903b6eaaf81 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 16:57:32.189965 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3a4111-fb59-4e63-883f-60f10f3f688d None None] REQUEST_METHOD: `POST` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:57:32.190496 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3a4111-fb59-4e63-883f-60f10f3f688d None None] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:57:32.191023 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3a4111-fb59-4e63-883f-60f10f3f688d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:57:32.222612 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.auth.core [None req-8d3a4111-fb59-4e63-883f-60f10f3f688d None None] MFA Rules not processed for user `5ffd0f4aeef741b1b1a84d61005cd79f`. Rule list: `[]` (Enabled: `True`). {{(pid=87058) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:57:32.223886 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.fernet_utils [None req-8d3a4111-fb59-4e63-883f-60f10f3f688d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:57:32.306194 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 1173/2336] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 1 16:57:32 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:57:32.316657 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee576ea-10f3-42df-bc3d-817e0a95b227 admin admin] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:57:32.318402 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.fernet_utils [None req-aee576ea-10f3-42df-bc3d-817e0a95b227 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87057) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:57:32.379950 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee576ea-10f3-42df-bc3d-817e0a95b227 admin admin] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:57:32.383551 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d98f08c-bb71-41b4-9a81-2f18c375d3da service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd80694e439e2401d9491712666409016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b992d6a65df4bf9b27c13192d57c5f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:57:32.384172 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d98f08c-bb71-41b4-9a81-2f18c375d3da service glance] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:57:32.384451 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d98f08c-bb71-41b4-9a81-2f18c375d3da service glance] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:57:32.384726 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d98f08c-bb71-41b4-9a81-2f18c375d3da service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:57:32.395643 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG dbcounter [-] [87057] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=87057) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:57:32.444632 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d98f08c-bb71-41b4-9a81-2f18c375d3da service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:57:32.446097 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d98f08c-bb71-41b4-9a81-2f18c375d3da service glance] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:57:32.456443 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 1164/2337] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 1 16:57:32 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 140 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 16:57:32.771917 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1df850-ace1-4cd4-87a7-1951fa985769 None None] REQUEST_METHOD: `POST` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:57:32.772493 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1df850-ace1-4cd4-87a7-1951fa985769 None None] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:57:32.773051 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1df850-ace1-4cd4-87a7-1951fa985769 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:57:32.795574 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.auth.core [None req-3d1df850-ace1-4cd4-87a7-1951fa985769 None None] MFA Rules not processed for user `b6a93d1c6c82489099adc8b413280fc0`. Rule list: `[]` (Enabled: `True`). {{(pid=87058) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:57:32.796793 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.fernet_utils [None req-3d1df850-ace1-4cd4-87a7-1951fa985769 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:57:32.858211 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 1174/2338] 10.222.0.20 () {58 vars in 1009 bytes} [Mon Aug 1 16:57:32 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:57:32.922968 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d98f08c-bb71-41b4-9a81-2f18c375d3da service glance] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:57:32.924490 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.fernet_utils [None req-3d98f08c-bb71-41b4-9a81-2f18c375d3da service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87057) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:57:33.000877 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d98f08c-bb71-41b4-9a81-2f18c375d3da service glance] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:57:33.006089 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b47c07f5-07cd-42f5-92e0-0dceaf9aa6db service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ab4add1faa04faa811ef7a9341cc35c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b992d6a65df4bf9b27c13192d57c5f0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:57:33.006712 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47c07f5-07cd-42f5-92e0-0dceaf9aa6db service swift] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:57:33.006994 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47c07f5-07cd-42f5-92e0-0dceaf9aa6db service swift] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:57:33.007272 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47c07f5-07cd-42f5-92e0-0dceaf9aa6db service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:57:33.016314 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b47c07f5-07cd-42f5-92e0-0dceaf9aa6db service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:57:33.017908 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b47c07f5-07cd-42f5-92e0-0dceaf9aa6db service swift] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:57:33.025920 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 1165/2339] 10.222.0.20 () {60 vars in 1384 bytes} [Mon Aug 1 16:57:32 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 01 16:57:42.855145 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG dbcounter [-] [87058] Writing DB stats keystone:SELECT=66 {{(pid=87058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:57:43.022791 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG dbcounter [-] [87057] Writing DB stats keystone:SELECT=54 {{(pid=87057) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:57:44.477650 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e189fa3-3cd4-4902-92ba-f761e31e69ae None None] REQUEST_METHOD: `POST` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:57:44.483438 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e189fa3-3cd4-4902-92ba-f761e31e69ae None None] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:57:44.483438 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e189fa3-3cd4-4902-92ba-f761e31e69ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:57:44.502611 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.auth.core [None req-7e189fa3-3cd4-4902-92ba-f761e31e69ae None None] MFA Rules not processed for user `b2d83a4ae8684f3183756b2dcfc4d514`. Rule list: `[]` (Enabled: `True`). {{(pid=87058) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:57:44.503973 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.fernet_utils [None req-7e189fa3-3cd4-4902-92ba-f761e31e69ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:57:44.511207 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 1175/2340] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 1 16:57:44 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:57:50.550634 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b47c07f5-07cd-42f5-92e0-0dceaf9aa6db service swift] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:57:50.550634 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.fernet_utils [None req-b47c07f5-07cd-42f5-92e0-0dceaf9aa6db service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87057) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:57:50.618742 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b47c07f5-07cd-42f5-92e0-0dceaf9aa6db service swift] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:57:50.625067 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-360e0d33-2e9f-48ec-88ec-b75babb0fb1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:57:50.626053 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360e0d33-2e9f-48ec-88ec-b75babb0fb1b admin admin] REQUEST_METHOD: `DELETE` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:57:50.626334 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360e0d33-2e9f-48ec-88ec-b75babb0fb1b admin admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:57:50.626614 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360e0d33-2e9f-48ec-88ec-b75babb0fb1b admin admin] PATH_INFO: `/v3/users/69ee6584adbd480e8523a1043160f907` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:57:50.628016 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-360e0d33-2e9f-48ec-88ec-b75babb0fb1b admin admin] RBAC: Authorizing `identity:delete_user(user_id=69ee6584adbd480e8523a1043160f907)` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:57:50.630070 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-360e0d33-2e9f-48ec-88ec-b75babb0fb1b admin admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:57:50.859887 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.notifications [None req-360e0d33-2e9f-48ec-88ec-b75babb0fb1b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '69ee6584adbd480e8523a1043160f907'} {{(pid=87057) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:57:50.860645 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.notifications [None req-360e0d33-2e9f-48ec-88ec-b75babb0fb1b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '69ee6584adbd480e8523a1043160f907'} {{(pid=87057) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:57:50.929928 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.notifications [None req-360e0d33-2e9f-48ec-88ec-b75babb0fb1b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '69ee6584adbd480e8523a1043160f907'} {{(pid=87057) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:57:50.930673 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.notifications [None req-360e0d33-2e9f-48ec-88ec-b75babb0fb1b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '69ee6584adbd480e8523a1043160f907'} {{(pid=87057) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:57:50.941027 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.notifications [None req-360e0d33-2e9f-48ec-88ec-b75babb0fb1b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '69ee6584adbd480e8523a1043160f907'} {{(pid=87057) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:57:50.975418 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 1166/2341] 10.222.0.20 () {58 vars in 1224 bytes} [Mon Aug 1 16:57:50 2022] DELETE /identity/v3/users/69ee6584adbd480e8523a1043160f907 => generated 0 bytes in 429 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 16:57:50.980129 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0dcf70-9b26-4bd1-ad03-e6728ff6d1b0 None None] REQUEST_METHOD: `POST` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:57:50.981201 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0dcf70-9b26-4bd1-ad03-e6728ff6d1b0 None None] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:57:50.981345 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0dcf70-9b26-4bd1-ad03-e6728ff6d1b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:57:51.003092 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.auth.core [None req-3a0dcf70-9b26-4bd1-ad03-e6728ff6d1b0 None None] MFA Rules not processed for user `1007f8e379b74e169b98fb7c3c0ba5b4`. Rule list: `[]` (Enabled: `True`). {{(pid=87058) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:57:51.004548 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.fernet_utils [None req-3a0dcf70-9b26-4bd1-ad03-e6728ff6d1b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:57:51.076098 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 1176/2342] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 1 16:57:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:57:51.087373 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-360e0d33-2e9f-48ec-88ec-b75babb0fb1b admin admin] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:57:51.089169 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.fernet_utils [None req-360e0d33-2e9f-48ec-88ec-b75babb0fb1b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87057) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:57:51.155281 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-360e0d33-2e9f-48ec-88ec-b75babb0fb1b admin admin] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:57:51.159875 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c629ca0-2ec0-47a8-bdcc-7bf339f16fad service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d9bd8d649154c979be3744bfd016f95', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b992d6a65df4bf9b27c13192d57c5f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:57:51.160761 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c629ca0-2ec0-47a8-bdcc-7bf339f16fad service neutron] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:57:51.161118 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c629ca0-2ec0-47a8-bdcc-7bf339f16fad service neutron] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:57:51.161459 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c629ca0-2ec0-47a8-bdcc-7bf339f16fad service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:57:51.166086 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a642777f-fbf6-4ac2-bcd5-a0771fe3c01c None None] REQUEST_METHOD: `POST` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:57:51.166480 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a642777f-fbf6-4ac2-bcd5-a0771fe3c01c None None] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:57:51.166947 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a642777f-fbf6-4ac2-bcd5-a0771fe3c01c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:57:51.192244 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c629ca0-2ec0-47a8-bdcc-7bf339f16fad service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:57:51.194147 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c629ca0-2ec0-47a8-bdcc-7bf339f16fad service neutron] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:57:51.208243 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 1167/2343] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 1 16:57:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 16:57:51.224497 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.auth.core [None req-a642777f-fbf6-4ac2-bcd5-a0771fe3c01c None None] MFA Rules not processed for user `b6a93d1c6c82489099adc8b413280fc0`. Rule list: `[]` (Enabled: `True`). {{(pid=87058) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:57:51.225725 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.fernet_utils [None req-a642777f-fbf6-4ac2-bcd5-a0771fe3c01c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:57:51.297952 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 1177/2344] 10.222.0.20 () {58 vars in 1009 bytes} [Mon Aug 1 16:57:51 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 16:57:51.310992 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c629ca0-2ec0-47a8-bdcc-7bf339f16fad service neutron] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:57:51.312560 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.fernet_utils [None req-3c629ca0-2ec0-47a8-bdcc-7bf339f16fad service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87057) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:57:51.342911 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG dbcounter [-] [87057] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87057) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:57:51.390377 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c629ca0-2ec0-47a8-bdcc-7bf339f16fad service neutron] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:57:51.395606 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a22151e-3bc6-4fb9-9879-bf691da0ef94 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ab4add1faa04faa811ef7a9341cc35c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b992d6a65df4bf9b27c13192d57c5f0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:57:51.396227 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a22151e-3bc6-4fb9-9879-bf691da0ef94 service swift] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:57:51.396848 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a22151e-3bc6-4fb9-9879-bf691da0ef94 service swift] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:57:51.396848 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a22151e-3bc6-4fb9-9879-bf691da0ef94 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:57:51.405668 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a22151e-3bc6-4fb9-9879-bf691da0ef94 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:57:51.407291 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a22151e-3bc6-4fb9-9879-bf691da0ef94 service swift] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:57:51.416287 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 1168/2345] 10.222.0.20 () {60 vars in 1384 bytes} [Mon Aug 1 16:57:51 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 01 16:57:51.831125 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a642777f-fbf6-4ac2-bcd5-a0771fe3c01c None None] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:57:51.833488 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.fernet_utils [None req-a642777f-fbf6-4ac2-bcd5-a0771fe3c01c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:57:51.873675 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a642777f-fbf6-4ac2-bcd5-a0771fe3c01c None None] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:57:51.882194 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca78988e-b5cc-4212-a453-6a6a2038b5c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:57:51.882910 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca78988e-b5cc-4212-a453-6a6a2038b5c8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:57:51.883194 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca78988e-b5cc-4212-a453-6a6a2038b5c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:57:51.883472 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca78988e-b5cc-4212-a453-6a6a2038b5c8 admin admin] PATH_INFO: `/v3/projects/1f618f3c053d4a4d8d75a64e3ead4a61` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:57:51.887247 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca78988e-b5cc-4212-a453-6a6a2038b5c8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1f618f3c053d4a4d8d75a64e3ead4a61)` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:57:51.892103 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca78988e-b5cc-4212-a453-6a6a2038b5c8 admin admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:57:52.057723 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.notifications [None req-ca78988e-b5cc-4212-a453-6a6a2038b5c8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1f618f3c053d4a4d8d75a64e3ead4a61', 'request_id': 'req-ca78988e-b5cc-4212-a453-6a6a2038b5c8'} {{(pid=87058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:57:52.072047 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.notifications [None req-ca78988e-b5cc-4212-a453-6a6a2038b5c8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1f618f3c053d4a4d8d75a64e3ead4a61', 'request_id': 'req-ca78988e-b5cc-4212-a453-6a6a2038b5c8'} {{(pid=87058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:57:52.085370 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.notifications [None req-ca78988e-b5cc-4212-a453-6a6a2038b5c8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1f618f3c053d4a4d8d75a64e3ead4a61', 'request_id': 'req-ca78988e-b5cc-4212-a453-6a6a2038b5c8'} {{(pid=87058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:57:52.211906 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.notifications [None req-ca78988e-b5cc-4212-a453-6a6a2038b5c8 admin admin] The token cache is being invalidate because project 1f618f3c053d4a4d8d75a64e3ead4a61 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87058) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 16:57:52.212204 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.notifications [None req-ca78988e-b5cc-4212-a453-6a6a2038b5c8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:57:52.213170 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.notifications [None req-ca78988e-b5cc-4212-a453-6a6a2038b5c8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:57:52.216312 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 1178/2346] 10.222.0.20 () {58 vars in 1233 bytes} [Mon Aug 1 16:57:51 2022] DELETE /identity/v3/projects/1f618f3c053d4a4d8d75a64e3ead4a61 => generated 0 bytes in 385 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 16:57:52.221798 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d888535-78d0-4696-a83f-460118287005 None None] REQUEST_METHOD: `POST` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:57:52.221798 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d888535-78d0-4696-a83f-460118287005 None None] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:57:52.222420 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d888535-78d0-4696-a83f-460118287005 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:57:52.263086 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.auth.core [None req-5d888535-78d0-4696-a83f-460118287005 None None] MFA Rules not processed for user `1007f8e379b74e169b98fb7c3c0ba5b4`. Rule list: `[]` (Enabled: `True`). {{(pid=87057) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:57:52.264368 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.fernet_utils [None req-5d888535-78d0-4696-a83f-460118287005 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87057) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:57:52.385020 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 1169/2347] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 1 16:57:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:57:52.385467 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46964939-34f7-4391-a60e-2a13edf4975a None None] REQUEST_METHOD: `POST` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:57:52.385831 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46964939-34f7-4391-a60e-2a13edf4975a None None] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:57:52.386313 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46964939-34f7-4391-a60e-2a13edf4975a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:57:52.437897 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.auth.core [None req-46964939-34f7-4391-a60e-2a13edf4975a None None] MFA Rules not processed for user `1007f8e379b74e169b98fb7c3c0ba5b4`. Rule list: `[]` (Enabled: `True`). {{(pid=87058) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:57:52.439267 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.fernet_utils [None req-46964939-34f7-4391-a60e-2a13edf4975a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:57:52.453039 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 1179/2348] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 1 16:57:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 68 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:57:52.457212 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d888535-78d0-4696-a83f-460118287005 None None] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:57:52.473108 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d888535-78d0-4696-a83f-460118287005 None None] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:57:52.478644 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd30e04-db99-462b-a584-33f51e5f2134 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:57:52.479322 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd30e04-db99-462b-a584-33f51e5f2134 admin admin] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:57:52.479605 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd30e04-db99-462b-a584-33f51e5f2134 admin admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:57:52.479882 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd30e04-db99-462b-a584-33f51e5f2134 admin admin] PATH_INFO: `/v3/domains` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:57:52.480655 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcd30e04-db99-462b-a584-33f51e5f2134 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87057) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:57:52.481639 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcd30e04-db99-462b-a584-33f51e5f2134 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:57:52.483368 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcd30e04-db99-462b-a584-33f51e5f2134 admin admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:57:52.502075 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 1170/2349] 10.222.0.20 () {58 vars in 1153 bytes} [Mon Aug 1 16:57:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:57:52.505775 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46964939-34f7-4391-a60e-2a13edf4975a None None] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:57:52.516304 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46964939-34f7-4391-a60e-2a13edf4975a None None] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:57:52.521762 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb49a6ad-5351-4636-9008-234766ff4772 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:57:52.522412 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb49a6ad-5351-4636-9008-234766ff4772 admin admin] REQUEST_METHOD: `POST` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:57:52.522693 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb49a6ad-5351-4636-9008-234766ff4772 admin admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:57:52.522963 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb49a6ad-5351-4636-9008-234766ff4772 admin admin] PATH_INFO: `/v3/projects` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:57:52.523638 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb49a6ad-5351-4636-9008-234766ff4772 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:57:52.525662 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb49a6ad-5351-4636-9008-234766ff4772 admin admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:57:52.547345 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG dbcounter [-] [87058] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=87058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:57:52.617305 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 1180/2350] 10.222.0.20 () {60 vars in 1153 bytes} [Mon Aug 1 16:57:52 2022] POST /identity/v3/projects => generated 375 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 16:57:52.622645 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd30e04-db99-462b-a584-33f51e5f2134 admin admin] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:57:52.633960 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd30e04-db99-462b-a584-33f51e5f2134 admin admin] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:57:52.725374 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4507d672-f239-4573-b7d3-b966d988ea01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:57:52.726299 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4507d672-f239-4573-b7d3-b966d988ea01 admin admin] REQUEST_METHOD: `POST` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:57:52.726585 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4507d672-f239-4573-b7d3-b966d988ea01 admin admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:57:52.726860 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4507d672-f239-4573-b7d3-b966d988ea01 admin admin] PATH_INFO: `/v3/users` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:57:52.727849 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4507d672-f239-4573-b7d3-b966d988ea01 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:57:52.730024 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4507d672-f239-4573-b7d3-b966d988ea01 admin admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:57:52.839462 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 1171/2351] 10.222.0.20 () {60 vars in 1144 bytes} [Mon Aug 1 16:57:52 2022] POST /identity/v3/users => generated 288 bytes in 218 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 16:57:52.844547 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb49a6ad-5351-4636-9008-234766ff4772 admin admin] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:57:52.858250 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb49a6ad-5351-4636-9008-234766ff4772 admin admin] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:57:52.864184 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b917259-69b6-48bc-aad7-9ad764ae9da9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:57:52.864884 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b917259-69b6-48bc-aad7-9ad764ae9da9 admin admin] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:57:52.872588 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b917259-69b6-48bc-aad7-9ad764ae9da9 admin admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:57:52.872588 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b917259-69b6-48bc-aad7-9ad764ae9da9 admin admin] PATH_INFO: `/v3/roles` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:57:52.872588 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b917259-69b6-48bc-aad7-9ad764ae9da9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:57:52.872588 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b917259-69b6-48bc-aad7-9ad764ae9da9 admin admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:57:52.880632 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 1181/2352] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Aug 1 16:57:52 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 16:57:52.884911 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4507d672-f239-4573-b7d3-b966d988ea01 admin admin] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:57:52.896179 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4507d672-f239-4573-b7d3-b966d988ea01 admin admin] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:57:52.900986 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c019054f-0dbc-48c1-95c2-a63e1be40ec1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:57:52.901532 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c019054f-0dbc-48c1-95c2-a63e1be40ec1 admin admin] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:57:52.901941 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c019054f-0dbc-48c1-95c2-a63e1be40ec1 admin admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:57:52.903201 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c019054f-0dbc-48c1-95c2-a63e1be40ec1 admin admin] PATH_INFO: `/v3/roles` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:57:52.903201 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c019054f-0dbc-48c1-95c2-a63e1be40ec1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:57:52.905193 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c019054f-0dbc-48c1-95c2-a63e1be40ec1 admin admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:57:52.922873 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 1172/2353] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Aug 1 16:57:52 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 16:57:52.923934 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b917259-69b6-48bc-aad7-9ad764ae9da9 admin admin] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:57:52.935432 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b917259-69b6-48bc-aad7-9ad764ae9da9 admin admin] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:57:52.942459 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5576bd30-5020-4df3-a09d-742305841713 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:57:52.944106 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5576bd30-5020-4df3-a09d-742305841713 admin admin] REQUEST_METHOD: `PUT` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:57:52.944477 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5576bd30-5020-4df3-a09d-742305841713 admin admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:57:52.944896 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5576bd30-5020-4df3-a09d-742305841713 admin admin] PATH_INFO: `/v3/projects/3c22a11b9ed64d57a8e8f750947e11cb/users/909efafdae624e82b72c304273e066a1/roles/0999d3da63a545f18ff3252afdc90518` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:57:52.978955 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5576bd30-5020-4df3-a09d-742305841713 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3c22a11b9ed64d57a8e8f750947e11cb, user_id=909efafdae624e82b72c304273e066a1, role_id=0999d3da63a545f18ff3252afdc90518)` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:57:52.981963 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5576bd30-5020-4df3-a09d-742305841713 admin admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:57:53.050933 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 1182/2354] 10.222.0.20 () {60 vars in 1483 bytes} [Mon Aug 1 16:57:52 2022] PUT /identity/v3/projects/3c22a11b9ed64d57a8e8f750947e11cb/users/909efafdae624e82b72c304273e066a1/roles/0999d3da63a545f18ff3252afdc90518 => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 16:57:53.062309 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2a8169-113d-4694-9ff4-e293ee688bc1 None None] REQUEST_METHOD: `POST` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:57:53.062865 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2a8169-113d-4694-9ff4-e293ee688bc1 None None] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:57:53.063276 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2a8169-113d-4694-9ff4-e293ee688bc1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:57:53.101759 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.auth.core [None req-5b2a8169-113d-4694-9ff4-e293ee688bc1 None None] MFA Rules not processed for user `909efafdae624e82b72c304273e066a1`. Rule list: `[]` (Enabled: `True`). {{(pid=87057) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:57:53.103056 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.fernet_utils [None req-5b2a8169-113d-4694-9ff4-e293ee688bc1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87057) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:57:53.109798 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG dbcounter [-] [87057] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87057) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:57:53.173213 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 1173/2355] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 1 16:57:53 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:57:53.184513 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5576bd30-5020-4df3-a09d-742305841713 admin admin] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:57:53.187345 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.fernet_utils [None req-5576bd30-5020-4df3-a09d-742305841713 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:57:53.266839 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5576bd30-5020-4df3-a09d-742305841713 admin admin] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:57:53.271035 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f010a288-f7f9-4642-8e44-7fe9df4e493e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25970b52ca15448fa1abb1f974d64038', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b992d6a65df4bf9b27c13192d57c5f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:57:53.271757 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f010a288-f7f9-4642-8e44-7fe9df4e493e service cinder] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:57:53.272304 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f010a288-f7f9-4642-8e44-7fe9df4e493e service cinder] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:57:53.272304 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f010a288-f7f9-4642-8e44-7fe9df4e493e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:57:53.283329 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f010a288-f7f9-4642-8e44-7fe9df4e493e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:57:53.284836 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f010a288-f7f9-4642-8e44-7fe9df4e493e service cinder] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:57:53.296446 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 1183/2356] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 1 16:57:53 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 16:57:56.130687 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953501e8-f870-4759-9306-bedfdbea01e8 None None] REQUEST_METHOD: `POST` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:57:56.131243 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953501e8-f870-4759-9306-bedfdbea01e8 None None] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:57:56.131243 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953501e8-f870-4759-9306-bedfdbea01e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:57:56.154014 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.auth.core [None req-953501e8-f870-4759-9306-bedfdbea01e8 None None] MFA Rules not processed for user `b353fdd302394227987a6ae8c2943173`. Rule list: `[]` (Enabled: `True`). {{(pid=87057) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:57:56.155277 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.fernet_utils [None req-953501e8-f870-4759-9306-bedfdbea01e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87057) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:57:56.228150 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 1174/2357] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 1 16:57:56 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:58:03.291248 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG dbcounter [-] [87058] Writing DB stats keystone:INSERT=2,keystone:SELECT=43 {{(pid=87058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:58:06.218557 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG dbcounter [-] [87057] Writing DB stats keystone:SELECT=49 {{(pid=87057) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:58:08.292721 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f010a288-f7f9-4642-8e44-7fe9df4e493e service cinder] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:58:08.294415 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.fernet_utils [None req-f010a288-f7f9-4642-8e44-7fe9df4e493e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:58:08.428457 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f010a288-f7f9-4642-8e44-7fe9df4e493e service cinder] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:58:08.437006 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db14e03-0d67-462d-bcf8-452246b983b9 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8362293261614623bcdabc4120a97a9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b992d6a65df4bf9b27c13192d57c5f0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:58:08.438334 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db14e03-0d67-462d-bcf8-452246b983b9 service placement] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:58:08.438625 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db14e03-0d67-462d-bcf8-452246b983b9 service placement] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:58:08.438954 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db14e03-0d67-462d-bcf8-452246b983b9 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:58:08.441432 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.fernet_utils [None req-7db14e03-0d67-462d-bcf8-452246b983b9 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:58:08.461527 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7db14e03-0d67-462d-bcf8-452246b983b9 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:58:08.463240 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7db14e03-0d67-462d-bcf8-452246b983b9 service placement] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:58:08.477649 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 1184/2358] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 1 16:58:08 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 185 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 16:58:09.648237 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-953501e8-f870-4759-9306-bedfdbea01e8 None None] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:58:09.649842 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.fernet_utils [None req-953501e8-f870-4759-9306-bedfdbea01e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87057) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:58:09.722885 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-953501e8-f870-4759-9306-bedfdbea01e8 None None] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:58:09.727456 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3f92ec2-27c1-46ad-a8f6-774604a1cf7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:58:09.728103 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f92ec2-27c1-46ad-a8f6-774604a1cf7c admin admin] REQUEST_METHOD: `DELETE` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:58:09.728394 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f92ec2-27c1-46ad-a8f6-774604a1cf7c admin admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:58:09.728662 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f92ec2-27c1-46ad-a8f6-774604a1cf7c admin admin] PATH_INFO: `/v3/users/cdb443e00f0b420a8365c0975194f5e2` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:58:09.732586 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3f92ec2-27c1-46ad-a8f6-774604a1cf7c admin admin] RBAC: Authorizing `identity:delete_user(user_id=cdb443e00f0b420a8365c0975194f5e2)` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:58:09.732586 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3f92ec2-27c1-46ad-a8f6-774604a1cf7c admin admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:58:09.983881 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.notifications [None req-e3f92ec2-27c1-46ad-a8f6-774604a1cf7c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cdb443e00f0b420a8365c0975194f5e2'} {{(pid=87057) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:58:09.985296 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.notifications [None req-e3f92ec2-27c1-46ad-a8f6-774604a1cf7c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cdb443e00f0b420a8365c0975194f5e2'} {{(pid=87057) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:58:10.083340 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.notifications [None req-e3f92ec2-27c1-46ad-a8f6-774604a1cf7c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cdb443e00f0b420a8365c0975194f5e2'} {{(pid=87057) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:58:10.084265 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.notifications [None req-e3f92ec2-27c1-46ad-a8f6-774604a1cf7c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cdb443e00f0b420a8365c0975194f5e2'} {{(pid=87057) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:58:10.106009 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.notifications [None req-e3f92ec2-27c1-46ad-a8f6-774604a1cf7c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cdb443e00f0b420a8365c0975194f5e2'} {{(pid=87057) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:58:10.136816 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 1175/2359] 10.222.0.20 () {58 vars in 1224 bytes} [Mon Aug 1 16:58:09 2022] DELETE /identity/v3/users/cdb443e00f0b420a8365c0975194f5e2 => generated 0 bytes in 489 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 16:58:10.151234 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67129acd-4ea9-4a4c-8d21-93d1131efe2d None None] REQUEST_METHOD: `POST` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:58:10.152157 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67129acd-4ea9-4a4c-8d21-93d1131efe2d None None] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:58:10.152478 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67129acd-4ea9-4a4c-8d21-93d1131efe2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:58:10.177650 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.auth.core [None req-67129acd-4ea9-4a4c-8d21-93d1131efe2d None None] MFA Rules not processed for user `1007f8e379b74e169b98fb7c3c0ba5b4`. Rule list: `[]` (Enabled: `True`). {{(pid=87058) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:58:10.179163 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.fernet_utils [None req-67129acd-4ea9-4a4c-8d21-93d1131efe2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:58:10.277579 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 1185/2360] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 1 16:58:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:58:10.318104 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3f92ec2-27c1-46ad-a8f6-774604a1cf7c admin admin] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:58:10.321671 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.fernet_utils [None req-e3f92ec2-27c1-46ad-a8f6-774604a1cf7c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87057) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:58:10.433918 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3f92ec2-27c1-46ad-a8f6-774604a1cf7c admin admin] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:58:10.440872 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11bcd962-8602-44c0-86e2-361b049b0492 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d9bd8d649154c979be3744bfd016f95', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b992d6a65df4bf9b27c13192d57c5f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:58:10.441659 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11bcd962-8602-44c0-86e2-361b049b0492 service neutron] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:58:10.442201 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11bcd962-8602-44c0-86e2-361b049b0492 service neutron] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:58:10.448589 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11bcd962-8602-44c0-86e2-361b049b0492 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:58:10.475415 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11bcd962-8602-44c0-86e2-361b049b0492 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:58:10.484371 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11bcd962-8602-44c0-86e2-361b049b0492 service neutron] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:58:10.500736 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 1176/2361] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 1 16:58:10 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 183 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 16:58:11.087753 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67129acd-4ea9-4a4c-8d21-93d1131efe2d None None] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:58:11.089182 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.fernet_utils [None req-67129acd-4ea9-4a4c-8d21-93d1131efe2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:58:11.106321 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67129acd-4ea9-4a4c-8d21-93d1131efe2d None None] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:58:11.150170 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f8b8082-5d8f-460a-b33c-8b9d5104a3fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:58:11.167403 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8b8082-5d8f-460a-b33c-8b9d5104a3fe admin admin] REQUEST_METHOD: `DELETE` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:58:11.168024 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8b8082-5d8f-460a-b33c-8b9d5104a3fe admin admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:58:11.169563 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8b8082-5d8f-460a-b33c-8b9d5104a3fe admin admin] PATH_INFO: `/v3/projects/dca2fb3173af463392caaf34369ae99e` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:58:11.173050 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f8b8082-5d8f-460a-b33c-8b9d5104a3fe admin admin] RBAC: Authorizing `identity:delete_project(project_id=dca2fb3173af463392caaf34369ae99e)` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:58:11.177779 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f8b8082-5d8f-460a-b33c-8b9d5104a3fe admin admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:58:11.329891 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.notifications [None req-7f8b8082-5d8f-460a-b33c-8b9d5104a3fe admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dca2fb3173af463392caaf34369ae99e', 'request_id': 'req-7f8b8082-5d8f-460a-b33c-8b9d5104a3fe'} {{(pid=87058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:58:11.338999 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.notifications [None req-7f8b8082-5d8f-460a-b33c-8b9d5104a3fe admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dca2fb3173af463392caaf34369ae99e', 'request_id': 'req-7f8b8082-5d8f-460a-b33c-8b9d5104a3fe'} {{(pid=87058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:58:11.348209 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.notifications [None req-7f8b8082-5d8f-460a-b33c-8b9d5104a3fe admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dca2fb3173af463392caaf34369ae99e', 'request_id': 'req-7f8b8082-5d8f-460a-b33c-8b9d5104a3fe'} {{(pid=87058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:58:11.435012 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.notifications [None req-7f8b8082-5d8f-460a-b33c-8b9d5104a3fe admin admin] The token cache is being invalidate because project dca2fb3173af463392caaf34369ae99e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87058) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 16:58:11.435533 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.notifications [None req-7f8b8082-5d8f-460a-b33c-8b9d5104a3fe admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:58:11.437521 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.notifications [None req-7f8b8082-5d8f-460a-b33c-8b9d5104a3fe admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:58:11.439153 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 1186/2362] 10.222.0.20 () {58 vars in 1233 bytes} [Mon Aug 1 16:58:11 2022] DELETE /identity/v3/projects/dca2fb3173af463392caaf34369ae99e => generated 0 bytes in 352 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 16:58:11.443945 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d837e06-533f-4562-9ceb-80c9ecc59295 None None] REQUEST_METHOD: `POST` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:58:11.444445 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d837e06-533f-4562-9ceb-80c9ecc59295 None None] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:58:11.444534 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d837e06-533f-4562-9ceb-80c9ecc59295 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:58:11.468918 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.auth.core [None req-2d837e06-533f-4562-9ceb-80c9ecc59295 None None] MFA Rules not processed for user `1007f8e379b74e169b98fb7c3c0ba5b4`. Rule list: `[]` (Enabled: `True`). {{(pid=87057) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:58:11.470242 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.fernet_utils [None req-2d837e06-533f-4562-9ceb-80c9ecc59295 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87057) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:58:11.490577 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG dbcounter [-] [87057] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87057) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:58:11.538756 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 1177/2363] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 1 16:58:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:58:11.547587 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b321e0c-6a55-4b06-abe1-349bd413d019 None None] REQUEST_METHOD: `POST` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:58:11.547898 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b321e0c-6a55-4b06-abe1-349bd413d019 None None] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:58:11.548173 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b321e0c-6a55-4b06-abe1-349bd413d019 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:58:11.570189 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.auth.core [None req-4b321e0c-6a55-4b06-abe1-349bd413d019 None None] MFA Rules not processed for user `1007f8e379b74e169b98fb7c3c0ba5b4`. Rule list: `[]` (Enabled: `True`). {{(pid=87058) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:58:11.571427 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.fernet_utils [None req-4b321e0c-6a55-4b06-abe1-349bd413d019 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:58:11.580376 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 1187/2364] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 1 16:58:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:58:11.584744 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d837e06-533f-4562-9ceb-80c9ecc59295 None None] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:58:11.594723 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d837e06-533f-4562-9ceb-80c9ecc59295 None None] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:58:11.599646 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e8ab2b3-cb98-4c7d-8caf-9a8df28fda40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:58:11.600258 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8ab2b3-cb98-4c7d-8caf-9a8df28fda40 admin admin] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:58:11.600539 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8ab2b3-cb98-4c7d-8caf-9a8df28fda40 admin admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:58:11.600944 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8ab2b3-cb98-4c7d-8caf-9a8df28fda40 admin admin] PATH_INFO: `/v3/domains` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:58:11.601643 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e8ab2b3-cb98-4c7d-8caf-9a8df28fda40 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87057) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:58:11.601961 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e8ab2b3-cb98-4c7d-8caf-9a8df28fda40 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:58:11.603599 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e8ab2b3-cb98-4c7d-8caf-9a8df28fda40 admin admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:58:11.617432 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 1178/2365] 10.222.0.20 () {58 vars in 1153 bytes} [Mon Aug 1 16:58:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:58:11.621919 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b321e0c-6a55-4b06-abe1-349bd413d019 None None] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:58:11.632121 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b321e0c-6a55-4b06-abe1-349bd413d019 None None] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:58:11.637000 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-085ab68b-fe12-41a6-8041-14a9210e5f06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:58:11.637603 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085ab68b-fe12-41a6-8041-14a9210e5f06 admin admin] REQUEST_METHOD: `POST` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:58:11.637883 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085ab68b-fe12-41a6-8041-14a9210e5f06 admin admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:58:11.638200 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085ab68b-fe12-41a6-8041-14a9210e5f06 admin admin] PATH_INFO: `/v3/projects` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:58:11.638895 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-085ab68b-fe12-41a6-8041-14a9210e5f06 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:58:11.641460 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-085ab68b-fe12-41a6-8041-14a9210e5f06 admin admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:58:11.645986 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG dbcounter [-] [87058] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=87058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:58:11.748102 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 1188/2366] 10.222.0.20 () {60 vars in 1153 bytes} [Mon Aug 1 16:58:11 2022] POST /identity/v3/projects => generated 381 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 16:58:11.752933 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e8ab2b3-cb98-4c7d-8caf-9a8df28fda40 admin admin] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:58:11.763331 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e8ab2b3-cb98-4c7d-8caf-9a8df28fda40 admin admin] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:58:11.822696 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc87053-f578-4b7a-be21-51b8a9371e36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:58:11.823327 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc87053-f578-4b7a-be21-51b8a9371e36 admin admin] REQUEST_METHOD: `POST` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:58:11.823608 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc87053-f578-4b7a-be21-51b8a9371e36 admin admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:58:11.823883 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc87053-f578-4b7a-be21-51b8a9371e36 admin admin] PATH_INFO: `/v3/users` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:58:11.824595 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dc87053-f578-4b7a-be21-51b8a9371e36 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:58:11.826869 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dc87053-f578-4b7a-be21-51b8a9371e36 admin admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:58:11.913670 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 1179/2367] 10.222.0.20 () {60 vars in 1144 bytes} [Mon Aug 1 16:58:11 2022] POST /identity/v3/users => generated 291 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 16:58:11.914970 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-085ab68b-fe12-41a6-8041-14a9210e5f06 admin admin] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:58:11.926009 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-085ab68b-fe12-41a6-8041-14a9210e5f06 admin admin] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:58:11.930992 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea34de21-c965-4bfd-92f3-d3cec10f1a1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:58:11.931604 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea34de21-c965-4bfd-92f3-d3cec10f1a1f admin admin] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:58:11.931882 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea34de21-c965-4bfd-92f3-d3cec10f1a1f admin admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:58:11.932379 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea34de21-c965-4bfd-92f3-d3cec10f1a1f admin admin] PATH_INFO: `/v3/roles` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:58:11.932895 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea34de21-c965-4bfd-92f3-d3cec10f1a1f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:58:11.934561 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea34de21-c965-4bfd-92f3-d3cec10f1a1f admin admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:58:11.946785 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 1189/2368] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Aug 1 16:58:11 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 01 16:58:11.950562 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc87053-f578-4b7a-be21-51b8a9371e36 admin admin] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:58:11.961924 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc87053-f578-4b7a-be21-51b8a9371e36 admin admin] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:58:11.966797 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c754d4dd-8b89-4015-a837-e53b099b1578 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:58:11.967412 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c754d4dd-8b89-4015-a837-e53b099b1578 admin admin] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:58:11.967812 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c754d4dd-8b89-4015-a837-e53b099b1578 admin admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:58:11.968093 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c754d4dd-8b89-4015-a837-e53b099b1578 admin admin] PATH_INFO: `/v3/roles` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:58:11.968795 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c754d4dd-8b89-4015-a837-e53b099b1578 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:58:11.970474 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c754d4dd-8b89-4015-a837-e53b099b1578 admin admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:58:11.984227 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 1180/2369] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Aug 1 16:58:11 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 16:58:11.988427 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea34de21-c965-4bfd-92f3-d3cec10f1a1f admin admin] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:58:11.999880 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea34de21-c965-4bfd-92f3-d3cec10f1a1f admin admin] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:58:12.004651 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a32a00-c96d-4945-b1d7-624b77dc80b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:58:12.005368 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a32a00-c96d-4945-b1d7-624b77dc80b6 admin admin] REQUEST_METHOD: `PUT` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:58:12.005666 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a32a00-c96d-4945-b1d7-624b77dc80b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:58:12.008282 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a32a00-c96d-4945-b1d7-624b77dc80b6 admin admin] PATH_INFO: `/v3/projects/025776af0f594ddc9bfd2eedb6eb7602/users/d720864cb0854d549c05e425bffafa86/roles/0999d3da63a545f18ff3252afdc90518` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:58:12.021141 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18a32a00-c96d-4945-b1d7-624b77dc80b6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=025776af0f594ddc9bfd2eedb6eb7602, user_id=d720864cb0854d549c05e425bffafa86, role_id=0999d3da63a545f18ff3252afdc90518)` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:58:12.023878 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18a32a00-c96d-4945-b1d7-624b77dc80b6 admin admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:58:12.101399 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 1190/2370] 10.222.0.20 () {60 vars in 1483 bytes} [Mon Aug 1 16:58:11 2022] PUT /identity/v3/projects/025776af0f594ddc9bfd2eedb6eb7602/users/d720864cb0854d549c05e425bffafa86/roles/0999d3da63a545f18ff3252afdc90518 => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 16:58:12.106585 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be627640-fff9-4d5a-918e-e3844cf67218 None None] REQUEST_METHOD: `POST` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:58:12.106880 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be627640-fff9-4d5a-918e-e3844cf67218 None None] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:58:12.108951 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be627640-fff9-4d5a-918e-e3844cf67218 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:58:12.134712 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.auth.core [None req-be627640-fff9-4d5a-918e-e3844cf67218 None None] MFA Rules not processed for user `1007f8e379b74e169b98fb7c3c0ba5b4`. Rule list: `[]` (Enabled: `True`). {{(pid=87057) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:58:12.136037 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.fernet_utils [None req-be627640-fff9-4d5a-918e-e3844cf67218 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87057) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:58:12.198508 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 1181/2371] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 1 16:58:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:58:12.209154 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a32a00-c96d-4945-b1d7-624b77dc80b6 admin admin] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:58:12.214256 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.fernet_utils [None req-18a32a00-c96d-4945-b1d7-624b77dc80b6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:58:12.274464 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a32a00-c96d-4945-b1d7-624b77dc80b6 admin admin] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:58:12.278140 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a82b40-56e9-41f3-9b9d-3285af93c3e6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d9bd8d649154c979be3744bfd016f95', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b992d6a65df4bf9b27c13192d57c5f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:58:12.278756 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a82b40-56e9-41f3-9b9d-3285af93c3e6 service neutron] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:58:12.279032 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a82b40-56e9-41f3-9b9d-3285af93c3e6 service neutron] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:58:12.279307 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a82b40-56e9-41f3-9b9d-3285af93c3e6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:58:12.289073 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41a82b40-56e9-41f3-9b9d-3285af93c3e6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:58:12.290538 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41a82b40-56e9-41f3-9b9d-3285af93c3e6 service neutron] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:58:12.300846 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 1191/2372] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 1 16:58:12 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 16:58:13.803015 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e4e34d-a0e0-4e2c-a4b9-8b3cfd3a6ec2 None None] REQUEST_METHOD: `POST` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:58:13.804240 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e4e34d-a0e0-4e2c-a4b9-8b3cfd3a6ec2 None None] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:58:13.804906 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e4e34d-a0e0-4e2c-a4b9-8b3cfd3a6ec2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:58:13.841233 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.auth.core [None req-36e4e34d-a0e0-4e2c-a4b9-8b3cfd3a6ec2 None None] MFA Rules not processed for user `d720864cb0854d549c05e425bffafa86`. Rule list: `[]` (Enabled: `True`). {{(pid=87057) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:58:13.842571 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.fernet_utils [None req-36e4e34d-a0e0-4e2c-a4b9-8b3cfd3a6ec2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87057) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:58:13.861476 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG dbcounter [-] [87057] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87057) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:58:13.915149 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 1182/2373] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 1 16:58:13 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:58:13.927073 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a82b40-56e9-41f3-9b9d-3285af93c3e6 service neutron] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:58:13.928456 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.fernet_utils [None req-41a82b40-56e9-41f3-9b9d-3285af93c3e6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:58:13.987604 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a82b40-56e9-41f3-9b9d-3285af93c3e6 service neutron] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:58:13.991496 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13496b7f-cab2-45f5-9104-575717c32bab service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25970b52ca15448fa1abb1f974d64038', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b992d6a65df4bf9b27c13192d57c5f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:58:13.992107 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13496b7f-cab2-45f5-9104-575717c32bab service cinder] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:58:13.992385 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13496b7f-cab2-45f5-9104-575717c32bab service cinder] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:58:13.992659 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13496b7f-cab2-45f5-9104-575717c32bab service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:58:14.001138 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13496b7f-cab2-45f5-9104-575717c32bab service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:58:14.002570 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13496b7f-cab2-45f5-9104-575717c32bab service cinder] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:58:14.011594 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 1192/2374] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 1 16:58:13 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 16:58:15.926812 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36e4e34d-a0e0-4e2c-a4b9-8b3cfd3a6ec2 None None] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:58:15.928260 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.fernet_utils [None req-36e4e34d-a0e0-4e2c-a4b9-8b3cfd3a6ec2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87057) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:58:15.942525 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36e4e34d-a0e0-4e2c-a4b9-8b3cfd3a6ec2 None None] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:58:15.947179 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b50ff70-f5a7-4809-9c1d-15dfc6a6b2b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:58:15.947817 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b50ff70-f5a7-4809-9c1d-15dfc6a6b2b3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:58:15.948213 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b50ff70-f5a7-4809-9c1d-15dfc6a6b2b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:58:15.948487 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b50ff70-f5a7-4809-9c1d-15dfc6a6b2b3 admin admin] PATH_INFO: `/v3/users/909efafdae624e82b72c304273e066a1` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:58:15.949648 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b50ff70-f5a7-4809-9c1d-15dfc6a6b2b3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=909efafdae624e82b72c304273e066a1)` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:58:15.951718 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b50ff70-f5a7-4809-9c1d-15dfc6a6b2b3 admin admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:58:16.258121 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.notifications [None req-0b50ff70-f5a7-4809-9c1d-15dfc6a6b2b3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '909efafdae624e82b72c304273e066a1'} {{(pid=87057) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:58:16.258842 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.notifications [None req-0b50ff70-f5a7-4809-9c1d-15dfc6a6b2b3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '909efafdae624e82b72c304273e066a1'} {{(pid=87057) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:58:16.317881 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.notifications [None req-0b50ff70-f5a7-4809-9c1d-15dfc6a6b2b3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '909efafdae624e82b72c304273e066a1'} {{(pid=87057) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:58:16.318562 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.notifications [None req-0b50ff70-f5a7-4809-9c1d-15dfc6a6b2b3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '909efafdae624e82b72c304273e066a1'} {{(pid=87057) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:58:16.326054 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.notifications [None req-0b50ff70-f5a7-4809-9c1d-15dfc6a6b2b3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '909efafdae624e82b72c304273e066a1'} {{(pid=87057) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:58:16.346588 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 1183/2375] 10.222.0.20 () {58 vars in 1224 bytes} [Mon Aug 1 16:58:15 2022] DELETE /identity/v3/users/909efafdae624e82b72c304273e066a1 => generated 0 bytes in 420 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 16:58:16.351495 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d21aee1-1890-4bfb-ab7a-ae05d25ac07d None None] REQUEST_METHOD: `POST` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:58:16.352323 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d21aee1-1890-4bfb-ab7a-ae05d25ac07d None None] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:58:16.354505 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d21aee1-1890-4bfb-ab7a-ae05d25ac07d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:58:16.372665 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.auth.core [None req-0d21aee1-1890-4bfb-ab7a-ae05d25ac07d None None] MFA Rules not processed for user `1007f8e379b74e169b98fb7c3c0ba5b4`. Rule list: `[]` (Enabled: `True`). {{(pid=87058) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:58:16.373967 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.fernet_utils [None req-0d21aee1-1890-4bfb-ab7a-ae05d25ac07d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:58:16.447733 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG dbcounter [-] [87058] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=87058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:58:16.461953 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 1193/2376] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 1 16:58:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:58:16.472428 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b50ff70-f5a7-4809-9c1d-15dfc6a6b2b3 admin admin] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:58:16.473960 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.fernet_utils [None req-0b50ff70-f5a7-4809-9c1d-15dfc6a6b2b3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87057) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:58:16.532381 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b50ff70-f5a7-4809-9c1d-15dfc6a6b2b3 admin admin] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:58:16.536127 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea03615-345e-42b7-8a45-015b1ceae922 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d9bd8d649154c979be3744bfd016f95', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b992d6a65df4bf9b27c13192d57c5f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:58:16.536754 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea03615-345e-42b7-8a45-015b1ceae922 service neutron] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:58:16.537033 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea03615-345e-42b7-8a45-015b1ceae922 service neutron] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:58:16.537313 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea03615-345e-42b7-8a45-015b1ceae922 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:58:16.547731 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ea03615-345e-42b7-8a45-015b1ceae922 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:58:16.549463 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ea03615-345e-42b7-8a45-015b1ceae922 service neutron] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:58:16.559572 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 1184/2377] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 1 16:58:16 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 16:58:16.673988 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3c0ceb-b0a9-4030-995d-31e7ed79544e None None] REQUEST_METHOD: `POST` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:58:16.674603 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3c0ceb-b0a9-4030-995d-31e7ed79544e None None] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:58:16.675020 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3c0ceb-b0a9-4030-995d-31e7ed79544e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:58:16.694824 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.auth.core [None req-ac3c0ceb-b0a9-4030-995d-31e7ed79544e None None] MFA Rules not processed for user `d720864cb0854d549c05e425bffafa86`. Rule list: `[]` (Enabled: `True`). {{(pid=87058) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:58:16.696313 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.fernet_utils [None req-ac3c0ceb-b0a9-4030-995d-31e7ed79544e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:58:16.756734 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 1194/2378] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 1 16:58:16 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:58:17.417447 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea03615-345e-42b7-8a45-015b1ceae922 service neutron] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:58:17.420370 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.fernet_utils [None req-6ea03615-345e-42b7-8a45-015b1ceae922 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87057) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:58:17.451642 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea03615-345e-42b7-8a45-015b1ceae922 service neutron] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:58:17.458199 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d9d3be-86a8-4bac-b26a-11dff8b086f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:58:17.458974 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d9d3be-86a8-4bac-b26a-11dff8b086f6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:58:17.459316 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d9d3be-86a8-4bac-b26a-11dff8b086f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:58:17.459927 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d9d3be-86a8-4bac-b26a-11dff8b086f6 admin admin] PATH_INFO: `/v3/projects/3c22a11b9ed64d57a8e8f750947e11cb` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:58:17.461446 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32d9d3be-86a8-4bac-b26a-11dff8b086f6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3c22a11b9ed64d57a8e8f750947e11cb)` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:58:17.464082 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32d9d3be-86a8-4bac-b26a-11dff8b086f6 admin admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:58:17.566739 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG dbcounter [-] [87057] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=87057) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:58:17.582294 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.notifications [None req-32d9d3be-86a8-4bac-b26a-11dff8b086f6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3c22a11b9ed64d57a8e8f750947e11cb', 'request_id': 'req-32d9d3be-86a8-4bac-b26a-11dff8b086f6'} {{(pid=87057) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:58:17.588562 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.notifications [None req-32d9d3be-86a8-4bac-b26a-11dff8b086f6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3c22a11b9ed64d57a8e8f750947e11cb', 'request_id': 'req-32d9d3be-86a8-4bac-b26a-11dff8b086f6'} {{(pid=87057) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:58:17.594661 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.notifications [None req-32d9d3be-86a8-4bac-b26a-11dff8b086f6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3c22a11b9ed64d57a8e8f750947e11cb', 'request_id': 'req-32d9d3be-86a8-4bac-b26a-11dff8b086f6'} {{(pid=87057) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:58:17.654022 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.notifications [None req-32d9d3be-86a8-4bac-b26a-11dff8b086f6 admin admin] The token cache is being invalidate because project 3c22a11b9ed64d57a8e8f750947e11cb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87057) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 16:58:17.654341 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.notifications [None req-32d9d3be-86a8-4bac-b26a-11dff8b086f6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87057) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:58:17.655065 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.notifications [None req-32d9d3be-86a8-4bac-b26a-11dff8b086f6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87057) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:58:17.657165 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 1185/2379] 10.222.0.20 () {58 vars in 1233 bytes} [Mon Aug 1 16:58:17 2022] DELETE /identity/v3/projects/3c22a11b9ed64d57a8e8f750947e11cb => generated 0 bytes in 240 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 16:58:26.749092 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG dbcounter [-] [87058] Writing DB stats keystone:SELECT=27 {{(pid=87058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:58:27.826375 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG dbcounter [-] [87057] Writing DB stats keystone:SELECT=14,keystone:DELETE=5,keystone:INSERT=1 {{(pid=87057) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:58:37.051531 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac3c0ceb-b0a9-4030-995d-31e7ed79544e None None] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:58:37.052954 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.fernet_utils [None req-ac3c0ceb-b0a9-4030-995d-31e7ed79544e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:58:37.067808 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac3c0ceb-b0a9-4030-995d-31e7ed79544e None None] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:58:37.121656 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff42dba0-a0ba-47e7-9440-816665eab3ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:58:37.122313 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff42dba0-a0ba-47e7-9440-816665eab3ec admin admin] REQUEST_METHOD: `DELETE` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:58:37.122597 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff42dba0-a0ba-47e7-9440-816665eab3ec admin admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:58:37.122865 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff42dba0-a0ba-47e7-9440-816665eab3ec admin admin] PATH_INFO: `/v3/users/f56ebe2d12ed45b9ba1f6dec45fda595` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:58:37.124016 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff42dba0-a0ba-47e7-9440-816665eab3ec admin admin] RBAC: Authorizing `identity:delete_user(user_id=f56ebe2d12ed45b9ba1f6dec45fda595)` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:58:37.125928 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff42dba0-a0ba-47e7-9440-816665eab3ec admin admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:58:37.273470 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.notifications [None req-ff42dba0-a0ba-47e7-9440-816665eab3ec admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f56ebe2d12ed45b9ba1f6dec45fda595'} {{(pid=87058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:58:37.274143 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.notifications [None req-ff42dba0-a0ba-47e7-9440-816665eab3ec admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f56ebe2d12ed45b9ba1f6dec45fda595'} {{(pid=87058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:58:37.274751 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.notifications [None req-ff42dba0-a0ba-47e7-9440-816665eab3ec admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f56ebe2d12ed45b9ba1f6dec45fda595'} {{(pid=87058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:58:37.286438 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.notifications [None req-ff42dba0-a0ba-47e7-9440-816665eab3ec admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f56ebe2d12ed45b9ba1f6dec45fda595'} {{(pid=87058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:58:37.293417 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.notifications [None req-ff42dba0-a0ba-47e7-9440-816665eab3ec admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f56ebe2d12ed45b9ba1f6dec45fda595'} {{(pid=87058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:58:37.363309 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 1195/2380] 10.222.0.20 () {58 vars in 1224 bytes} [Mon Aug 1 16:58:37 2022] DELETE /identity/v3/users/f56ebe2d12ed45b9ba1f6dec45fda595 => generated 0 bytes in 312 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 16:58:37.368418 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e6920f-eb56-4f17-b479-394ebc86bceb None None] REQUEST_METHOD: `POST` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:58:37.368919 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e6920f-eb56-4f17-b479-394ebc86bceb None None] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:58:37.369016 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e6920f-eb56-4f17-b479-394ebc86bceb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:58:37.388704 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.auth.core [None req-a2e6920f-eb56-4f17-b479-394ebc86bceb None None] MFA Rules not processed for user `1007f8e379b74e169b98fb7c3c0ba5b4`. Rule list: `[]` (Enabled: `True`). {{(pid=87057) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:58:37.389925 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.fernet_utils [None req-a2e6920f-eb56-4f17-b479-394ebc86bceb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87057) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:58:37.449147 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 1186/2381] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 1 16:58:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:58:37.458991 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff42dba0-a0ba-47e7-9440-816665eab3ec admin admin] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:58:37.460429 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.fernet_utils [None req-ff42dba0-a0ba-47e7-9440-816665eab3ec admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:58:37.518551 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff42dba0-a0ba-47e7-9440-816665eab3ec admin admin] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:58:37.522295 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-688dc174-e718-4db1-8778-a5f4dfed72fb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d9bd8d649154c979be3744bfd016f95', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b992d6a65df4bf9b27c13192d57c5f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:58:37.522902 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688dc174-e718-4db1-8778-a5f4dfed72fb service neutron] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:58:37.523179 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688dc174-e718-4db1-8778-a5f4dfed72fb service neutron] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:58:37.523452 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688dc174-e718-4db1-8778-a5f4dfed72fb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:58:37.532437 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-688dc174-e718-4db1-8778-a5f4dfed72fb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:58:37.533789 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-688dc174-e718-4db1-8778-a5f4dfed72fb service neutron] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:58:37.543882 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 1196/2382] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 1 16:58:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 16:58:37.925033 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2e6920f-eb56-4f17-b479-394ebc86bceb None None] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:58:37.926438 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.fernet_utils [None req-a2e6920f-eb56-4f17-b479-394ebc86bceb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87057) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:58:37.938631 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2e6920f-eb56-4f17-b479-394ebc86bceb None None] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:58:37.964152 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f37a72-6298-43c8-8aa2-7deb44fb416b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:58:37.964817 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f37a72-6298-43c8-8aa2-7deb44fb416b admin admin] REQUEST_METHOD: `DELETE` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:58:37.965128 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f37a72-6298-43c8-8aa2-7deb44fb416b admin admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:58:37.965415 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f37a72-6298-43c8-8aa2-7deb44fb416b admin admin] PATH_INFO: `/v3/projects/1ddce3c8d30f4343a6c94f95caebcd97` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:58:37.966617 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2f37a72-6298-43c8-8aa2-7deb44fb416b admin admin] RBAC: Authorizing `identity:delete_project(project_id=1ddce3c8d30f4343a6c94f95caebcd97)` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:58:37.968582 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2f37a72-6298-43c8-8aa2-7deb44fb416b admin admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:58:38.113051 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.notifications [None req-b2f37a72-6298-43c8-8aa2-7deb44fb416b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1ddce3c8d30f4343a6c94f95caebcd97', 'request_id': 'req-b2f37a72-6298-43c8-8aa2-7deb44fb416b'} {{(pid=87057) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:58:38.119564 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.notifications [None req-b2f37a72-6298-43c8-8aa2-7deb44fb416b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1ddce3c8d30f4343a6c94f95caebcd97', 'request_id': 'req-b2f37a72-6298-43c8-8aa2-7deb44fb416b'} {{(pid=87057) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:58:38.125063 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.notifications [None req-b2f37a72-6298-43c8-8aa2-7deb44fb416b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1ddce3c8d30f4343a6c94f95caebcd97', 'request_id': 'req-b2f37a72-6298-43c8-8aa2-7deb44fb416b'} {{(pid=87057) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:58:38.208372 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.notifications [None req-b2f37a72-6298-43c8-8aa2-7deb44fb416b admin admin] The token cache is being invalidate because project 1ddce3c8d30f4343a6c94f95caebcd97 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87057) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 16:58:38.208660 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.notifications [None req-b2f37a72-6298-43c8-8aa2-7deb44fb416b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87057) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:58:38.209296 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.notifications [None req-b2f37a72-6298-43c8-8aa2-7deb44fb416b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87057) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:58:38.210868 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 1187/2383] 10.222.0.20 () {58 vars in 1233 bytes} [Mon Aug 1 16:58:37 2022] DELETE /identity/v3/projects/1ddce3c8d30f4343a6c94f95caebcd97 => generated 0 bytes in 286 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 16:58:38.215760 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e12895b-08df-4f64-ab49-4ac11044b22e None None] REQUEST_METHOD: `POST` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:58:38.216264 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e12895b-08df-4f64-ab49-4ac11044b22e None None] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:58:38.216363 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e12895b-08df-4f64-ab49-4ac11044b22e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:58:38.236576 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.auth.core [None req-2e12895b-08df-4f64-ab49-4ac11044b22e None None] MFA Rules not processed for user `1007f8e379b74e169b98fb7c3c0ba5b4`. Rule list: `[]` (Enabled: `True`). {{(pid=87058) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:58:38.237855 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.fernet_utils [None req-2e12895b-08df-4f64-ab49-4ac11044b22e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:58:38.255553 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG dbcounter [-] [87058] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:58:38.297821 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 1197/2384] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 1 16:58:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 16:58:38.306099 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84d8800-504e-4055-b9f5-9bffbb54f360 None None] REQUEST_METHOD: `POST` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:58:38.306340 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84d8800-504e-4055-b9f5-9bffbb54f360 None None] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:58:38.306619 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84d8800-504e-4055-b9f5-9bffbb54f360 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:58:38.328479 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.auth.core [None req-d84d8800-504e-4055-b9f5-9bffbb54f360 None None] MFA Rules not processed for user `1007f8e379b74e169b98fb7c3c0ba5b4`. Rule list: `[]` (Enabled: `True`). {{(pid=87057) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:58:38.329693 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.fernet_utils [None req-d84d8800-504e-4055-b9f5-9bffbb54f360 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87057) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:58:38.338267 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 1188/2385] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 1 16:58:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:58:38.342550 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e12895b-08df-4f64-ab49-4ac11044b22e None None] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:58:38.352250 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e12895b-08df-4f64-ab49-4ac11044b22e None None] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:58:38.356821 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c0ea98-acd6-4b65-ae4d-c7778ae2dac0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:58:38.357424 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c0ea98-acd6-4b65-ae4d-c7778ae2dac0 admin admin] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:58:38.357703 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c0ea98-acd6-4b65-ae4d-c7778ae2dac0 admin admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:58:38.357974 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c0ea98-acd6-4b65-ae4d-c7778ae2dac0 admin admin] PATH_INFO: `/v3/domains` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:58:38.358725 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0c0ea98-acd6-4b65-ae4d-c7778ae2dac0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87058) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:58:38.359052 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0c0ea98-acd6-4b65-ae4d-c7778ae2dac0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:58:38.360633 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0c0ea98-acd6-4b65-ae4d-c7778ae2dac0 admin admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:58:38.372413 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 1198/2386] 10.222.0.20 () {58 vars in 1153 bytes} [Mon Aug 1 16:58:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:58:38.376603 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d84d8800-504e-4055-b9f5-9bffbb54f360 None None] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:58:38.386776 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d84d8800-504e-4055-b9f5-9bffbb54f360 None None] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:58:38.391422 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f0b1a82-3e52-4798-a918-5607222a47d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:58:38.392024 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0b1a82-3e52-4798-a918-5607222a47d6 admin admin] REQUEST_METHOD: `POST` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:58:38.392303 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0b1a82-3e52-4798-a918-5607222a47d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:58:38.392576 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0b1a82-3e52-4798-a918-5607222a47d6 admin admin] PATH_INFO: `/v3/projects` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:58:38.393296 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f0b1a82-3e52-4798-a918-5607222a47d6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:58:38.395771 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f0b1a82-3e52-4798-a918-5607222a47d6 admin admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:58:38.518869 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 1189/2387] 10.222.0.20 () {60 vars in 1153 bytes} [Mon Aug 1 16:58:38 2022] POST /identity/v3/projects => generated 375 bytes in 142 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 16:58:38.523185 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c0ea98-acd6-4b65-ae4d-c7778ae2dac0 admin admin] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:58:38.534081 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c0ea98-acd6-4b65-ae4d-c7778ae2dac0 admin admin] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:58:38.590710 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0186efb-9a57-4bd9-b8fc-9203a351f655 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:58:38.591290 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0186efb-9a57-4bd9-b8fc-9203a351f655 admin admin] REQUEST_METHOD: `POST` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:58:38.591575 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0186efb-9a57-4bd9-b8fc-9203a351f655 admin admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:58:38.591852 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0186efb-9a57-4bd9-b8fc-9203a351f655 admin admin] PATH_INFO: `/v3/users` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:58:38.592525 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0186efb-9a57-4bd9-b8fc-9203a351f655 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:58:38.594494 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0186efb-9a57-4bd9-b8fc-9203a351f655 admin admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:58:38.678460 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 1199/2388] 10.222.0.20 () {60 vars in 1144 bytes} [Mon Aug 1 16:58:38 2022] POST /identity/v3/users => generated 288 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 16:58:38.682574 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f0b1a82-3e52-4798-a918-5607222a47d6 admin admin] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:58:38.693874 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f0b1a82-3e52-4798-a918-5607222a47d6 admin admin] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:58:38.698685 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-892cfe1c-4edc-40b1-a9bb-6b2dfbf30a2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:58:38.699300 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892cfe1c-4edc-40b1-a9bb-6b2dfbf30a2e admin admin] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:58:38.699585 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892cfe1c-4edc-40b1-a9bb-6b2dfbf30a2e admin admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:58:38.699864 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892cfe1c-4edc-40b1-a9bb-6b2dfbf30a2e admin admin] PATH_INFO: `/v3/roles` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:58:38.700568 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-892cfe1c-4edc-40b1-a9bb-6b2dfbf30a2e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:58:38.702205 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-892cfe1c-4edc-40b1-a9bb-6b2dfbf30a2e admin admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:58:38.712863 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 1190/2389] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Aug 1 16:58:38 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 16:58:38.717032 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0186efb-9a57-4bd9-b8fc-9203a351f655 admin admin] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:58:38.727651 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0186efb-9a57-4bd9-b8fc-9203a351f655 admin admin] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:58:38.732390 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-337bc8e3-3222-45f5-8f79-3963bca5be6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:58:38.736190 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337bc8e3-3222-45f5-8f79-3963bca5be6d admin admin] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:58:38.736190 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337bc8e3-3222-45f5-8f79-3963bca5be6d admin admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:58:38.736190 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337bc8e3-3222-45f5-8f79-3963bca5be6d admin admin] PATH_INFO: `/v3/roles` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:58:38.736190 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-337bc8e3-3222-45f5-8f79-3963bca5be6d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:58:38.736190 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-337bc8e3-3222-45f5-8f79-3963bca5be6d admin admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:58:38.747600 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 1200/2390] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Aug 1 16:58:38 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 16:58:38.751732 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-892cfe1c-4edc-40b1-a9bb-6b2dfbf30a2e admin admin] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:58:38.762502 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-892cfe1c-4edc-40b1-a9bb-6b2dfbf30a2e admin admin] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:58:38.767184 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e32f95d-c053-4b10-97ef-57b02bd39b52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:58:38.767834 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e32f95d-c053-4b10-97ef-57b02bd39b52 admin admin] REQUEST_METHOD: `PUT` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:58:38.768114 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e32f95d-c053-4b10-97ef-57b02bd39b52 admin admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:58:38.768453 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e32f95d-c053-4b10-97ef-57b02bd39b52 admin admin] PATH_INFO: `/v3/projects/ce67b7a6f3034531936a300a6a3cdc90/users/0320d6dd02b54fcca5ac1240b5eff961/roles/0999d3da63a545f18ff3252afdc90518` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:58:38.784849 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e32f95d-c053-4b10-97ef-57b02bd39b52 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ce67b7a6f3034531936a300a6a3cdc90, user_id=0320d6dd02b54fcca5ac1240b5eff961, role_id=0999d3da63a545f18ff3252afdc90518)` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:58:38.787658 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e32f95d-c053-4b10-97ef-57b02bd39b52 admin admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:58:38.837444 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 1191/2391] 10.222.0.20 () {60 vars in 1483 bytes} [Mon Aug 1 16:58:38 2022] PUT /identity/v3/projects/ce67b7a6f3034531936a300a6a3cdc90/users/0320d6dd02b54fcca5ac1240b5eff961/roles/0999d3da63a545f18ff3252afdc90518 => generated 0 bytes in 86 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 16:58:38.851449 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1285a6bc-e3a3-416e-8ea9-dcf3d0104e58 None None] REQUEST_METHOD: `POST` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:58:38.851706 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1285a6bc-e3a3-416e-8ea9-dcf3d0104e58 None None] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:58:38.851986 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1285a6bc-e3a3-416e-8ea9-dcf3d0104e58 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:58:38.881274 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.auth.core [None req-1285a6bc-e3a3-416e-8ea9-dcf3d0104e58 None None] MFA Rules not processed for user `0320d6dd02b54fcca5ac1240b5eff961`. Rule list: `[]` (Enabled: `True`). {{(pid=87058) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:58:38.883369 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.fernet_utils [None req-1285a6bc-e3a3-416e-8ea9-dcf3d0104e58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:58:38.947181 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 1201/2392] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 1 16:58:38 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:58:38.961513 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e32f95d-c053-4b10-97ef-57b02bd39b52 admin admin] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:58:38.963281 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.fernet_utils [None req-9e32f95d-c053-4b10-97ef-57b02bd39b52 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87057) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:58:38.985260 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG dbcounter [-] [87057] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=87057) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:58:39.021577 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e32f95d-c053-4b10-97ef-57b02bd39b52 admin admin] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:58:39.025526 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd76573-1e88-46c0-ac73-c2589ff9269d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25970b52ca15448fa1abb1f974d64038', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b992d6a65df4bf9b27c13192d57c5f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:58:39.026175 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd76573-1e88-46c0-ac73-c2589ff9269d service cinder] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:58:39.026619 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd76573-1e88-46c0-ac73-c2589ff9269d service cinder] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:58:39.027232 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd76573-1e88-46c0-ac73-c2589ff9269d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:58:39.035993 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecd76573-1e88-46c0-ac73-c2589ff9269d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:58:39.039070 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecd76573-1e88-46c0-ac73-c2589ff9269d service cinder] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:58:39.046364 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 1192/2393] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 1 16:58:38 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 16:58:41.804027 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84be2c1-8722-4ea0-8a54-54d84f4b6467 None None] REQUEST_METHOD: `POST` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:58:41.804630 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84be2c1-8722-4ea0-8a54-54d84f4b6467 None None] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:58:41.804630 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84be2c1-8722-4ea0-8a54-54d84f4b6467 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:58:41.824401 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.auth.core [None req-f84be2c1-8722-4ea0-8a54-54d84f4b6467 None None] MFA Rules not processed for user `0320d6dd02b54fcca5ac1240b5eff961`. Rule list: `[]` (Enabled: `True`). {{(pid=87058) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:58:41.825610 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.fernet_utils [None req-f84be2c1-8722-4ea0-8a54-54d84f4b6467 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:58:41.833063 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 1202/2394] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 1 16:58:41 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:58:42.646617 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd76573-1e88-46c0-ac73-c2589ff9269d service cinder] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:58:42.648089 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.fernet_utils [None req-ecd76573-1e88-46c0-ac73-c2589ff9269d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87057) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:58:42.705585 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd76573-1e88-46c0-ac73-c2589ff9269d service cinder] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:58:42.709435 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee9abb8-a985-458e-96ea-42504487b2c2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d9bd8d649154c979be3744bfd016f95', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b992d6a65df4bf9b27c13192d57c5f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:58:42.710204 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee9abb8-a985-458e-96ea-42504487b2c2 service neutron] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:58:42.710402 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee9abb8-a985-458e-96ea-42504487b2c2 service neutron] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:58:42.710685 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee9abb8-a985-458e-96ea-42504487b2c2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:58:42.712366 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.fernet_utils [None req-2ee9abb8-a985-458e-96ea-42504487b2c2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87057) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:58:42.781908 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ee9abb8-a985-458e-96ea-42504487b2c2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:58:42.783514 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ee9abb8-a985-458e-96ea-42504487b2c2 service neutron] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:58:42.794498 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 1193/2395] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 1 16:58:42 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 148 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 01 16:58:51.819037 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG dbcounter [-] [87058] Writing DB stats keystone:SELECT=92,keystone:INSERT=3 {{(pid=87058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:58:52.789482 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG dbcounter [-] [87057] Writing DB stats keystone:SELECT=67 {{(pid=87057) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:59:04.219998 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c17b554-9260-4afe-9728-d139bf06d2ab None None] REQUEST_METHOD: `POST` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:59:04.220523 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c17b554-9260-4afe-9728-d139bf06d2ab None None] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:59:04.220523 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c17b554-9260-4afe-9728-d139bf06d2ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:59:04.239391 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.auth.core [None req-2c17b554-9260-4afe-9728-d139bf06d2ab None None] MFA Rules not processed for user `b6a93d1c6c82489099adc8b413280fc0`. Rule list: `[]` (Enabled: `True`). {{(pid=87058) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:59:04.240570 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.fernet_utils [None req-2c17b554-9260-4afe-9728-d139bf06d2ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:59:04.297051 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 1203/2396] 10.222.0.20 () {58 vars in 1009 bytes} [Mon Aug 1 16:59:04 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:59:04.340780 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee9abb8-a985-458e-96ea-42504487b2c2 service neutron] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:59:04.342311 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.fernet_utils [None req-2ee9abb8-a985-458e-96ea-42504487b2c2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87057) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:59:04.411110 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee9abb8-a985-458e-96ea-42504487b2c2 service neutron] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:59:04.416220 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cb47eb8-4b7e-49a0-bc55-a8e826e65e5b service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ab4add1faa04faa811ef7a9341cc35c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b992d6a65df4bf9b27c13192d57c5f0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:59:04.416827 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb47eb8-4b7e-49a0-bc55-a8e826e65e5b service swift] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:59:04.417107 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb47eb8-4b7e-49a0-bc55-a8e826e65e5b service swift] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:59:04.417380 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb47eb8-4b7e-49a0-bc55-a8e826e65e5b service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:59:04.425344 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cb47eb8-4b7e-49a0-bc55-a8e826e65e5b service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:59:04.426972 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cb47eb8-4b7e-49a0-bc55-a8e826e65e5b service swift] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:59:04.435174 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 1194/2397] 10.222.0.20 () {60 vars in 1384 bytes} [Mon Aug 1 16:59:04 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 01 16:59:05.255660 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66a810e-cb52-4870-80ed-ddc1025db6a7 None None] REQUEST_METHOD: `POST` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:59:05.256123 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66a810e-cb52-4870-80ed-ddc1025db6a7 None None] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:59:05.256231 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66a810e-cb52-4870-80ed-ddc1025db6a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:59:05.276537 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.auth.core [None req-c66a810e-cb52-4870-80ed-ddc1025db6a7 None None] MFA Rules not processed for user `b6a93d1c6c82489099adc8b413280fc0`. Rule list: `[]` (Enabled: `True`). {{(pid=87058) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:59:05.277767 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.fernet_utils [None req-c66a810e-cb52-4870-80ed-ddc1025db6a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:59:05.284999 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 1204/2398] 10.222.0.20 () {58 vars in 967 bytes} [Mon Aug 1 16:59:05 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:59:05.297973 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cb47eb8-4b7e-49a0-bc55-a8e826e65e5b service swift] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:59:05.308321 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cb47eb8-4b7e-49a0-bc55-a8e826e65e5b service swift] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:59:05.313274 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c32c4b1f-5dfa-4155-bcb4-249c33504ec9 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ab4add1faa04faa811ef7a9341cc35c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b992d6a65df4bf9b27c13192d57c5f0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:59:05.313868 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32c4b1f-5dfa-4155-bcb4-249c33504ec9 service swift] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:59:05.314173 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32c4b1f-5dfa-4155-bcb4-249c33504ec9 service swift] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:59:05.314446 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32c4b1f-5dfa-4155-bcb4-249c33504ec9 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:59:05.322938 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c32c4b1f-5dfa-4155-bcb4-249c33504ec9 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:59:05.324589 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c32c4b1f-5dfa-4155-bcb4-249c33504ec9 service swift] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:59:05.332883 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 1195/2399] 10.222.0.20 () {60 vars in 1384 bytes} [Mon Aug 1 16:59:05 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 01 16:59:05.970095 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c9749c-0332-43b7-8a4d-1670eb294b5e None None] REQUEST_METHOD: `POST` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:59:05.970390 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c9749c-0332-43b7-8a4d-1670eb294b5e None None] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:59:05.970666 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c9749c-0332-43b7-8a4d-1670eb294b5e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:59:05.990396 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.auth.core [None req-b4c9749c-0332-43b7-8a4d-1670eb294b5e None None] MFA Rules not processed for user `b6a93d1c6c82489099adc8b413280fc0`. Rule list: `[]` (Enabled: `True`). {{(pid=87058) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:59:05.991923 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.fernet_utils [None req-b4c9749c-0332-43b7-8a4d-1670eb294b5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:59:05.999221 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 1205/2400] 10.222.0.20 () {58 vars in 1009 bytes} [Mon Aug 1 16:59:05 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:59:06.013058 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c32c4b1f-5dfa-4155-bcb4-249c33504ec9 service swift] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:59:06.021989 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c32c4b1f-5dfa-4155-bcb4-249c33504ec9 service swift] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:59:06.027718 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ff8a9d-de37-4f61-956c-1a4efd007fd7 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ab4add1faa04faa811ef7a9341cc35c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b992d6a65df4bf9b27c13192d57c5f0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:59:06.028312 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ff8a9d-de37-4f61-956c-1a4efd007fd7 service swift] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:59:06.028590 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ff8a9d-de37-4f61-956c-1a4efd007fd7 service swift] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:59:06.028862 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ff8a9d-de37-4f61-956c-1a4efd007fd7 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:59:06.037082 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30ff8a9d-de37-4f61-956c-1a4efd007fd7 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:59:06.047529 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30ff8a9d-de37-4f61-956c-1a4efd007fd7 service swift] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:59:06.047529 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 1196/2401] 10.222.0.20 () {60 vars in 1384 bytes} [Mon Aug 1 16:59:06 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 01 16:59:06.701266 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cfc3086-5677-4508-a5d0-ac19dfe1b57f None None] REQUEST_METHOD: `POST` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:59:06.701748 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cfc3086-5677-4508-a5d0-ac19dfe1b57f None None] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:59:06.701852 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cfc3086-5677-4508-a5d0-ac19dfe1b57f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:59:06.722806 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.auth.core [None req-7cfc3086-5677-4508-a5d0-ac19dfe1b57f None None] MFA Rules not processed for user `b6a93d1c6c82489099adc8b413280fc0`. Rule list: `[]` (Enabled: `True`). {{(pid=87058) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:59:06.724001 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.fernet_utils [None req-7cfc3086-5677-4508-a5d0-ac19dfe1b57f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:59:06.731354 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 1206/2402] 10.222.0.20 () {58 vars in 967 bytes} [Mon Aug 1 16:59:06 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:59:06.744352 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ff8a9d-de37-4f61-956c-1a4efd007fd7 service swift] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:59:06.755500 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ff8a9d-de37-4f61-956c-1a4efd007fd7 service swift] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:59:06.761246 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddc473cd-a6f0-4ac9-81d2-62edd3e8a64b service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ab4add1faa04faa811ef7a9341cc35c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b992d6a65df4bf9b27c13192d57c5f0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:59:06.762086 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc473cd-a6f0-4ac9-81d2-62edd3e8a64b service swift] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:59:06.762369 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc473cd-a6f0-4ac9-81d2-62edd3e8a64b service swift] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:59:06.762646 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc473cd-a6f0-4ac9-81d2-62edd3e8a64b service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:59:06.771277 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddc473cd-a6f0-4ac9-81d2-62edd3e8a64b service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:59:06.772854 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddc473cd-a6f0-4ac9-81d2-62edd3e8a64b service swift] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:59:06.781567 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 1197/2403] 10.222.0.20 () {60 vars in 1384 bytes} [Mon Aug 1 16:59:06 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 01 16:59:09.687258 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cfc3086-5677-4508-a5d0-ac19dfe1b57f None None] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:59:09.749153 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cfc3086-5677-4508-a5d0-ac19dfe1b57f None None] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:59:09.753611 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e4f956d-d083-46c2-a60b-d92ccf749006 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:59:09.754278 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4f956d-d083-46c2-a60b-d92ccf749006 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:59:09.754560 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4f956d-d083-46c2-a60b-d92ccf749006 admin admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:59:09.754833 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4f956d-d083-46c2-a60b-d92ccf749006 admin admin] PATH_INFO: `/v3/users/0320d6dd02b54fcca5ac1240b5eff961` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:59:09.755987 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e4f956d-d083-46c2-a60b-d92ccf749006 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0320d6dd02b54fcca5ac1240b5eff961)` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:59:09.757899 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e4f956d-d083-46c2-a60b-d92ccf749006 admin admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:59:09.948175 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.notifications [None req-6e4f956d-d083-46c2-a60b-d92ccf749006 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0320d6dd02b54fcca5ac1240b5eff961'} {{(pid=87058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:59:09.948913 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.notifications [None req-6e4f956d-d083-46c2-a60b-d92ccf749006 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0320d6dd02b54fcca5ac1240b5eff961'} {{(pid=87058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:59:09.949552 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.notifications [None req-6e4f956d-d083-46c2-a60b-d92ccf749006 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0320d6dd02b54fcca5ac1240b5eff961'} {{(pid=87058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:59:09.961380 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.notifications [None req-6e4f956d-d083-46c2-a60b-d92ccf749006 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0320d6dd02b54fcca5ac1240b5eff961'} {{(pid=87058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:59:09.968732 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.notifications [None req-6e4f956d-d083-46c2-a60b-d92ccf749006 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0320d6dd02b54fcca5ac1240b5eff961'} {{(pid=87058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:59:10.037168 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 1207/2404] 10.222.0.20 () {58 vars in 1224 bytes} [Mon Aug 1 16:59:09 2022] DELETE /identity/v3/users/0320d6dd02b54fcca5ac1240b5eff961 => generated 0 bytes in 350 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 16:59:10.041667 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b532c4-9a9c-4512-be4e-229ffb713944 None None] REQUEST_METHOD: `POST` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:59:10.042201 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b532c4-9a9c-4512-be4e-229ffb713944 None None] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:59:10.042315 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b532c4-9a9c-4512-be4e-229ffb713944 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:59:10.061973 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.auth.core [None req-10b532c4-9a9c-4512-be4e-229ffb713944 None None] MFA Rules not processed for user `1007f8e379b74e169b98fb7c3c0ba5b4`. Rule list: `[]` (Enabled: `True`). {{(pid=87057) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:59:10.063224 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.fernet_utils [None req-10b532c4-9a9c-4512-be4e-229ffb713944 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87057) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:59:10.120974 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 1198/2405] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 1 16:59:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:59:10.130519 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e4f956d-d083-46c2-a60b-d92ccf749006 admin admin] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:59:10.132006 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.fernet_utils [None req-6e4f956d-d083-46c2-a60b-d92ccf749006 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:59:10.138703 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG dbcounter [-] [87058] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:59:10.190279 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e4f956d-d083-46c2-a60b-d92ccf749006 admin admin] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:59:10.194418 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df9b795b-116d-45f0-a9fe-55d30d6da56f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d9bd8d649154c979be3744bfd016f95', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b992d6a65df4bf9b27c13192d57c5f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:59:10.195034 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9b795b-116d-45f0-a9fe-55d30d6da56f service neutron] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:59:10.195313 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9b795b-116d-45f0-a9fe-55d30d6da56f service neutron] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:59:10.195591 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9b795b-116d-45f0-a9fe-55d30d6da56f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:59:10.204833 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df9b795b-116d-45f0-a9fe-55d30d6da56f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:59:10.206372 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df9b795b-116d-45f0-a9fe-55d30d6da56f service neutron] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:59:10.217091 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 1208/2406] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 1 16:59:10 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 16:59:10.473239 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b532c4-9a9c-4512-be4e-229ffb713944 None None] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:59:10.474811 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.fernet_utils [None req-10b532c4-9a9c-4512-be4e-229ffb713944 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87057) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:59:10.487434 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b532c4-9a9c-4512-be4e-229ffb713944 None None] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:59:10.492376 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb4d343d-bc8f-4cf3-8d42-05222fdb7fb1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:59:10.493029 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4d343d-bc8f-4cf3-8d42-05222fdb7fb1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:59:10.493311 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4d343d-bc8f-4cf3-8d42-05222fdb7fb1 admin admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:59:10.493588 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4d343d-bc8f-4cf3-8d42-05222fdb7fb1 admin admin] PATH_INFO: `/v3/projects/ce67b7a6f3034531936a300a6a3cdc90` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:59:10.494833 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb4d343d-bc8f-4cf3-8d42-05222fdb7fb1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ce67b7a6f3034531936a300a6a3cdc90)` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:59:10.496867 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb4d343d-bc8f-4cf3-8d42-05222fdb7fb1 admin admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:59:10.591250 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG dbcounter [-] [87057] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=87057) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:59:10.594055 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.notifications [None req-fb4d343d-bc8f-4cf3-8d42-05222fdb7fb1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ce67b7a6f3034531936a300a6a3cdc90', 'request_id': 'req-fb4d343d-bc8f-4cf3-8d42-05222fdb7fb1'} {{(pid=87057) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:59:10.600161 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.notifications [None req-fb4d343d-bc8f-4cf3-8d42-05222fdb7fb1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ce67b7a6f3034531936a300a6a3cdc90', 'request_id': 'req-fb4d343d-bc8f-4cf3-8d42-05222fdb7fb1'} {{(pid=87057) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:59:10.605808 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.notifications [None req-fb4d343d-bc8f-4cf3-8d42-05222fdb7fb1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ce67b7a6f3034531936a300a6a3cdc90', 'request_id': 'req-fb4d343d-bc8f-4cf3-8d42-05222fdb7fb1'} {{(pid=87057) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:59:10.654731 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.notifications [None req-fb4d343d-bc8f-4cf3-8d42-05222fdb7fb1 admin admin] The token cache is being invalidate because project ce67b7a6f3034531936a300a6a3cdc90 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87057) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 16:59:10.655022 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.notifications [None req-fb4d343d-bc8f-4cf3-8d42-05222fdb7fb1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87057) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:59:10.655683 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.notifications [None req-fb4d343d-bc8f-4cf3-8d42-05222fdb7fb1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87057) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 16:59:10.657226 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 1199/2407] 10.222.0.20 () {58 vars in 1233 bytes} [Mon Aug 1 16:59:10 2022] DELETE /identity/v3/projects/ce67b7a6f3034531936a300a6a3cdc90 => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 16:59:10.662714 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9595689-9438-45cd-babf-7b1190bec645 None None] REQUEST_METHOD: `POST` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:59:10.662999 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9595689-9438-45cd-babf-7b1190bec645 None None] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:59:10.663301 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9595689-9438-45cd-babf-7b1190bec645 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:59:10.683553 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.auth.core [None req-e9595689-9438-45cd-babf-7b1190bec645 None None] MFA Rules not processed for user `1007f8e379b74e169b98fb7c3c0ba5b4`. Rule list: `[]` (Enabled: `True`). {{(pid=87058) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:59:10.688278 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.fernet_utils [None req-e9595689-9438-45cd-babf-7b1190bec645 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:59:10.744885 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 1209/2408] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 1 16:59:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:59:10.753291 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a926d4d1-17b3-4802-8984-ddd0af198f56 None None] REQUEST_METHOD: `POST` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:59:10.753583 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a926d4d1-17b3-4802-8984-ddd0af198f56 None None] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:59:10.753946 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a926d4d1-17b3-4802-8984-ddd0af198f56 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:59:10.774503 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.auth.core [None req-a926d4d1-17b3-4802-8984-ddd0af198f56 None None] MFA Rules not processed for user `1007f8e379b74e169b98fb7c3c0ba5b4`. Rule list: `[]` (Enabled: `True`). {{(pid=87057) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:59:10.775716 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.fernet_utils [None req-a926d4d1-17b3-4802-8984-ddd0af198f56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87057) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:59:10.783650 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 1200/2409] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 1 16:59:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:59:10.787955 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9595689-9438-45cd-babf-7b1190bec645 None None] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:59:10.797707 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9595689-9438-45cd-babf-7b1190bec645 None None] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:59:10.802298 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f857bc00-7632-4045-b3dd-c52dace6f1b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:59:10.802902 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f857bc00-7632-4045-b3dd-c52dace6f1b5 admin admin] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:59:10.803181 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f857bc00-7632-4045-b3dd-c52dace6f1b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:59:10.803453 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f857bc00-7632-4045-b3dd-c52dace6f1b5 admin admin] PATH_INFO: `/v3/domains` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:59:10.804147 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f857bc00-7632-4045-b3dd-c52dace6f1b5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87058) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 16:59:10.804448 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f857bc00-7632-4045-b3dd-c52dace6f1b5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:59:10.806060 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f857bc00-7632-4045-b3dd-c52dace6f1b5 admin admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:59:10.817784 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 1210/2410] 10.222.0.20 () {58 vars in 1153 bytes} [Mon Aug 1 16:59:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 16:59:10.822093 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a926d4d1-17b3-4802-8984-ddd0af198f56 None None] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:59:10.831956 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a926d4d1-17b3-4802-8984-ddd0af198f56 None None] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:59:10.836594 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1271674-4593-4af6-a8a1-3bb47ef70c61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:59:10.837198 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1271674-4593-4af6-a8a1-3bb47ef70c61 admin admin] REQUEST_METHOD: `POST` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:59:10.837479 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1271674-4593-4af6-a8a1-3bb47ef70c61 admin admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:59:10.837753 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1271674-4593-4af6-a8a1-3bb47ef70c61 admin admin] PATH_INFO: `/v3/projects` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:59:10.838473 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1271674-4593-4af6-a8a1-3bb47ef70c61 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:59:10.840989 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1271674-4593-4af6-a8a1-3bb47ef70c61 admin admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:59:10.902550 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 1201/2411] 10.222.0.20 () {60 vars in 1153 bytes} [Mon Aug 1 16:59:10 2022] POST /identity/v3/projects => generated 373 bytes in 81 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 16:59:10.912211 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f857bc00-7632-4045-b3dd-c52dace6f1b5 admin admin] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:59:10.923595 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f857bc00-7632-4045-b3dd-c52dace6f1b5 admin admin] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:59:10.981385 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf191c6-16bc-4986-9dce-34ac766e0659 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:59:10.982004 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf191c6-16bc-4986-9dce-34ac766e0659 admin admin] REQUEST_METHOD: `POST` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:59:10.982312 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf191c6-16bc-4986-9dce-34ac766e0659 admin admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:59:10.982608 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf191c6-16bc-4986-9dce-34ac766e0659 admin admin] PATH_INFO: `/v3/users` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:59:10.983293 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cf191c6-16bc-4986-9dce-34ac766e0659 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:59:10.985198 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cf191c6-16bc-4986-9dce-34ac766e0659 admin admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:59:11.134966 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 1211/2412] 10.222.0.20 () {60 vars in 1144 bytes} [Mon Aug 1 16:59:10 2022] POST /identity/v3/users => generated 287 bytes in 228 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 16:59:11.139376 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1271674-4593-4af6-a8a1-3bb47ef70c61 admin admin] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:59:11.150015 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1271674-4593-4af6-a8a1-3bb47ef70c61 admin admin] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:59:11.155143 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f76782d3-f90a-48f5-995f-e71da2655155 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:59:11.155757 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76782d3-f90a-48f5-995f-e71da2655155 admin admin] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:59:11.161431 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76782d3-f90a-48f5-995f-e71da2655155 admin admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:59:11.161431 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76782d3-f90a-48f5-995f-e71da2655155 admin admin] PATH_INFO: `/v3/roles` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:59:11.161431 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f76782d3-f90a-48f5-995f-e71da2655155 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:59:11.161431 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f76782d3-f90a-48f5-995f-e71da2655155 admin admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:59:11.168960 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 1202/2413] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Aug 1 16:59:11 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 16:59:11.173056 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf191c6-16bc-4986-9dce-34ac766e0659 admin admin] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:59:11.183781 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf191c6-16bc-4986-9dce-34ac766e0659 admin admin] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:59:11.188384 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd57f1fb-d4ce-4463-8a5a-66bac1b88f3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:59:11.188986 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd57f1fb-d4ce-4463-8a5a-66bac1b88f3f admin admin] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:59:11.189285 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd57f1fb-d4ce-4463-8a5a-66bac1b88f3f admin admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:59:11.189537 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd57f1fb-d4ce-4463-8a5a-66bac1b88f3f admin admin] PATH_INFO: `/v3/roles` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:59:11.190258 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd57f1fb-d4ce-4463-8a5a-66bac1b88f3f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:59:11.191840 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd57f1fb-d4ce-4463-8a5a-66bac1b88f3f admin admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:59:11.202471 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 1212/2414] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Aug 1 16:59:11 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 16:59:11.206618 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f76782d3-f90a-48f5-995f-e71da2655155 admin admin] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:59:11.216668 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f76782d3-f90a-48f5-995f-e71da2655155 admin admin] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:59:11.221196 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03bc95aa-2129-4ced-bf8e-f00484884635 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:59:11.221848 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03bc95aa-2129-4ced-bf8e-f00484884635 admin admin] REQUEST_METHOD: `PUT` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:59:11.222176 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03bc95aa-2129-4ced-bf8e-f00484884635 admin admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:59:11.222452 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03bc95aa-2129-4ced-bf8e-f00484884635 admin admin] PATH_INFO: `/v3/projects/26251afa3c5e43839015776368defa1d/users/8c4a2a503fa14203bcf5b98bfbcccbf8/roles/0999d3da63a545f18ff3252afdc90518` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:59:11.237224 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03bc95aa-2129-4ced-bf8e-f00484884635 admin admin] RBAC: Authorizing `identity:create_grant(project_id=26251afa3c5e43839015776368defa1d, user_id=8c4a2a503fa14203bcf5b98bfbcccbf8, role_id=0999d3da63a545f18ff3252afdc90518)` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:59:11.239913 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03bc95aa-2129-4ced-bf8e-f00484884635 admin admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:59:11.299877 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 1203/2415] 10.222.0.20 () {60 vars in 1483 bytes} [Mon Aug 1 16:59:11 2022] PUT /identity/v3/projects/26251afa3c5e43839015776368defa1d/users/8c4a2a503fa14203bcf5b98bfbcccbf8/roles/0999d3da63a545f18ff3252afdc90518 => generated 0 bytes in 94 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 16:59:11.305018 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21212ec-f49b-4b6c-b432-d289be2663dd None None] REQUEST_METHOD: `POST` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:59:11.305310 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21212ec-f49b-4b6c-b432-d289be2663dd None None] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:59:11.305600 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21212ec-f49b-4b6c-b432-d289be2663dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:59:11.325288 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.auth.core [None req-b21212ec-f49b-4b6c-b432-d289be2663dd None None] MFA Rules not processed for user `1007f8e379b74e169b98fb7c3c0ba5b4`. Rule list: `[]` (Enabled: `True`). {{(pid=87058) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:59:11.326540 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.fernet_utils [None req-b21212ec-f49b-4b6c-b432-d289be2663dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:59:11.334689 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG dbcounter [-] [87058] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:59:11.387741 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 1213/2416] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 1 16:59:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:59:11.397096 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03bc95aa-2129-4ced-bf8e-f00484884635 admin admin] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:59:11.398767 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.fernet_utils [None req-03bc95aa-2129-4ced-bf8e-f00484884635 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87057) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:59:11.454976 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03bc95aa-2129-4ced-bf8e-f00484884635 admin admin] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:59:11.458755 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92adc256-cb64-4ad1-a6c6-c3b291b092a8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d9bd8d649154c979be3744bfd016f95', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b992d6a65df4bf9b27c13192d57c5f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:59:11.459385 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92adc256-cb64-4ad1-a6c6-c3b291b092a8 service neutron] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:59:11.459663 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92adc256-cb64-4ad1-a6c6-c3b291b092a8 service neutron] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:59:11.459937 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92adc256-cb64-4ad1-a6c6-c3b291b092a8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:59:11.470059 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92adc256-cb64-4ad1-a6c6-c3b291b092a8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:59:11.471535 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92adc256-cb64-4ad1-a6c6-c3b291b092a8 service neutron] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:59:11.481153 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 1204/2417] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 1 16:59:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 16:59:16.695814 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d465fd0-7d6f-43d4-b426-4f7afdfe957b None None] REQUEST_METHOD: `POST` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:59:16.696353 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d465fd0-7d6f-43d4-b426-4f7afdfe957b None None] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:59:16.696353 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d465fd0-7d6f-43d4-b426-4f7afdfe957b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:59:16.726616 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.auth.core [None req-9d465fd0-7d6f-43d4-b426-4f7afdfe957b None None] MFA Rules not processed for user `8c4a2a503fa14203bcf5b98bfbcccbf8`. Rule list: `[]` (Enabled: `True`). {{(pid=87058) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:59:16.727848 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.fernet_utils [None req-9d465fd0-7d6f-43d4-b426-4f7afdfe957b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:59:16.796383 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 1214/2418] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 1 16:59:16 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:59:16.800771 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92adc256-cb64-4ad1-a6c6-c3b291b092a8 service neutron] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:59:16.811423 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92adc256-cb64-4ad1-a6c6-c3b291b092a8 service neutron] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:59:16.816098 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b537989a-ed66-41f1-998c-d2e6021bbf90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:59:16.816719 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b537989a-ed66-41f1-998c-d2e6021bbf90 admin admin] REQUEST_METHOD: `POST` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:59:16.816999 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b537989a-ed66-41f1-998c-d2e6021bbf90 admin admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:59:16.817274 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b537989a-ed66-41f1-998c-d2e6021bbf90 admin admin] PATH_INFO: `/v3/projects` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:59:16.817954 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b537989a-ed66-41f1-998c-d2e6021bbf90 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:59:16.819925 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b537989a-ed66-41f1-998c-d2e6021bbf90 admin admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:59:16.896377 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 1205/2419] 10.222.0.20 () {60 vars in 1153 bytes} [Mon Aug 1 16:59:16 2022] POST /identity/v3/projects => generated 373 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 16:59:16.901346 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d465fd0-7d6f-43d4-b426-4f7afdfe957b None None] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:59:16.914241 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d465fd0-7d6f-43d4-b426-4f7afdfe957b None None] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:59:16.998297 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-319a3c37-5861-423d-8306-a878e4111da4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:59:16.998928 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319a3c37-5861-423d-8306-a878e4111da4 admin admin] REQUEST_METHOD: `POST` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:59:16.999211 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319a3c37-5861-423d-8306-a878e4111da4 admin admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:59:16.999489 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319a3c37-5861-423d-8306-a878e4111da4 admin admin] PATH_INFO: `/v3/users` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:59:17.000189 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-319a3c37-5861-423d-8306-a878e4111da4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:59:17.002370 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-319a3c37-5861-423d-8306-a878e4111da4 admin admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:59:17.083986 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 1215/2420] 10.222.0.20 () {60 vars in 1144 bytes} [Mon Aug 1 16:59:16 2022] POST /identity/v3/users => generated 293 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 16:59:17.088071 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b537989a-ed66-41f1-998c-d2e6021bbf90 admin admin] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:59:17.100251 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b537989a-ed66-41f1-998c-d2e6021bbf90 admin admin] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:59:17.105541 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e720213-d7c1-4614-a5c6-3a65be328d9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:59:17.106537 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e720213-d7c1-4614-a5c6-3a65be328d9d admin admin] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:59:17.106820 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e720213-d7c1-4614-a5c6-3a65be328d9d admin admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:59:17.107159 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e720213-d7c1-4614-a5c6-3a65be328d9d admin admin] PATH_INFO: `/v3/roles` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:59:17.107802 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e720213-d7c1-4614-a5c6-3a65be328d9d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:59:17.109435 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e720213-d7c1-4614-a5c6-3a65be328d9d admin admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:59:17.124024 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 1206/2421] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Aug 1 16:59:17 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 16:59:17.128356 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-319a3c37-5861-423d-8306-a878e4111da4 admin admin] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:59:17.148026 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-319a3c37-5861-423d-8306-a878e4111da4 admin admin] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:59:17.154616 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5cdcf1c-6da4-40a7-8dd0-a3a7a8d3b585 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:59:17.154616 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5cdcf1c-6da4-40a7-8dd0-a3a7a8d3b585 admin admin] REQUEST_METHOD: `PUT` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:59:17.154616 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5cdcf1c-6da4-40a7-8dd0-a3a7a8d3b585 admin admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:59:17.155150 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5cdcf1c-6da4-40a7-8dd0-a3a7a8d3b585 admin admin] PATH_INFO: `/v3/projects/71157b72fafd4c8dbb3b1681d56992c0/users/27ba54a036294a119db116883672fea9/roles/440bb943dd5b46fea038b903b6eaaf81` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:59:17.174685 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5cdcf1c-6da4-40a7-8dd0-a3a7a8d3b585 admin admin] RBAC: Authorizing `identity:create_grant(project_id=71157b72fafd4c8dbb3b1681d56992c0, user_id=27ba54a036294a119db116883672fea9, role_id=440bb943dd5b46fea038b903b6eaaf81)` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:59:17.177768 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5cdcf1c-6da4-40a7-8dd0-a3a7a8d3b585 admin admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:59:17.267807 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 1216/2422] 10.222.0.20 () {60 vars in 1483 bytes} [Mon Aug 1 16:59:17 2022] PUT /identity/v3/projects/71157b72fafd4c8dbb3b1681d56992c0/users/27ba54a036294a119db116883672fea9/roles/440bb943dd5b46fea038b903b6eaaf81 => generated 0 bytes in 140 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 16:59:17.272322 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e720213-d7c1-4614-a5c6-3a65be328d9d admin admin] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:59:17.283586 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e720213-d7c1-4614-a5c6-3a65be328d9d admin admin] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:59:17.329906 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG dbcounter [-] [87057] Writing DB stats keystone:DELETE=4,keystone:SELECT=92,keystone:INSERT=4 {{(pid=87057) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:59:17.347368 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ac3e3e-8420-4628-938d-eb0ef4622b94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:59:17.348044 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ac3e3e-8420-4628-938d-eb0ef4622b94 admin admin] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:59:17.348315 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ac3e3e-8420-4628-938d-eb0ef4622b94 admin admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:59:17.348592 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ac3e3e-8420-4628-938d-eb0ef4622b94 admin admin] PATH_INFO: `/v3/roles` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:59:17.349302 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79ac3e3e-8420-4628-938d-eb0ef4622b94 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:59:17.350981 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79ac3e3e-8420-4628-938d-eb0ef4622b94 admin admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:59:17.365672 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 1207/2423] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Aug 1 16:59:17 2022] GET /identity/v3/roles => generated 1420 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 16:59:17.367045 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5cdcf1c-6da4-40a7-8dd0-a3a7a8d3b585 admin admin] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:59:17.378759 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5cdcf1c-6da4-40a7-8dd0-a3a7a8d3b585 admin admin] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:59:17.383639 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7388ecb-47f0-4bb0-8b1d-98a6b9c86529 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:59:17.384353 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7388ecb-47f0-4bb0-8b1d-98a6b9c86529 admin admin] REQUEST_METHOD: `PUT` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:59:17.384593 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7388ecb-47f0-4bb0-8b1d-98a6b9c86529 admin admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:59:17.384866 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7388ecb-47f0-4bb0-8b1d-98a6b9c86529 admin admin] PATH_INFO: `/v3/projects/71157b72fafd4c8dbb3b1681d56992c0/users/27ba54a036294a119db116883672fea9/roles/440bb943dd5b46fea038b903b6eaaf81` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:59:17.387234 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7388ecb-47f0-4bb0-8b1d-98a6b9c86529 admin admin] RBAC: Authorizing `identity:create_grant(project_id=71157b72fafd4c8dbb3b1681d56992c0, user_id=27ba54a036294a119db116883672fea9, role_id=440bb943dd5b46fea038b903b6eaaf81)` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:59:17.389884 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7388ecb-47f0-4bb0-8b1d-98a6b9c86529 admin admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:59:17.393280 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG dbcounter [-] [87058] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=87058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:59:17.401509 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 1217/2424] 10.222.0.20 () {60 vars in 1483 bytes} [Mon Aug 1 16:59:17 2022] PUT /identity/v3/projects/71157b72fafd4c8dbb3b1681d56992c0/users/27ba54a036294a119db116883672fea9/roles/440bb943dd5b46fea038b903b6eaaf81 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 16:59:22.741601 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d73e131-445f-43f6-907d-bb064f6b4432 None None] REQUEST_METHOD: `POST` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:59:22.742103 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d73e131-445f-43f6-907d-bb064f6b4432 None None] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:59:22.742228 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d73e131-445f-43f6-907d-bb064f6b4432 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:59:22.782823 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.auth.core [None req-3d73e131-445f-43f6-907d-bb064f6b4432 None None] MFA Rules not processed for user `27ba54a036294a119db116883672fea9`. Rule list: `[]` (Enabled: `True`). {{(pid=87057) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:59:22.784406 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.fernet_utils [None req-3d73e131-445f-43f6-907d-bb064f6b4432 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87057) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:59:22.861943 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 1208/2425] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 1 16:59:22 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:59:22.878913 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7388ecb-47f0-4bb0-8b1d-98a6b9c86529 admin admin] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:59:22.880713 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.fernet_utils [None req-b7388ecb-47f0-4bb0-8b1d-98a6b9c86529 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:59:22.966499 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7388ecb-47f0-4bb0-8b1d-98a6b9c86529 admin admin] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:59:22.972179 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d8ec949-8bad-4c74-ae60-8c33494375ae service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6533106d03eb4ca185a8839336d2aff9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b992d6a65df4bf9b27c13192d57c5f0', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:59:22.972862 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8ec949-8bad-4c74-ae60-8c33494375ae service nova] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:59:22.973136 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8ec949-8bad-4c74-ae60-8c33494375ae service nova] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:59:22.973405 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8ec949-8bad-4c74-ae60-8c33494375ae service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:59:23.036507 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d8ec949-8bad-4c74-ae60-8c33494375ae service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:59:23.038149 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d8ec949-8bad-4c74-ae60-8c33494375ae service nova] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:59:23.047837 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 1218/2426] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 1 16:59:22 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 169 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 16:59:25.515363 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-badfe2c2-13c5-4ce9-8124-3b23046d4114 None None] REQUEST_METHOD: `POST` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:59:25.515886 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-badfe2c2-13c5-4ce9-8124-3b23046d4114 None None] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:59:25.516095 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-badfe2c2-13c5-4ce9-8124-3b23046d4114 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:59:25.536503 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.auth.core [None req-badfe2c2-13c5-4ce9-8124-3b23046d4114 None None] MFA Rules not processed for user `8c4a2a503fa14203bcf5b98bfbcccbf8`. Rule list: `[]` (Enabled: `True`). {{(pid=87057) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 16:59:25.541313 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.fernet_utils [None req-badfe2c2-13c5-4ce9-8124-3b23046d4114 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87057) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:59:25.545293 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 1209/2427] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 1 16:59:25 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 16:59:33.043430 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG dbcounter [-] [87058] Writing DB stats keystone:SELECT=58 {{(pid=87058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:59:35.531543 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG dbcounter [-] [87057] Writing DB stats keystone:SELECT=48 {{(pid=87057) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 16:59:57.465359 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d8ec949-8bad-4c74-ae60-8c33494375ae service nova] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 16:59:57.520719 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d8ec949-8bad-4c74-ae60-8c33494375ae service nova] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 16:59:57.524273 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86edb744-90f0-4a6c-8f27-446d4e318d35 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d9bd8d649154c979be3744bfd016f95', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b992d6a65df4bf9b27c13192d57c5f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 16:59:57.524879 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86edb744-90f0-4a6c-8f27-446d4e318d35 service neutron] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 16:59:57.525162 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86edb744-90f0-4a6c-8f27-446d4e318d35 service neutron] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 16:59:57.525432 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86edb744-90f0-4a6c-8f27-446d4e318d35 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 16:59:57.527175 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.fernet_utils [None req-86edb744-90f0-4a6c-8f27-446d4e318d35 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 16:59:57.540925 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86edb744-90f0-4a6c-8f27-446d4e318d35 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 16:59:57.542378 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86edb744-90f0-4a6c-8f27-446d4e318d35 service neutron] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 16:59:57.553287 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 1219/2428] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 1 16:59:57 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 17:00:01.505885 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-badfe2c2-13c5-4ce9-8124-3b23046d4114 None None] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 17:00:01.507298 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.fernet_utils [None req-badfe2c2-13c5-4ce9-8124-3b23046d4114 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87057) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 17:00:01.520823 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-badfe2c2-13c5-4ce9-8124-3b23046d4114 None None] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 17:00:01.524734 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09080fc4-1e1e-491a-8105-d9625d25ff95 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d9bd8d649154c979be3744bfd016f95', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b992d6a65df4bf9b27c13192d57c5f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 17:00:01.525427 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09080fc4-1e1e-491a-8105-d9625d25ff95 service neutron] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 17:00:01.525781 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09080fc4-1e1e-491a-8105-d9625d25ff95 service neutron] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 17:00:01.526085 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09080fc4-1e1e-491a-8105-d9625d25ff95 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 17:00:01.531006 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.fernet_utils [None req-09080fc4-1e1e-491a-8105-d9625d25ff95 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87057) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 17:00:01.547221 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09080fc4-1e1e-491a-8105-d9625d25ff95 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 17:00:01.548670 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09080fc4-1e1e-491a-8105-d9625d25ff95 service neutron] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 17:00:01.560877 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 1210/2429] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 1 17:00:01 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 63 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 01 17:00:06.689538 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee92bed-8426-4bae-bb0f-38b085fe6183 None None] REQUEST_METHOD: `POST` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 17:00:06.690146 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee92bed-8426-4bae-bb0f-38b085fe6183 None None] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 17:00:06.690146 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee92bed-8426-4bae-bb0f-38b085fe6183 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 17:00:06.709804 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.auth.core [None req-2ee92bed-8426-4bae-bb0f-38b085fe6183 None None] MFA Rules not processed for user `d720864cb0854d549c05e425bffafa86`. Rule list: `[]` (Enabled: `True`). {{(pid=87058) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 17:00:06.711062 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.fernet_utils [None req-2ee92bed-8426-4bae-bb0f-38b085fe6183 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 17:00:06.775953 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 1220/2430] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 1 17:00:06 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 17:00:07.310202 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09080fc4-1e1e-491a-8105-d9625d25ff95 service neutron] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 17:00:07.322586 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09080fc4-1e1e-491a-8105-d9625d25ff95 service neutron] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 17:00:07.329147 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b76665-6167-4d25-b8f0-8e6411c7a19a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d9bd8d649154c979be3744bfd016f95', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b992d6a65df4bf9b27c13192d57c5f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 17:00:07.332601 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b76665-6167-4d25-b8f0-8e6411c7a19a service neutron] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 17:00:07.332601 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b76665-6167-4d25-b8f0-8e6411c7a19a service neutron] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 17:00:07.332601 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b76665-6167-4d25-b8f0-8e6411c7a19a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 17:00:07.332998 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.fernet_utils [None req-d0b76665-6167-4d25-b8f0-8e6411c7a19a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87057) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 17:00:07.343345 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0b76665-6167-4d25-b8f0-8e6411c7a19a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 17:00:07.346979 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0b76665-6167-4d25-b8f0-8e6411c7a19a service neutron] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 17:00:07.356648 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 1211/2431] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 1 17:00:07 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 17:00:16.770647 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG dbcounter [-] [87058] Writing DB stats keystone:SELECT=51 {{(pid=87058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 17:00:17.352112 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG dbcounter [-] [87057] Writing DB stats keystone:SELECT=12 {{(pid=87057) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 17:00:21.667844 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee92bed-8426-4bae-bb0f-38b085fe6183 None None] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 17:00:21.679114 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee92bed-8426-4bae-bb0f-38b085fe6183 None None] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 17:00:21.683453 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da116085-58f1-4d43-bb23-e8ccdd87f561 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d9bd8d649154c979be3744bfd016f95', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b992d6a65df4bf9b27c13192d57c5f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 17:00:21.684156 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da116085-58f1-4d43-bb23-e8ccdd87f561 service neutron] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 17:00:21.684461 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da116085-58f1-4d43-bb23-e8ccdd87f561 service neutron] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 17:00:21.684733 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da116085-58f1-4d43-bb23-e8ccdd87f561 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 17:00:21.686574 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.fernet_utils [None req-da116085-58f1-4d43-bb23-e8ccdd87f561 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 17:00:21.698581 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da116085-58f1-4d43-bb23-e8ccdd87f561 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 17:00:21.699994 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da116085-58f1-4d43-bb23-e8ccdd87f561 service neutron] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 17:00:21.712660 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 1221/2432] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 1 17:00:21 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 17:00:27.951602 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b76665-6167-4d25-b8f0-8e6411c7a19a service neutron] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 17:00:27.960683 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b76665-6167-4d25-b8f0-8e6411c7a19a service neutron] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 17:00:27.964387 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-595b63c8-330d-41ea-8045-d58a71b1652d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d9bd8d649154c979be3744bfd016f95', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b992d6a65df4bf9b27c13192d57c5f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 17:00:27.964992 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595b63c8-330d-41ea-8045-d58a71b1652d service neutron] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 17:00:27.965272 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595b63c8-330d-41ea-8045-d58a71b1652d service neutron] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 17:00:27.965547 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595b63c8-330d-41ea-8045-d58a71b1652d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 17:00:27.974480 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-595b63c8-330d-41ea-8045-d58a71b1652d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 17:00:27.975914 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-595b63c8-330d-41ea-8045-d58a71b1652d service neutron] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 17:00:27.985242 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 1212/2433] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 1 17:00:27 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 34 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 17:00:29.998532 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da116085-58f1-4d43-bb23-e8ccdd87f561 service neutron] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 17:00:30.000758 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.fernet_utils [None req-da116085-58f1-4d43-bb23-e8ccdd87f561 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 17:00:30.018263 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da116085-58f1-4d43-bb23-e8ccdd87f561 service neutron] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 17:00:30.029365 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8deac54-9b9e-44a5-bcba-3038d9029d3d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6533106d03eb4ca185a8839336d2aff9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b992d6a65df4bf9b27c13192d57c5f0', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 17:00:30.029365 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8deac54-9b9e-44a5-bcba-3038d9029d3d service nova] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 17:00:30.029365 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8deac54-9b9e-44a5-bcba-3038d9029d3d service nova] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 17:00:30.029365 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8deac54-9b9e-44a5-bcba-3038d9029d3d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 17:00:30.029807 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.fernet_utils [None req-d8deac54-9b9e-44a5-bcba-3038d9029d3d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 17:00:30.044748 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8deac54-9b9e-44a5-bcba-3038d9029d3d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 17:00:30.046382 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8deac54-9b9e-44a5-bcba-3038d9029d3d service nova] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 17:00:30.058947 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 1222/2434] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 1 17:00:29 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 61 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 17:00:31.307384 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76848ce8-0046-4dc0-b0f2-6a39c1ad2210 None None] REQUEST_METHOD: `POST` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 17:00:31.307862 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76848ce8-0046-4dc0-b0f2-6a39c1ad2210 None None] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 17:00:31.307955 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76848ce8-0046-4dc0-b0f2-6a39c1ad2210 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 17:00:31.327816 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.auth.core [None req-76848ce8-0046-4dc0-b0f2-6a39c1ad2210 None None] MFA Rules not processed for user `b6a93d1c6c82489099adc8b413280fc0`. Rule list: `[]` (Enabled: `True`). {{(pid=87057) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 17:00:31.329197 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.fernet_utils [None req-76848ce8-0046-4dc0-b0f2-6a39c1ad2210 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87057) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 17:00:31.384535 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 1213/2435] 10.222.0.20 () {58 vars in 1009 bytes} [Mon Aug 1 17:00:31 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 17:00:31.504822 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8deac54-9b9e-44a5-bcba-3038d9029d3d service nova] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 17:00:31.506276 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.fernet_utils [None req-d8deac54-9b9e-44a5-bcba-3038d9029d3d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 17:00:31.574737 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8deac54-9b9e-44a5-bcba-3038d9029d3d service nova] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 17:00:31.579783 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8645ef2b-5476-4c1e-94f5-5564a15c89a2 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ab4add1faa04faa811ef7a9341cc35c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b992d6a65df4bf9b27c13192d57c5f0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 17:00:31.580401 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8645ef2b-5476-4c1e-94f5-5564a15c89a2 service swift] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 17:00:31.580684 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8645ef2b-5476-4c1e-94f5-5564a15c89a2 service swift] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 17:00:31.580963 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8645ef2b-5476-4c1e-94f5-5564a15c89a2 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 17:00:31.589036 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8645ef2b-5476-4c1e-94f5-5564a15c89a2 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 17:00:31.590674 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8645ef2b-5476-4c1e-94f5-5564a15c89a2 service swift] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 17:00:31.600103 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 1223/2436] 10.222.0.20 () {60 vars in 1384 bytes} [Mon Aug 1 17:00:31 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 01 17:00:41.377789 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG dbcounter [-] [87057] Writing DB stats keystone:SELECT=31 {{(pid=87057) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 17:00:41.596767 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG dbcounter [-] [87058] Writing DB stats keystone:SELECT=44 {{(pid=87058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 17:00:53.763078 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76848ce8-0046-4dc0-b0f2-6a39c1ad2210 None None] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 17:00:53.780660 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76848ce8-0046-4dc0-b0f2-6a39c1ad2210 None None] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 17:00:53.787294 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c27873-eaaa-4141-9096-941cc66032e5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6533106d03eb4ca185a8839336d2aff9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b992d6a65df4bf9b27c13192d57c5f0', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 17:00:53.788258 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c27873-eaaa-4141-9096-941cc66032e5 service nova] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 17:00:53.788578 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c27873-eaaa-4141-9096-941cc66032e5 service nova] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 17:00:53.788852 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c27873-eaaa-4141-9096-941cc66032e5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 17:00:53.790691 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.fernet_utils [None req-87c27873-eaaa-4141-9096-941cc66032e5 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87057) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 17:00:53.806240 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87c27873-eaaa-4141-9096-941cc66032e5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 17:00:53.809926 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87c27873-eaaa-4141-9096-941cc66032e5 service nova] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 17:00:53.824254 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 1214/2437] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 1 17:00:53 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 62 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 17:00:55.383405 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a354af4-e238-47b3-8188-85ae5fff1205 None None] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 17:00:55.383974 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a354af4-e238-47b3-8188-85ae5fff1205 None None] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 17:00:55.383974 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a354af4-e238-47b3-8188-85ae5fff1205 None None] PATH_INFO: `/` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 17:00:55.385489 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 1224/2438] 10.222.0.20 () {54 vars in 921 bytes} [Mon Aug 1 17:00:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 17:00:55.392269 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d9df06-2506-4bdf-a026-310ab700cdb9 None None] REQUEST_METHOD: `POST` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 17:00:55.399619 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d9df06-2506-4bdf-a026-310ab700cdb9 None None] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 17:00:55.399619 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d9df06-2506-4bdf-a026-310ab700cdb9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 17:00:55.412501 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.auth.core [None req-d7d9df06-2506-4bdf-a026-310ab700cdb9 None None] MFA Rules not processed for user `6533106d03eb4ca185a8839336d2aff9`. Rule list: `[]` (Enabled: `True`). {{(pid=87057) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 17:00:55.413723 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.fernet_utils [None req-d7d9df06-2506-4bdf-a026-310ab700cdb9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87057) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 17:00:55.424404 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 1215/2439] 10.222.0.20 () {58 vars in 1018 bytes} [Mon Aug 1 17:00:55 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 17:00:55.435360 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a354af4-e238-47b3-8188-85ae5fff1205 None None] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 17:00:55.437066 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.fernet_utils [None req-5a354af4-e238-47b3-8188-85ae5fff1205 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 17:00:55.495058 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a354af4-e238-47b3-8188-85ae5fff1205 None None] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 17:00:55.498606 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f1a98f4-9236-4a74-8c82-e52a1afe2da2 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd80694e439e2401d9491712666409016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b992d6a65df4bf9b27c13192d57c5f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 17:00:55.499253 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f1a98f4-9236-4a74-8c82-e52a1afe2da2 service glance] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 17:00:55.499533 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f1a98f4-9236-4a74-8c82-e52a1afe2da2 service glance] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 17:00:55.500424 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f1a98f4-9236-4a74-8c82-e52a1afe2da2 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 17:00:55.510734 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f1a98f4-9236-4a74-8c82-e52a1afe2da2 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 17:00:55.512198 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f1a98f4-9236-4a74-8c82-e52a1afe2da2 service glance] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 17:00:55.522920 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 1225/2440] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 1 17:00:55 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 17:00:55.752917 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15aec7e9-9364-4892-a911-fd292e7fabaf None None] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 17:00:55.753128 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15aec7e9-9364-4892-a911-fd292e7fabaf None None] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 17:00:55.753380 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15aec7e9-9364-4892-a911-fd292e7fabaf None None] PATH_INFO: `/` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 17:00:55.754846 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 1216/2441] 10.222.0.20 () {54 vars in 921 bytes} [Mon Aug 1 17:00:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 17:00:55.759486 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac90ffc-8c33-47e9-887e-4e3b1737af3e None None] REQUEST_METHOD: `POST` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 17:00:55.759777 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac90ffc-8c33-47e9-887e-4e3b1737af3e None None] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 17:00:55.760055 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac90ffc-8c33-47e9-887e-4e3b1737af3e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 17:00:55.779300 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.auth.core [None req-4ac90ffc-8c33-47e9-887e-4e3b1737af3e None None] MFA Rules not processed for user `3d9bd8d649154c979be3744bfd016f95`. Rule list: `[]` (Enabled: `True`). {{(pid=87058) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 17:00:55.780497 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.fernet_utils [None req-4ac90ffc-8c33-47e9-887e-4e3b1737af3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 17:00:55.787595 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 1226/2442] 10.222.0.20 () {58 vars in 1018 bytes} [Mon Aug 1 17:00:55 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 17:00:55.800021 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15aec7e9-9364-4892-a911-fd292e7fabaf None None] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 17:00:55.810850 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15aec7e9-9364-4892-a911-fd292e7fabaf None None] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 17:00:55.814435 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc5ee57-04b1-4bfc-b264-2a72a5f57c27 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d9bd8d649154c979be3744bfd016f95', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b992d6a65df4bf9b27c13192d57c5f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 17:00:55.815036 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc5ee57-04b1-4bfc-b264-2a72a5f57c27 service neutron] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 17:00:55.815316 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc5ee57-04b1-4bfc-b264-2a72a5f57c27 service neutron] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 17:00:55.815591 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc5ee57-04b1-4bfc-b264-2a72a5f57c27 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 17:00:55.822513 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffc5ee57-04b1-4bfc-b264-2a72a5f57c27 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 17:00:55.823949 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffc5ee57-04b1-4bfc-b264-2a72a5f57c27 service neutron] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 17:00:55.831784 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 1217/2443] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 1 17:00:55 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 32 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 17:00:55.913729 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a486d4-c860-4049-883a-ab9fd605347e None None] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 17:00:55.914226 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a486d4-c860-4049-883a-ab9fd605347e None None] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 17:00:55.914355 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a486d4-c860-4049-883a-ab9fd605347e None None] PATH_INFO: `/` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 17:00:55.916768 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 1227/2444] 10.222.0.20 () {54 vars in 921 bytes} [Mon Aug 1 17:00:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 17:00:55.920569 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d9ff19-2f77-499c-b8c3-edec8d73ccd2 None None] REQUEST_METHOD: `POST` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 17:00:55.920938 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d9ff19-2f77-499c-b8c3-edec8d73ccd2 None None] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 17:00:55.921282 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d9ff19-2f77-499c-b8c3-edec8d73ccd2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 17:00:55.941233 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.auth.core [None req-22d9ff19-2f77-499c-b8c3-edec8d73ccd2 None None] MFA Rules not processed for user `6533106d03eb4ca185a8839336d2aff9`. Rule list: `[]` (Enabled: `True`). {{(pid=87057) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 17:00:55.942483 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.fernet_utils [None req-22d9ff19-2f77-499c-b8c3-edec8d73ccd2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87057) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 17:00:55.952229 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 1218/2445] 10.222.0.20 () {58 vars in 1018 bytes} [Mon Aug 1 17:00:55 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 17:00:55.963369 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90a486d4-c860-4049-883a-ab9fd605347e None None] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 17:00:55.975497 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90a486d4-c860-4049-883a-ab9fd605347e None None] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 17:00:55.979123 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-029e8735-d9e9-4dd0-a1f6-76ecc3b671f5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d9bd8d649154c979be3744bfd016f95', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b992d6a65df4bf9b27c13192d57c5f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 17:00:55.979729 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029e8735-d9e9-4dd0-a1f6-76ecc3b671f5 service neutron] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 17:00:55.980004 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029e8735-d9e9-4dd0-a1f6-76ecc3b671f5 service neutron] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 17:00:55.980273 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029e8735-d9e9-4dd0-a1f6-76ecc3b671f5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 17:00:55.992279 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-029e8735-d9e9-4dd0-a1f6-76ecc3b671f5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 17:00:55.993746 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-029e8735-d9e9-4dd0-a1f6-76ecc3b671f5 service neutron] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 17:00:56.005199 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 1228/2446] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 1 17:00:55 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 17:00:56.072716 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22d9ff19-2f77-499c-b8c3-edec8d73ccd2 None None] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 17:00:56.074138 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.fernet_utils [None req-22d9ff19-2f77-499c-b8c3-edec8d73ccd2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87057) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 17:00:56.142391 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22d9ff19-2f77-499c-b8c3-edec8d73ccd2 None None] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 17:00:56.147489 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50798510-1efb-4e84-a55b-c98f4d796d13 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8362293261614623bcdabc4120a97a9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b992d6a65df4bf9b27c13192d57c5f0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 17:00:56.148133 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50798510-1efb-4e84-a55b-c98f4d796d13 service placement] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 17:00:56.148415 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50798510-1efb-4e84-a55b-c98f4d796d13 service placement] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 17:00:56.148692 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50798510-1efb-4e84-a55b-c98f4d796d13 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 17:00:56.150476 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.fernet_utils [None req-50798510-1efb-4e84-a55b-c98f4d796d13 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87057) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 17:00:56.162364 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50798510-1efb-4e84-a55b-c98f4d796d13 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 17:00:56.163943 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50798510-1efb-4e84-a55b-c98f4d796d13 service placement] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 17:00:56.174318 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 1219/2447] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 1 17:00:56 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 17:00:59.068895 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-029e8735-d9e9-4dd0-a1f6-76ecc3b671f5 service neutron] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 17:00:59.080967 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-029e8735-d9e9-4dd0-a1f6-76ecc3b671f5 service neutron] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 17:00:59.090852 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e621d830-b00d-4a34-9e2f-c257680addfc service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6533106d03eb4ca185a8839336d2aff9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b992d6a65df4bf9b27c13192d57c5f0', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 17:00:59.091464 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e621d830-b00d-4a34-9e2f-c257680addfc service nova] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 17:00:59.091748 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e621d830-b00d-4a34-9e2f-c257680addfc service nova] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 17:00:59.092057 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e621d830-b00d-4a34-9e2f-c257680addfc service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 17:00:59.094904 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.fernet_utils [None req-e621d830-b00d-4a34-9e2f-c257680addfc service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 17:00:59.108051 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e621d830-b00d-4a34-9e2f-c257680addfc service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 17:00:59.113665 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e621d830-b00d-4a34-9e2f-c257680addfc service nova] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 17:00:59.120721 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 1229/2448] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 1 17:00:59 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 17:01:01.342078 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c7fb7f-09b9-4cea-b9e7-140a420e65e0 None None] REQUEST_METHOD: `POST` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 17:01:01.342078 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c7fb7f-09b9-4cea-b9e7-140a420e65e0 None None] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 17:01:01.342078 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c7fb7f-09b9-4cea-b9e7-140a420e65e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 17:01:01.360083 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.auth.core [None req-b9c7fb7f-09b9-4cea-b9e7-140a420e65e0 None None] MFA Rules not processed for user `b6a93d1c6c82489099adc8b413280fc0`. Rule list: `[]` (Enabled: `True`). {{(pid=87057) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 17:01:01.361442 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.fernet_utils [None req-b9c7fb7f-09b9-4cea-b9e7-140a420e65e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87057) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 17:01:01.369207 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 1220/2449] 10.222.0.20 () {58 vars in 1009 bytes} [Mon Aug 1 17:01:01 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 17:01:01.384019 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e621d830-b00d-4a34-9e2f-c257680addfc service nova] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 17:01:01.395030 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e621d830-b00d-4a34-9e2f-c257680addfc service nova] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 17:01:01.400050 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ffd0ccd-1179-4d47-bb67-f8779608a1b5 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ab4add1faa04faa811ef7a9341cc35c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b992d6a65df4bf9b27c13192d57c5f0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 17:01:01.400659 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ffd0ccd-1179-4d47-bb67-f8779608a1b5 service swift] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 17:01:01.400943 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ffd0ccd-1179-4d47-bb67-f8779608a1b5 service swift] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 17:01:01.401217 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ffd0ccd-1179-4d47-bb67-f8779608a1b5 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 17:01:01.409805 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ffd0ccd-1179-4d47-bb67-f8779608a1b5 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 17:01:01.411420 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ffd0ccd-1179-4d47-bb67-f8779608a1b5 service swift] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 17:01:01.419975 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 1230/2450] 10.222.0.20 () {60 vars in 1384 bytes} [Mon Aug 1 17:01:01 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 01 17:01:11.355063 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG dbcounter [-] [87057] Writing DB stats keystone:SELECT=53 {{(pid=87057) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 17:01:11.416376 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG dbcounter [-] [87058] Writing DB stats keystone:SELECT=47 {{(pid=87058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 17:01:37.748278 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c7fb7f-09b9-4cea-b9e7-140a420e65e0 None None] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 17:01:37.749701 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.fernet_utils [None req-b9c7fb7f-09b9-4cea-b9e7-140a420e65e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87057) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 17:01:37.764997 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c7fb7f-09b9-4cea-b9e7-140a420e65e0 None None] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 17:01:37.770697 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93433487-5d40-475d-881a-0b7f29fb837a service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8362293261614623bcdabc4120a97a9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b992d6a65df4bf9b27c13192d57c5f0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 17:01:37.771319 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93433487-5d40-475d-881a-0b7f29fb837a service placement] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 17:01:37.771598 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93433487-5d40-475d-881a-0b7f29fb837a service placement] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 17:01:37.771877 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93433487-5d40-475d-881a-0b7f29fb837a service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 17:01:37.774994 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.fernet_utils [None req-93433487-5d40-475d-881a-0b7f29fb837a service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87057) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 17:01:37.787541 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93433487-5d40-475d-881a-0b7f29fb837a service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 17:01:37.789287 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93433487-5d40-475d-881a-0b7f29fb837a service placement] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 17:01:37.799789 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 1221/2451] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 1 17:01:37 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 17:01:37.883595 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ffd0ccd-1179-4d47-bb67-f8779608a1b5 service swift] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 17:01:37.885064 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.fernet_utils [None req-7ffd0ccd-1179-4d47-bb67-f8779608a1b5 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 17:01:37.953756 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ffd0ccd-1179-4d47-bb67-f8779608a1b5 service swift] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 17:01:37.958617 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e183566-ee84-4b9b-babf-3fadeda08bbf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 17:01:37.960661 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e183566-ee84-4b9b-babf-3fadeda08bbf admin admin] REQUEST_METHOD: `DELETE` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 17:01:37.960954 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e183566-ee84-4b9b-babf-3fadeda08bbf admin admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 17:01:37.961288 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e183566-ee84-4b9b-babf-3fadeda08bbf admin admin] PATH_INFO: `/v3/users/d720864cb0854d549c05e425bffafa86` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 17:01:37.962502 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e183566-ee84-4b9b-babf-3fadeda08bbf admin admin] RBAC: Authorizing `identity:delete_user(user_id=d720864cb0854d549c05e425bffafa86)` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 17:01:37.964482 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e183566-ee84-4b9b-babf-3fadeda08bbf admin admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 17:01:38.181511 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.notifications [None req-0e183566-ee84-4b9b-babf-3fadeda08bbf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd720864cb0854d549c05e425bffafa86'} {{(pid=87058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 17:01:38.182192 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.notifications [None req-0e183566-ee84-4b9b-babf-3fadeda08bbf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd720864cb0854d549c05e425bffafa86'} {{(pid=87058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 17:01:38.182795 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.notifications [None req-0e183566-ee84-4b9b-babf-3fadeda08bbf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd720864cb0854d549c05e425bffafa86'} {{(pid=87058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 17:01:38.195285 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.notifications [None req-0e183566-ee84-4b9b-babf-3fadeda08bbf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd720864cb0854d549c05e425bffafa86'} {{(pid=87058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 17:01:38.202379 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.notifications [None req-0e183566-ee84-4b9b-babf-3fadeda08bbf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd720864cb0854d549c05e425bffafa86'} {{(pid=87058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 17:01:38.288405 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 1231/2452] 10.222.0.20 () {58 vars in 1224 bytes} [Mon Aug 1 17:01:37 2022] DELETE /identity/v3/users/d720864cb0854d549c05e425bffafa86 => generated 0 bytes in 405 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 17:01:38.570285 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93433487-5d40-475d-881a-0b7f29fb837a service placement] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 17:01:38.585751 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93433487-5d40-475d-881a-0b7f29fb837a service placement] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 17:01:38.645997 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b6ac940-5048-467a-a585-8893a0207100 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 17:01:38.743115 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6ac940-5048-467a-a585-8893a0207100 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 17:01:38.743115 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6ac940-5048-467a-a585-8893a0207100 admin admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 17:01:38.743115 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6ac940-5048-467a-a585-8893a0207100 admin admin] PATH_INFO: `/v3/projects/025776af0f594ddc9bfd2eedb6eb7602` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 17:01:38.743115 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b6ac940-5048-467a-a585-8893a0207100 admin admin] RBAC: Authorizing `identity:delete_project(project_id=025776af0f594ddc9bfd2eedb6eb7602)` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 17:01:38.743115 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b6ac940-5048-467a-a585-8893a0207100 admin admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 17:01:38.744919 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c57953e-7886-4307-a49e-d9a8839837ea None None] REQUEST_METHOD: `POST` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 17:01:38.744919 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c57953e-7886-4307-a49e-d9a8839837ea None None] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 17:01:38.744919 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c57953e-7886-4307-a49e-d9a8839837ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 17:01:38.762126 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.auth.core [None req-1c57953e-7886-4307-a49e-d9a8839837ea None None] MFA Rules not processed for user `b6a93d1c6c82489099adc8b413280fc0`. Rule list: `[]` (Enabled: `True`). {{(pid=87058) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 17:01:38.763327 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.fernet_utils [None req-1c57953e-7886-4307-a49e-d9a8839837ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 17:01:38.853523 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 1232/2453] 10.222.0.20 () {58 vars in 967 bytes} [Mon Aug 1 17:01:38 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 17:01:38.854447 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c57953e-7886-4307-a49e-d9a8839837ea None None] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 17:01:38.855814 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.fernet_utils [None req-1c57953e-7886-4307-a49e-d9a8839837ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 17:01:38.896806 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG dbcounter [-] [87058] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 17:01:38.957590 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c57953e-7886-4307-a49e-d9a8839837ea None None] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 17:01:38.963115 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15542efe-7362-4fe3-86ce-687a59a34fe7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6533106d03eb4ca185a8839336d2aff9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b992d6a65df4bf9b27c13192d57c5f0', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 17:01:38.963734 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15542efe-7362-4fe3-86ce-687a59a34fe7 service nova] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 17:01:38.964013 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15542efe-7362-4fe3-86ce-687a59a34fe7 service nova] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 17:01:38.964287 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15542efe-7362-4fe3-86ce-687a59a34fe7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 17:01:38.966084 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.fernet_utils [None req-15542efe-7362-4fe3-86ce-687a59a34fe7 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 17:01:39.025727 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15542efe-7362-4fe3-86ce-687a59a34fe7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 17:01:39.027762 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15542efe-7362-4fe3-86ce-687a59a34fe7 service nova] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 17:01:39.039412 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 1233/2454] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 1 17:01:38 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 185 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 17:01:39.040441 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15542efe-7362-4fe3-86ce-687a59a34fe7 service nova] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 17:01:39.041872 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.fernet_utils [None req-15542efe-7362-4fe3-86ce-687a59a34fe7 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 17:01:39.123517 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15542efe-7362-4fe3-86ce-687a59a34fe7 service nova] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 17:01:39.129641 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9def6565-4cd4-4cc2-9f49-5042a53b7f9f service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ab4add1faa04faa811ef7a9341cc35c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b992d6a65df4bf9b27c13192d57c5f0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 17:01:39.130369 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9def6565-4cd4-4cc2-9f49-5042a53b7f9f service swift] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 17:01:39.130721 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9def6565-4cd4-4cc2-9f49-5042a53b7f9f service swift] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 17:01:39.131002 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9def6565-4cd4-4cc2-9f49-5042a53b7f9f service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 17:01:39.140285 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9def6565-4cd4-4cc2-9f49-5042a53b7f9f service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 17:01:39.142302 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9def6565-4cd4-4cc2-9f49-5042a53b7f9f service swift] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 17:01:39.151633 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 1234/2455] 10.222.0.20 () {60 vars in 1384 bytes} [Mon Aug 1 17:01:39 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 01 17:01:39.397045 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.notifications [None req-7b6ac940-5048-467a-a585-8893a0207100 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '025776af0f594ddc9bfd2eedb6eb7602', 'request_id': 'req-7b6ac940-5048-467a-a585-8893a0207100'} {{(pid=87057) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 17:01:39.403573 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.notifications [None req-7b6ac940-5048-467a-a585-8893a0207100 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '025776af0f594ddc9bfd2eedb6eb7602', 'request_id': 'req-7b6ac940-5048-467a-a585-8893a0207100'} {{(pid=87057) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 17:01:39.409724 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.notifications [None req-7b6ac940-5048-467a-a585-8893a0207100 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '025776af0f594ddc9bfd2eedb6eb7602', 'request_id': 'req-7b6ac940-5048-467a-a585-8893a0207100'} {{(pid=87057) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 17:01:39.522744 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.notifications [None req-7b6ac940-5048-467a-a585-8893a0207100 admin admin] The token cache is being invalidate because project 025776af0f594ddc9bfd2eedb6eb7602 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87057) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 17:01:39.523073 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.notifications [None req-7b6ac940-5048-467a-a585-8893a0207100 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87057) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 17:01:39.523698 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.notifications [None req-7b6ac940-5048-467a-a585-8893a0207100 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87057) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 17:01:39.525372 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 1222/2456] 10.222.0.20 () {58 vars in 1233 bytes} [Mon Aug 1 17:01:38 2022] DELETE /identity/v3/projects/025776af0f594ddc9bfd2eedb6eb7602 => generated 0 bytes in 956 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 17:01:49.519309 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG dbcounter [-] [87057] Writing DB stats keystone:SELECT=57,keystone:DELETE=7,keystone:INSERT=1 {{(pid=87057) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 17:01:49.597512 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG dbcounter [-] [87058] Writing DB stats keystone:SELECT=77 {{(pid=87058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 17:01:56.062805 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9def6565-4cd4-4cc2-9f49-5042a53b7f9f service swift] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 17:01:56.065068 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.fernet_utils [None req-9def6565-4cd4-4cc2-9f49-5042a53b7f9f service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 17:01:56.080048 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9def6565-4cd4-4cc2-9f49-5042a53b7f9f service swift] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 17:01:56.142007 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b1045a-ae99-4a6c-8a10-b68da03111c4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6533106d03eb4ca185a8839336d2aff9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b992d6a65df4bf9b27c13192d57c5f0', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 17:01:56.142918 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b1045a-ae99-4a6c-8a10-b68da03111c4 service nova] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 17:01:56.143232 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b1045a-ae99-4a6c-8a10-b68da03111c4 service nova] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 17:01:56.143512 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b1045a-ae99-4a6c-8a10-b68da03111c4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 17:01:56.145293 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.fernet_utils [None req-f1b1045a-ae99-4a6c-8a10-b68da03111c4 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 17:01:56.203016 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1b1045a-ae99-4a6c-8a10-b68da03111c4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 17:01:56.204516 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1b1045a-ae99-4a6c-8a10-b68da03111c4 service nova] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 17:01:56.213555 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 1235/2457] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 1 17:01:56 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 151 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 17:02:06.209451 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG dbcounter [-] [87058] Writing DB stats keystone:SELECT=56 {{(pid=87058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 17:02:10.108559 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce0a42b-f0e6-43bd-96e4-202e0b9e70bf None None] REQUEST_METHOD: `POST` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 17:02:10.109093 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce0a42b-f0e6-43bd-96e4-202e0b9e70bf None None] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 17:02:10.109093 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce0a42b-f0e6-43bd-96e4-202e0b9e70bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 17:02:10.128945 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.auth.core [None req-cce0a42b-f0e6-43bd-96e4-202e0b9e70bf None None] MFA Rules not processed for user `b353fdd302394227987a6ae8c2943173`. Rule list: `[]` (Enabled: `True`). {{(pid=87057) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 17:02:10.130215 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.fernet_utils [None req-cce0a42b-f0e6-43bd-96e4-202e0b9e70bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87057) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 17:02:10.183410 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 1223/2458] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 1 17:02:10 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 17:02:20.176936 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG dbcounter [-] [87057] Writing DB stats keystone:SELECT=25 {{(pid=87057) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 17:02:58.385836 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b1045a-ae99-4a6c-8a10-b68da03111c4 service nova] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 17:02:58.387446 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.fernet_utils [None req-f1b1045a-ae99-4a6c-8a10-b68da03111c4 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 17:02:58.480596 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b1045a-ae99-4a6c-8a10-b68da03111c4 service nova] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 17:02:58.486260 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4abba9cd-a702-4e02-a187-606924c7760a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6533106d03eb4ca185a8839336d2aff9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b992d6a65df4bf9b27c13192d57c5f0', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 17:02:58.486879 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4abba9cd-a702-4e02-a187-606924c7760a service nova] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 17:02:58.487160 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4abba9cd-a702-4e02-a187-606924c7760a service nova] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 17:02:58.487435 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4abba9cd-a702-4e02-a187-606924c7760a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 17:02:58.489227 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.fernet_utils [None req-4abba9cd-a702-4e02-a187-606924c7760a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 17:02:58.546715 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4abba9cd-a702-4e02-a187-606924c7760a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 17:02:58.548301 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4abba9cd-a702-4e02-a187-606924c7760a service nova] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 17:02:58.557811 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 1236/2459] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 1 17:02:58 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 173 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 17:03:07.854491 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cce0a42b-f0e6-43bd-96e4-202e0b9e70bf None None] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 17:03:07.888611 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cce0a42b-f0e6-43bd-96e4-202e0b9e70bf None None] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 17:03:07.893894 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7591639a-6b42-4c2d-97f7-d7d718cbeae2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6533106d03eb4ca185a8839336d2aff9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b992d6a65df4bf9b27c13192d57c5f0', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 17:03:07.894688 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7591639a-6b42-4c2d-97f7-d7d718cbeae2 service nova] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 17:03:07.894986 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7591639a-6b42-4c2d-97f7-d7d718cbeae2 service nova] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 17:03:07.895256 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7591639a-6b42-4c2d-97f7-d7d718cbeae2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 17:03:07.897188 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.fernet_utils [None req-7591639a-6b42-4c2d-97f7-d7d718cbeae2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87057) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 17:03:07.960611 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7591639a-6b42-4c2d-97f7-d7d718cbeae2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 17:03:07.962245 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7591639a-6b42-4c2d-97f7-d7d718cbeae2 service nova] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 17:03:07.971855 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 1224/2460] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 1 17:03:07 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 01 17:03:08.554298 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG dbcounter [-] [87058] Writing DB stats keystone:SELECT=62 {{(pid=87058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 17:03:09.626655 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7279fb-7ccc-4dfa-81a0-cf353edd7dba None None] REQUEST_METHOD: `POST` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 17:03:09.627256 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7279fb-7ccc-4dfa-81a0-cf353edd7dba None None] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 17:03:09.627256 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7279fb-7ccc-4dfa-81a0-cf353edd7dba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 17:03:09.647843 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.auth.core [None req-6f7279fb-7ccc-4dfa-81a0-cf353edd7dba None None] MFA Rules not processed for user `48cc0715e7b04f8a8c9f0d8c455bd4c6`. Rule list: `[]` (Enabled: `True`). {{(pid=87058) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 17:03:09.649272 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.fernet_utils [None req-6f7279fb-7ccc-4dfa-81a0-cf353edd7dba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 17:03:09.656529 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 1237/2461] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 1 17:03:09 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 17:03:09.932082 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7591639a-6b42-4c2d-97f7-d7d718cbeae2 service nova] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 17:03:09.933512 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.fernet_utils [None req-7591639a-6b42-4c2d-97f7-d7d718cbeae2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87057) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 17:03:10.031561 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7591639a-6b42-4c2d-97f7-d7d718cbeae2 service nova] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 17:03:10.037285 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c3f56d2-4b9a-4ec8-b8d7-1aceb6694ee0 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8362293261614623bcdabc4120a97a9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b992d6a65df4bf9b27c13192d57c5f0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 17:03:10.037903 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3f56d2-4b9a-4ec8-b8d7-1aceb6694ee0 service placement] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 17:03:10.038215 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3f56d2-4b9a-4ec8-b8d7-1aceb6694ee0 service placement] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 17:03:10.038515 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3f56d2-4b9a-4ec8-b8d7-1aceb6694ee0 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 17:03:10.040342 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.fernet_utils [None req-9c3f56d2-4b9a-4ec8-b8d7-1aceb6694ee0 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87057) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 17:03:10.053224 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c3f56d2-4b9a-4ec8-b8d7-1aceb6694ee0 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 17:03:10.057443 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c3f56d2-4b9a-4ec8-b8d7-1aceb6694ee0 service placement] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 17:03:10.065436 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 1225/2462] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 1 17:03:09 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 134 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 17:03:19.641508 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG dbcounter [-] [87058] Writing DB stats keystone:SELECT=3 {{(pid=87058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 17:03:20.060810 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG dbcounter [-] [87057] Writing DB stats keystone:SELECT=77 {{(pid=87057) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 17:03:22.401084 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02518a3-762f-4a84-a783-99bf93332296 None None] REQUEST_METHOD: `POST` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 17:03:22.401546 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02518a3-762f-4a84-a783-99bf93332296 None None] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 17:03:22.401683 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02518a3-762f-4a84-a783-99bf93332296 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 17:03:22.434062 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.auth.core [None req-e02518a3-762f-4a84-a783-99bf93332296 None None] MFA Rules not processed for user `b6a93d1c6c82489099adc8b413280fc0`. Rule list: `[]` (Enabled: `True`). {{(pid=87058) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 17:03:22.439210 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.fernet_utils [None req-e02518a3-762f-4a84-a783-99bf93332296 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 17:03:22.491597 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 1238/2463] 10.222.0.20 () {58 vars in 967 bytes} [Mon Aug 1 17:03:22 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 17:03:22.651472 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c3f56d2-4b9a-4ec8-b8d7-1aceb6694ee0 service placement] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 17:03:22.652909 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.fernet_utils [None req-9c3f56d2-4b9a-4ec8-b8d7-1aceb6694ee0 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87057) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 17:03:22.756160 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c3f56d2-4b9a-4ec8-b8d7-1aceb6694ee0 service placement] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 17:03:22.762606 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a486959d-2f4c-4e5c-b098-8986130f775a service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ab4add1faa04faa811ef7a9341cc35c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b992d6a65df4bf9b27c13192d57c5f0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 17:03:22.763266 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a486959d-2f4c-4e5c-b098-8986130f775a service swift] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 17:03:22.763548 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a486959d-2f4c-4e5c-b098-8986130f775a service swift] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 17:03:22.763819 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a486959d-2f4c-4e5c-b098-8986130f775a service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 17:03:22.773471 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a486959d-2f4c-4e5c-b098-8986130f775a service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 17:03:22.775244 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a486959d-2f4c-4e5c-b098-8986130f775a service swift] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 17:03:22.787066 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 1226/2464] 10.222.0.20 () {60 vars in 1384 bytes} [Mon Aug 1 17:03:22 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 134 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 01 17:03:23.752492 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e02518a3-762f-4a84-a783-99bf93332296 None None] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 17:03:23.753939 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.fernet_utils [None req-e02518a3-762f-4a84-a783-99bf93332296 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 17:03:23.833060 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e02518a3-762f-4a84-a783-99bf93332296 None None] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 17:03:23.837260 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd9fe3f-2152-4854-84d5-6071d1ca3cb1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d9bd8d649154c979be3744bfd016f95', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b992d6a65df4bf9b27c13192d57c5f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 17:03:23.837876 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd9fe3f-2152-4854-84d5-6071d1ca3cb1 service neutron] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 17:03:23.838186 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd9fe3f-2152-4854-84d5-6071d1ca3cb1 service neutron] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 17:03:23.838459 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd9fe3f-2152-4854-84d5-6071d1ca3cb1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 17:03:23.840195 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.fernet_utils [None req-2bd9fe3f-2152-4854-84d5-6071d1ca3cb1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 17:03:23.936137 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bd9fe3f-2152-4854-84d5-6071d1ca3cb1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 17:03:23.937557 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bd9fe3f-2152-4854-84d5-6071d1ca3cb1 service neutron] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 17:03:23.952497 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG dbcounter [-] [87058] Writing DB stats keystone:SELECT=100 {{(pid=87058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 17:03:23.957900 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 1239/2465] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 1 17:03:23 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 206 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 17:03:27.855439 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a486959d-2f4c-4e5c-b098-8986130f775a service swift] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 17:03:27.869304 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a486959d-2f4c-4e5c-b098-8986130f775a service swift] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 17:03:27.874942 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-494adb76-af89-4efe-8ed2-6c02ff8e66fc service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8362293261614623bcdabc4120a97a9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b992d6a65df4bf9b27c13192d57c5f0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 17:03:27.875561 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-494adb76-af89-4efe-8ed2-6c02ff8e66fc service placement] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 17:03:27.875842 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-494adb76-af89-4efe-8ed2-6c02ff8e66fc service placement] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 17:03:27.876243 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-494adb76-af89-4efe-8ed2-6c02ff8e66fc service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 17:03:27.878683 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.fernet_utils [None req-494adb76-af89-4efe-8ed2-6c02ff8e66fc service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87057) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 17:03:27.891727 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-494adb76-af89-4efe-8ed2-6c02ff8e66fc service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 17:03:27.893320 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-494adb76-af89-4efe-8ed2-6c02ff8e66fc service placement] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 17:03:27.905034 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 1227/2466] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 1 17:03:27 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 17:03:31.873398 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd9fe3f-2152-4854-84d5-6071d1ca3cb1 service neutron] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 17:03:31.874946 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.fernet_utils [None req-2bd9fe3f-2152-4854-84d5-6071d1ca3cb1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 17:03:31.889592 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd9fe3f-2152-4854-84d5-6071d1ca3cb1 service neutron] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 17:03:31.894509 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58504fde-11ee-4a3b-9f39-6e907a2529f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 17:03:31.895099 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58504fde-11ee-4a3b-9f39-6e907a2529f3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 17:03:31.895446 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58504fde-11ee-4a3b-9f39-6e907a2529f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 17:03:31.896416 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58504fde-11ee-4a3b-9f39-6e907a2529f3 admin admin] PATH_INFO: `/v3/users/b2d83a4ae8684f3183756b2dcfc4d514` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 17:03:31.898141 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58504fde-11ee-4a3b-9f39-6e907a2529f3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b2d83a4ae8684f3183756b2dcfc4d514)` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 17:03:31.900071 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58504fde-11ee-4a3b-9f39-6e907a2529f3 admin admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 17:03:32.014812 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.notifications [None req-58504fde-11ee-4a3b-9f39-6e907a2529f3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b2d83a4ae8684f3183756b2dcfc4d514'} {{(pid=87058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 17:03:32.015464 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.notifications [None req-58504fde-11ee-4a3b-9f39-6e907a2529f3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b2d83a4ae8684f3183756b2dcfc4d514'} {{(pid=87058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 17:03:32.016111 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.notifications [None req-58504fde-11ee-4a3b-9f39-6e907a2529f3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b2d83a4ae8684f3183756b2dcfc4d514'} {{(pid=87058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 17:03:32.028068 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.notifications [None req-58504fde-11ee-4a3b-9f39-6e907a2529f3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b2d83a4ae8684f3183756b2dcfc4d514'} {{(pid=87058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 17:03:32.035115 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.notifications [None req-58504fde-11ee-4a3b-9f39-6e907a2529f3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b2d83a4ae8684f3183756b2dcfc4d514'} {{(pid=87058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 17:03:32.108810 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 1240/2467] 10.222.0.20 () {58 vars in 1224 bytes} [Mon Aug 1 17:03:31 2022] DELETE /identity/v3/users/b2d83a4ae8684f3183756b2dcfc4d514 => generated 0 bytes in 236 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 17:03:32.361790 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-494adb76-af89-4efe-8ed2-6c02ff8e66fc service placement] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 17:03:32.374623 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-494adb76-af89-4efe-8ed2-6c02ff8e66fc service placement] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 17:03:32.434804 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f76152-b990-485e-b422-ecb18a07ad3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 17:03:32.435442 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f76152-b990-485e-b422-ecb18a07ad3e admin admin] REQUEST_METHOD: `DELETE` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 17:03:32.435717 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f76152-b990-485e-b422-ecb18a07ad3e admin admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 17:03:32.435989 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f76152-b990-485e-b422-ecb18a07ad3e admin admin] PATH_INFO: `/v3/projects/d9fb92139d3643b2bb7ed4bfed2eae67` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 17:03:32.437131 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20f76152-b990-485e-b422-ecb18a07ad3e admin admin] RBAC: Authorizing `identity:delete_project(project_id=d9fb92139d3643b2bb7ed4bfed2eae67)` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 17:03:32.439076 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20f76152-b990-485e-b422-ecb18a07ad3e admin admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 17:03:32.533395 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.notifications [None req-20f76152-b990-485e-b422-ecb18a07ad3e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd9fb92139d3643b2bb7ed4bfed2eae67', 'request_id': 'req-20f76152-b990-485e-b422-ecb18a07ad3e'} {{(pid=87057) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 17:03:32.540652 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.notifications [None req-20f76152-b990-485e-b422-ecb18a07ad3e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd9fb92139d3643b2bb7ed4bfed2eae67', 'request_id': 'req-20f76152-b990-485e-b422-ecb18a07ad3e'} {{(pid=87057) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 17:03:32.546399 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.notifications [None req-20f76152-b990-485e-b422-ecb18a07ad3e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd9fb92139d3643b2bb7ed4bfed2eae67', 'request_id': 'req-20f76152-b990-485e-b422-ecb18a07ad3e'} {{(pid=87057) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 17:03:32.646539 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.notifications [None req-20f76152-b990-485e-b422-ecb18a07ad3e admin admin] The token cache is being invalidate because project d9fb92139d3643b2bb7ed4bfed2eae67 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87057) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 17:03:32.646839 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.notifications [None req-20f76152-b990-485e-b422-ecb18a07ad3e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87057) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 17:03:32.651115 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.notifications [None req-20f76152-b990-485e-b422-ecb18a07ad3e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87057) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 17:03:32.651115 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 1228/2468] 10.222.0.20 () {58 vars in 1233 bytes} [Mon Aug 1 17:03:32 2022] DELETE /identity/v3/projects/d9fb92139d3643b2bb7ed4bfed2eae67 => generated 0 bytes in 288 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 17:03:32.653424 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58504fde-11ee-4a3b-9f39-6e907a2529f3 admin admin] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 17:03:32.654858 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.fernet_utils [None req-58504fde-11ee-4a3b-9f39-6e907a2529f3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 17:03:32.668799 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58504fde-11ee-4a3b-9f39-6e907a2529f3 admin admin] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 17:03:32.725765 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f1189f6-7508-4369-8e70-9a17ddfa23b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 17:03:32.726475 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f1189f6-7508-4369-8e70-9a17ddfa23b5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 17:03:32.726753 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f1189f6-7508-4369-8e70-9a17ddfa23b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 17:03:32.727027 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f1189f6-7508-4369-8e70-9a17ddfa23b5 admin admin] PATH_INFO: `/v3/users/5ffd0f4aeef741b1b1a84d61005cd79f` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 17:03:32.728202 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f1189f6-7508-4369-8e70-9a17ddfa23b5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5ffd0f4aeef741b1b1a84d61005cd79f)` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 17:03:32.730165 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f1189f6-7508-4369-8e70-9a17ddfa23b5 admin admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 17:03:32.901141 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.notifications [None req-3f1189f6-7508-4369-8e70-9a17ddfa23b5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5ffd0f4aeef741b1b1a84d61005cd79f'} {{(pid=87058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 17:03:32.901859 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.notifications [None req-3f1189f6-7508-4369-8e70-9a17ddfa23b5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5ffd0f4aeef741b1b1a84d61005cd79f'} {{(pid=87058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 17:03:32.903102 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.notifications [None req-3f1189f6-7508-4369-8e70-9a17ddfa23b5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5ffd0f4aeef741b1b1a84d61005cd79f'} {{(pid=87058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 17:03:32.916370 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.notifications [None req-3f1189f6-7508-4369-8e70-9a17ddfa23b5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5ffd0f4aeef741b1b1a84d61005cd79f'} {{(pid=87058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 17:03:32.922855 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG dbcounter [-] [87058] Writing DB stats keystone:SELECT=80,keystone:DELETE=19,keystone:INSERT=1 {{(pid=87058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 17:03:32.925179 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.notifications [None req-3f1189f6-7508-4369-8e70-9a17ddfa23b5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5ffd0f4aeef741b1b1a84d61005cd79f'} {{(pid=87058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 17:03:32.989135 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 1241/2469] 10.222.0.20 () {58 vars in 1224 bytes} [Mon Aug 1 17:03:32 2022] DELETE /identity/v3/users/5ffd0f4aeef741b1b1a84d61005cd79f => generated 0 bytes in 336 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 17:03:33.225821 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f76152-b990-485e-b422-ecb18a07ad3e admin admin] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 17:03:33.227371 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.fernet_utils [None req-20f76152-b990-485e-b422-ecb18a07ad3e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87057) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 17:03:33.244712 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f76152-b990-485e-b422-ecb18a07ad3e admin admin] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 17:03:33.248595 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG dbcounter [-] [87057] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=87057) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 17:03:33.311631 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a4e7492-a377-4991-9eba-13ec8424695f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 17:03:33.312662 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4e7492-a377-4991-9eba-13ec8424695f admin admin] REQUEST_METHOD: `DELETE` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 17:03:33.313177 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4e7492-a377-4991-9eba-13ec8424695f admin admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 17:03:33.313754 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4e7492-a377-4991-9eba-13ec8424695f admin admin] PATH_INFO: `/v3/projects/58fb1e54fe2545ce9120812fea4b9a46` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 17:03:33.315231 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a4e7492-a377-4991-9eba-13ec8424695f admin admin] RBAC: Authorizing `identity:delete_project(project_id=58fb1e54fe2545ce9120812fea4b9a46)` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 17:03:33.317444 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a4e7492-a377-4991-9eba-13ec8424695f admin admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 17:03:33.419185 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.notifications [None req-2a4e7492-a377-4991-9eba-13ec8424695f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '58fb1e54fe2545ce9120812fea4b9a46', 'request_id': 'req-2a4e7492-a377-4991-9eba-13ec8424695f'} {{(pid=87057) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 17:03:33.425723 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.notifications [None req-2a4e7492-a377-4991-9eba-13ec8424695f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '58fb1e54fe2545ce9120812fea4b9a46', 'request_id': 'req-2a4e7492-a377-4991-9eba-13ec8424695f'} {{(pid=87057) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 17:03:33.432543 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.notifications [None req-2a4e7492-a377-4991-9eba-13ec8424695f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '58fb1e54fe2545ce9120812fea4b9a46', 'request_id': 'req-2a4e7492-a377-4991-9eba-13ec8424695f'} {{(pid=87057) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 17:03:33.492070 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.notifications [None req-2a4e7492-a377-4991-9eba-13ec8424695f admin admin] The token cache is being invalidate because project 58fb1e54fe2545ce9120812fea4b9a46 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87057) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 17:03:33.492309 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.notifications [None req-2a4e7492-a377-4991-9eba-13ec8424695f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87057) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 17:03:33.492880 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.notifications [None req-2a4e7492-a377-4991-9eba-13ec8424695f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87057) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 17:03:33.494446 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 1229/2470] 10.222.0.20 () {58 vars in 1233 bytes} [Mon Aug 1 17:03:33 2022] DELETE /identity/v3/projects/58fb1e54fe2545ce9120812fea4b9a46 => generated 0 bytes in 269 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 17:03:43.040734 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG dbcounter [-] [87058] Writing DB stats keystone:SELECT=3,keystone:INSERT=1,keystone:DELETE=1 {{(pid=87058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 17:03:43.487713 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG dbcounter [-] [87057] Writing DB stats keystone:SELECT=48,keystone:DELETE=7,keystone:INSERT=1 {{(pid=87057) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 17:03:47.107637 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da88def6-ea63-4c2b-9dd9-1e0a0183dbb6 None None] REQUEST_METHOD: `POST` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 17:03:47.107637 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da88def6-ea63-4c2b-9dd9-1e0a0183dbb6 None None] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 17:03:47.107637 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da88def6-ea63-4c2b-9dd9-1e0a0183dbb6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 17:03:47.125772 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.auth.core [None req-da88def6-ea63-4c2b-9dd9-1e0a0183dbb6 None None] MFA Rules not processed for user `c2079b0fbf7b4a8c88d2522da9f2c73b`. Rule list: `[]` (Enabled: `True`). {{(pid=87058) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 17:03:47.126944 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.fernet_utils [None req-da88def6-ea63-4c2b-9dd9-1e0a0183dbb6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 17:03:47.181566 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 1242/2471] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 1 17:03:47 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 17:03:57.174898 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG dbcounter [-] [87058] Writing DB stats keystone:SELECT=25 {{(pid=87058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 17:03:57.261058 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a4e7492-a377-4991-9eba-13ec8424695f admin admin] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 17:03:57.262674 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.fernet_utils [None req-2a4e7492-a377-4991-9eba-13ec8424695f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87057) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 17:03:57.353195 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a4e7492-a377-4991-9eba-13ec8424695f admin admin] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 17:03:57.357605 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc2f2f4-9ee4-40c5-949b-f2e557174db9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d9bd8d649154c979be3744bfd016f95', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b992d6a65df4bf9b27c13192d57c5f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 17:03:57.358316 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc2f2f4-9ee4-40c5-949b-f2e557174db9 service neutron] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 17:03:57.358683 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc2f2f4-9ee4-40c5-949b-f2e557174db9 service neutron] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 17:03:57.359074 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc2f2f4-9ee4-40c5-949b-f2e557174db9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 17:03:57.361599 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.fernet_utils [None req-5bc2f2f4-9ee4-40c5-949b-f2e557174db9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87057) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 17:03:57.429792 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bc2f2f4-9ee4-40c5-949b-f2e557174db9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 17:03:57.431292 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bc2f2f4-9ee4-40c5-949b-f2e557174db9 service neutron] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 17:03:57.441195 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 1230/2472] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 1 17:03:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 181 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 17:04:01.286579 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da88def6-ea63-4c2b-9dd9-1e0a0183dbb6 None None] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 17:04:01.288933 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.fernet_utils [None req-da88def6-ea63-4c2b-9dd9-1e0a0183dbb6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 17:04:01.304564 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da88def6-ea63-4c2b-9dd9-1e0a0183dbb6 None None] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 17:04:01.309283 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa09e5c9-7212-460b-b535-3b7286f5b214 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 17:04:01.309937 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa09e5c9-7212-460b-b535-3b7286f5b214 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 17:04:01.310247 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa09e5c9-7212-460b-b535-3b7286f5b214 admin admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 17:04:01.310726 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa09e5c9-7212-460b-b535-3b7286f5b214 admin admin] PATH_INFO: `/v3/users/b353fdd302394227987a6ae8c2943173` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 17:04:01.311911 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa09e5c9-7212-460b-b535-3b7286f5b214 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b353fdd302394227987a6ae8c2943173)` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 17:04:01.313900 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa09e5c9-7212-460b-b535-3b7286f5b214 admin admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 17:04:01.487831 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.notifications [None req-aa09e5c9-7212-460b-b535-3b7286f5b214 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b353fdd302394227987a6ae8c2943173'} {{(pid=87058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 17:04:01.488488 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.notifications [None req-aa09e5c9-7212-460b-b535-3b7286f5b214 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b353fdd302394227987a6ae8c2943173'} {{(pid=87058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 17:04:01.489102 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.notifications [None req-aa09e5c9-7212-460b-b535-3b7286f5b214 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b353fdd302394227987a6ae8c2943173'} {{(pid=87058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 17:04:01.501939 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.notifications [None req-aa09e5c9-7212-460b-b535-3b7286f5b214 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b353fdd302394227987a6ae8c2943173'} {{(pid=87058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 17:04:01.509525 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.notifications [None req-aa09e5c9-7212-460b-b535-3b7286f5b214 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b353fdd302394227987a6ae8c2943173'} {{(pid=87058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 17:04:01.588949 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 1243/2473] 10.222.0.20 () {58 vars in 1224 bytes} [Mon Aug 1 17:04:01 2022] DELETE /identity/v3/users/b353fdd302394227987a6ae8c2943173 => generated 0 bytes in 303 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 17:04:01.824839 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc2f2f4-9ee4-40c5-949b-f2e557174db9 service neutron] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 17:04:01.835293 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc2f2f4-9ee4-40c5-949b-f2e557174db9 service neutron] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 17:04:01.893825 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ef3745e-dc62-4a9c-af51-769b1a56900e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 17:04:01.894495 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef3745e-dc62-4a9c-af51-769b1a56900e admin admin] REQUEST_METHOD: `DELETE` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 17:04:01.894777 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef3745e-dc62-4a9c-af51-769b1a56900e admin admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 17:04:01.895052 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef3745e-dc62-4a9c-af51-769b1a56900e admin admin] PATH_INFO: `/v3/projects/64929e340dae4c0f90162702aab5bc21` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 17:04:01.896209 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ef3745e-dc62-4a9c-af51-769b1a56900e admin admin] RBAC: Authorizing `identity:delete_project(project_id=64929e340dae4c0f90162702aab5bc21)` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 17:04:01.898217 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ef3745e-dc62-4a9c-af51-769b1a56900e admin admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 17:04:02.001926 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.notifications [None req-7ef3745e-dc62-4a9c-af51-769b1a56900e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '64929e340dae4c0f90162702aab5bc21', 'request_id': 'req-7ef3745e-dc62-4a9c-af51-769b1a56900e'} {{(pid=87057) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 17:04:02.003924 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG dbcounter [-] [87057] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=87057) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 17:04:02.009053 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.notifications [None req-7ef3745e-dc62-4a9c-af51-769b1a56900e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '64929e340dae4c0f90162702aab5bc21', 'request_id': 'req-7ef3745e-dc62-4a9c-af51-769b1a56900e'} {{(pid=87057) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 17:04:02.014702 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.notifications [None req-7ef3745e-dc62-4a9c-af51-769b1a56900e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '64929e340dae4c0f90162702aab5bc21', 'request_id': 'req-7ef3745e-dc62-4a9c-af51-769b1a56900e'} {{(pid=87057) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 17:04:02.130161 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.notifications [None req-7ef3745e-dc62-4a9c-af51-769b1a56900e admin admin] The token cache is being invalidate because project 64929e340dae4c0f90162702aab5bc21 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87057) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 17:04:02.130161 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.notifications [None req-7ef3745e-dc62-4a9c-af51-769b1a56900e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87057) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 17:04:02.130161 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.notifications [None req-7ef3745e-dc62-4a9c-af51-769b1a56900e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87057) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 17:04:02.130161 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 1231/2474] 10.222.0.20 () {58 vars in 1233 bytes} [Mon Aug 1 17:04:01 2022] DELETE /identity/v3/projects/64929e340dae4c0f90162702aab5bc21 => generated 0 bytes in 299 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 17:04:02.130580 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f7ded7-be09-43a6-989c-0613c39d9395 None None] REQUEST_METHOD: `POST` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 17:04:02.130580 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f7ded7-be09-43a6-989c-0613c39d9395 None None] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 17:04:02.130580 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f7ded7-be09-43a6-989c-0613c39d9395 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 17:04:02.190484 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.auth.core [None req-97f7ded7-be09-43a6-989c-0613c39d9395 None None] MFA Rules not processed for user `1007f8e379b74e169b98fb7c3c0ba5b4`. Rule list: `[]` (Enabled: `True`). {{(pid=87058) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 17:04:02.191747 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.fernet_utils [None req-97f7ded7-be09-43a6-989c-0613c39d9395 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 17:04:02.253492 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 1244/2475] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 1 17:04:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 17:04:02.261441 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def57df9-c3fb-4519-9bdd-b3f25821cef7 None None] REQUEST_METHOD: `POST` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 17:04:02.261726 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def57df9-c3fb-4519-9bdd-b3f25821cef7 None None] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 17:04:02.262002 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def57df9-c3fb-4519-9bdd-b3f25821cef7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 17:04:02.282231 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.auth.core [None req-def57df9-c3fb-4519-9bdd-b3f25821cef7 None None] MFA Rules not processed for user `1007f8e379b74e169b98fb7c3c0ba5b4`. Rule list: `[]` (Enabled: `True`). {{(pid=87057) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 17:04:02.283525 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.fernet_utils [None req-def57df9-c3fb-4519-9bdd-b3f25821cef7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87057) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 17:04:02.292494 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 1232/2476] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 1 17:04:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 17:04:02.296538 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97f7ded7-be09-43a6-989c-0613c39d9395 None None] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 17:04:02.307303 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97f7ded7-be09-43a6-989c-0613c39d9395 None None] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 17:04:02.312717 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe97365-9817-4cba-a2ba-d5d23cfb000c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 17:04:02.313369 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe97365-9817-4cba-a2ba-d5d23cfb000c admin admin] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 17:04:02.313648 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe97365-9817-4cba-a2ba-d5d23cfb000c admin admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 17:04:02.313924 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe97365-9817-4cba-a2ba-d5d23cfb000c admin admin] PATH_INFO: `/v3/domains` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 17:04:02.314666 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbe97365-9817-4cba-a2ba-d5d23cfb000c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87058) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 17:04:02.314963 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbe97365-9817-4cba-a2ba-d5d23cfb000c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 17:04:02.316634 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbe97365-9817-4cba-a2ba-d5d23cfb000c admin admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 17:04:02.328891 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 1245/2477] 10.222.0.20 () {58 vars in 1153 bytes} [Mon Aug 1 17:04:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 17:04:02.333233 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def57df9-c3fb-4519-9bdd-b3f25821cef7 None None] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 17:04:02.342736 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def57df9-c3fb-4519-9bdd-b3f25821cef7 None None] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 17:04:02.347498 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f4ee52-e407-40f7-98e4-2dad2155acff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 17:04:02.348309 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f4ee52-e407-40f7-98e4-2dad2155acff admin admin] REQUEST_METHOD: `POST` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 17:04:02.348592 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f4ee52-e407-40f7-98e4-2dad2155acff admin admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 17:04:02.348867 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f4ee52-e407-40f7-98e4-2dad2155acff admin admin] PATH_INFO: `/v3/projects` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 17:04:02.349552 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7f4ee52-e407-40f7-98e4-2dad2155acff admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 17:04:02.352232 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7f4ee52-e407-40f7-98e4-2dad2155acff admin admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 17:04:02.487781 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 1233/2478] 10.222.0.20 () {60 vars in 1153 bytes} [Mon Aug 1 17:04:02 2022] POST /identity/v3/projects => generated 383 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 17:04:02.492754 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe97365-9817-4cba-a2ba-d5d23cfb000c admin admin] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 17:04:02.503030 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe97365-9817-4cba-a2ba-d5d23cfb000c admin admin] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 17:04:02.529315 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG dbcounter [-] [87058] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 17:04:02.565511 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad56be3-ae60-4c81-896c-2ca2d8253db4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 17:04:02.566332 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad56be3-ae60-4c81-896c-2ca2d8253db4 admin admin] REQUEST_METHOD: `POST` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 17:04:02.566627 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad56be3-ae60-4c81-896c-2ca2d8253db4 admin admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 17:04:02.567278 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad56be3-ae60-4c81-896c-2ca2d8253db4 admin admin] PATH_INFO: `/v3/users` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 17:04:02.567968 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ad56be3-ae60-4c81-896c-2ca2d8253db4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 17:04:02.570240 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ad56be3-ae60-4c81-896c-2ca2d8253db4 admin admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 17:04:02.635006 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 1246/2479] 10.222.0.20 () {60 vars in 1144 bytes} [Mon Aug 1 17:04:02 2022] POST /identity/v3/users => generated 292 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 17:04:02.639109 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f4ee52-e407-40f7-98e4-2dad2155acff admin admin] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 17:04:02.651041 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f4ee52-e407-40f7-98e4-2dad2155acff admin admin] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 17:04:02.656150 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e9607d6-e3f9-4021-86c8-0d95f6468ec9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 17:04:02.656792 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9607d6-e3f9-4021-86c8-0d95f6468ec9 admin admin] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 17:04:02.657065 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9607d6-e3f9-4021-86c8-0d95f6468ec9 admin admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 17:04:02.657362 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9607d6-e3f9-4021-86c8-0d95f6468ec9 admin admin] PATH_INFO: `/v3/roles` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 17:04:02.658074 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e9607d6-e3f9-4021-86c8-0d95f6468ec9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 17:04:02.659672 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e9607d6-e3f9-4021-86c8-0d95f6468ec9 admin admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 17:04:02.673561 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 1234/2480] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Aug 1 17:04:02 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 17:04:02.674792 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad56be3-ae60-4c81-896c-2ca2d8253db4 admin admin] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 17:04:02.684918 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad56be3-ae60-4c81-896c-2ca2d8253db4 admin admin] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 17:04:02.689608 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c402e86-dba1-4124-a382-6acd33799aa1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 17:04:02.690338 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c402e86-dba1-4124-a382-6acd33799aa1 admin admin] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 17:04:02.690613 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c402e86-dba1-4124-a382-6acd33799aa1 admin admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 17:04:02.690880 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c402e86-dba1-4124-a382-6acd33799aa1 admin admin] PATH_INFO: `/v3/roles` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 17:04:02.691597 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c402e86-dba1-4124-a382-6acd33799aa1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 17:04:02.693174 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c402e86-dba1-4124-a382-6acd33799aa1 admin admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 17:04:02.703560 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 1247/2481] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Aug 1 17:04:02 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 17:04:02.707557 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e9607d6-e3f9-4021-86c8-0d95f6468ec9 admin admin] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 17:04:02.718104 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e9607d6-e3f9-4021-86c8-0d95f6468ec9 admin admin] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 17:04:02.722752 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be3de2c-1750-484a-91f8-cef264e90acf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 17:04:02.723415 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be3de2c-1750-484a-91f8-cef264e90acf admin admin] REQUEST_METHOD: `PUT` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 17:04:02.723706 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be3de2c-1750-484a-91f8-cef264e90acf admin admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 17:04:02.723970 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be3de2c-1750-484a-91f8-cef264e90acf admin admin] PATH_INFO: `/v3/projects/29aa58f150be4722ad251dad9a7f1a7e/users/f3484bf0457a407d954de53bf9bc8a91/roles/0999d3da63a545f18ff3252afdc90518` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 17:04:02.739343 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6be3de2c-1750-484a-91f8-cef264e90acf admin admin] RBAC: Authorizing `identity:create_grant(project_id=29aa58f150be4722ad251dad9a7f1a7e, user_id=f3484bf0457a407d954de53bf9bc8a91, role_id=0999d3da63a545f18ff3252afdc90518)` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 17:04:02.742464 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6be3de2c-1750-484a-91f8-cef264e90acf admin admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 17:04:02.841305 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 1235/2482] 10.222.0.20 () {60 vars in 1483 bytes} [Mon Aug 1 17:04:02 2022] PUT /identity/v3/projects/29aa58f150be4722ad251dad9a7f1a7e/users/f3484bf0457a407d954de53bf9bc8a91/roles/0999d3da63a545f18ff3252afdc90518 => generated 0 bytes in 134 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 17:04:02.846351 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d83a73-d40c-49de-b867-704a32073cf7 None None] REQUEST_METHOD: `POST` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 17:04:02.846643 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d83a73-d40c-49de-b867-704a32073cf7 None None] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 17:04:02.846914 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d83a73-d40c-49de-b867-704a32073cf7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 17:04:02.867623 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.auth.core [None req-33d83a73-d40c-49de-b867-704a32073cf7 None None] MFA Rules not processed for user `1007f8e379b74e169b98fb7c3c0ba5b4`. Rule list: `[]` (Enabled: `True`). {{(pid=87058) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 17:04:02.868847 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.fernet_utils [None req-33d83a73-d40c-49de-b867-704a32073cf7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 17:04:02.934644 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 1248/2483] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 1 17:04:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 17:04:02.945067 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be3de2c-1750-484a-91f8-cef264e90acf admin admin] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 17:04:02.947753 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.fernet_utils [None req-6be3de2c-1750-484a-91f8-cef264e90acf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87057) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 17:04:03.006432 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be3de2c-1750-484a-91f8-cef264e90acf admin admin] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 17:04:03.010049 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38bb05c6-9012-45b9-988e-1feb477cb7a9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d9bd8d649154c979be3744bfd016f95', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b992d6a65df4bf9b27c13192d57c5f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 17:04:03.010681 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38bb05c6-9012-45b9-988e-1feb477cb7a9 service neutron] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 17:04:03.010985 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38bb05c6-9012-45b9-988e-1feb477cb7a9 service neutron] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 17:04:03.011260 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38bb05c6-9012-45b9-988e-1feb477cb7a9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 17:04:03.020643 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38bb05c6-9012-45b9-988e-1feb477cb7a9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 17:04:03.022212 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38bb05c6-9012-45b9-988e-1feb477cb7a9 service neutron] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 17:04:03.032388 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 1236/2484] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 1 17:04:02 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 17:04:08.327392 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f282b5df-afd8-46cf-b561-917e483946bf None None] REQUEST_METHOD: `POST` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 17:04:08.327860 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f282b5df-afd8-46cf-b561-917e483946bf None None] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 17:04:08.328133 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f282b5df-afd8-46cf-b561-917e483946bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 17:04:08.360068 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.auth.core [None req-f282b5df-afd8-46cf-b561-917e483946bf None None] MFA Rules not processed for user `f3484bf0457a407d954de53bf9bc8a91`. Rule list: `[]` (Enabled: `True`). {{(pid=87058) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 17:04:08.361643 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.fernet_utils [None req-f282b5df-afd8-46cf-b561-917e483946bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 17:04:08.435026 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 1249/2485] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 1 17:04:08 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 17:04:08.450520 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38bb05c6-9012-45b9-988e-1feb477cb7a9 service neutron] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 17:04:08.452035 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.fernet_utils [None req-38bb05c6-9012-45b9-988e-1feb477cb7a9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87057) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 17:04:08.528302 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38bb05c6-9012-45b9-988e-1feb477cb7a9 service neutron] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 17:04:08.534207 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68094e6b-17cb-4565-af78-132816723c3e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6533106d03eb4ca185a8839336d2aff9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b992d6a65df4bf9b27c13192d57c5f0', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 17:04:08.534838 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68094e6b-17cb-4565-af78-132816723c3e service nova] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 17:04:08.535121 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68094e6b-17cb-4565-af78-132816723c3e service nova] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 17:04:08.535395 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68094e6b-17cb-4565-af78-132816723c3e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 17:04:08.544533 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68094e6b-17cb-4565-af78-132816723c3e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 17:04:08.546168 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68094e6b-17cb-4565-af78-132816723c3e service nova] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 17:04:08.556230 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 1237/2486] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 1 17:04:08 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 17:04:10.773131 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfbe1876-3480-4e96-99fb-262cfcbbbf6d None None] REQUEST_METHOD: `POST` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 17:04:10.773651 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfbe1876-3480-4e96-99fb-262cfcbbbf6d None None] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 17:04:10.773837 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfbe1876-3480-4e96-99fb-262cfcbbbf6d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 17:04:10.792829 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.auth.core [None req-cfbe1876-3480-4e96-99fb-262cfcbbbf6d None None] MFA Rules not processed for user `f3484bf0457a407d954de53bf9bc8a91`. Rule list: `[]` (Enabled: `True`). {{(pid=87058) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 17:04:10.794185 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.fernet_utils [None req-cfbe1876-3480-4e96-99fb-262cfcbbbf6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 17:04:10.801604 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 1250/2487] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 1 17:04:10 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 17:04:11.132535 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68094e6b-17cb-4565-af78-132816723c3e service nova] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 17:04:11.138990 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG dbcounter [-] [87057] Writing DB stats keystone:SELECT=94,keystone:DELETE=3,keystone:INSERT=3 {{(pid=87057) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 17:04:11.142617 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68094e6b-17cb-4565-af78-132816723c3e service nova] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 17:04:11.146399 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a56a22d4-0723-433e-bc42-2cc95078c3b3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d9bd8d649154c979be3744bfd016f95', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b992d6a65df4bf9b27c13192d57c5f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 17:04:11.157188 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56a22d4-0723-433e-bc42-2cc95078c3b3 service neutron] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 17:04:11.157188 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56a22d4-0723-433e-bc42-2cc95078c3b3 service neutron] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 17:04:11.157188 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56a22d4-0723-433e-bc42-2cc95078c3b3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 17:04:11.157188 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.fernet_utils [None req-a56a22d4-0723-433e-bc42-2cc95078c3b3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87057) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 17:04:11.160449 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a56a22d4-0723-433e-bc42-2cc95078c3b3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 17:04:11.161917 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a56a22d4-0723-433e-bc42-2cc95078c3b3 service neutron] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 17:04:11.170876 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 1238/2488] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 1 17:04:11 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 17:04:20.787477 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG dbcounter [-] [87058] Writing DB stats keystone:SELECT=83,keystone:INSERT=3 {{(pid=87058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 17:04:21.262624 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG dbcounter [-] [87057] Writing DB stats keystone:SELECT=4 {{(pid=87057) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 17:04:57.346310 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfbe1876-3480-4e96-99fb-262cfcbbbf6d None None] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 17:04:57.356519 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfbe1876-3480-4e96-99fb-262cfcbbbf6d None None] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 17:04:57.360289 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e1eff9a-21d8-40ca-a258-5b8ce896ed02 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d9bd8d649154c979be3744bfd016f95', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b992d6a65df4bf9b27c13192d57c5f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 17:04:57.360918 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1eff9a-21d8-40ca-a258-5b8ce896ed02 service neutron] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 17:04:57.361197 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1eff9a-21d8-40ca-a258-5b8ce896ed02 service neutron] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 17:04:57.361472 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1eff9a-21d8-40ca-a258-5b8ce896ed02 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 17:04:57.363238 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.fernet_utils [None req-3e1eff9a-21d8-40ca-a258-5b8ce896ed02 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 17:04:57.377103 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e1eff9a-21d8-40ca-a258-5b8ce896ed02 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 17:04:57.378563 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e1eff9a-21d8-40ca-a258-5b8ce896ed02 service neutron] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 17:04:57.391662 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 1251/2489] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 1 17:04:57 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 17:05:00.923879 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a56a22d4-0723-433e-bc42-2cc95078c3b3 service neutron] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 17:05:00.926177 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.fernet_utils [None req-a56a22d4-0723-433e-bc42-2cc95078c3b3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87057) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 17:05:00.937505 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a56a22d4-0723-433e-bc42-2cc95078c3b3 service neutron] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 17:05:00.941142 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8841f341-71ee-4060-b54b-daf0d3c68c04 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d9bd8d649154c979be3744bfd016f95', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b992d6a65df4bf9b27c13192d57c5f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 17:05:00.941749 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8841f341-71ee-4060-b54b-daf0d3c68c04 service neutron] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 17:05:00.942050 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8841f341-71ee-4060-b54b-daf0d3c68c04 service neutron] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 17:05:00.942347 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8841f341-71ee-4060-b54b-daf0d3c68c04 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 17:05:00.944070 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.fernet_utils [None req-8841f341-71ee-4060-b54b-daf0d3c68c04 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87057) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 17:05:00.957874 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8841f341-71ee-4060-b54b-daf0d3c68c04 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 17:05:00.959375 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8841f341-71ee-4060-b54b-daf0d3c68c04 service neutron] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 17:05:00.970831 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 1239/2490] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 1 17:05:00 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 17:05:07.385978 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG dbcounter [-] [87058] Writing DB stats keystone:SELECT=6 {{(pid=87058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 17:05:10.966008 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG dbcounter [-] [87057] Writing DB stats keystone:SELECT=6 {{(pid=87057) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 17:05:16.715556 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9653d35-fe12-4305-9287-66cc45636905 None None] REQUEST_METHOD: `POST` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 17:05:16.716051 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9653d35-fe12-4305-9287-66cc45636905 None None] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 17:05:16.716180 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9653d35-fe12-4305-9287-66cc45636905 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 17:05:16.735905 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.auth.core [None req-b9653d35-fe12-4305-9287-66cc45636905 None None] MFA Rules not processed for user `b6a93d1c6c82489099adc8b413280fc0`. Rule list: `[]` (Enabled: `True`). {{(pid=87058) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 17:05:16.737110 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.fernet_utils [None req-b9653d35-fe12-4305-9287-66cc45636905 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 17:05:16.792260 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 1252/2491] 10.222.0.20 () {58 vars in 1009 bytes} [Mon Aug 1 17:05:16 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 17:05:16.952753 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8841f341-71ee-4060-b54b-daf0d3c68c04 service neutron] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 17:05:16.954205 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.fernet_utils [None req-8841f341-71ee-4060-b54b-daf0d3c68c04 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87057) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 17:05:17.024264 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8841f341-71ee-4060-b54b-daf0d3c68c04 service neutron] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 17:05:17.029381 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-232305b3-acdb-491b-9ff0-b6ad65566f76 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ab4add1faa04faa811ef7a9341cc35c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b992d6a65df4bf9b27c13192d57c5f0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 17:05:17.029998 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232305b3-acdb-491b-9ff0-b6ad65566f76 service swift] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 17:05:17.030304 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232305b3-acdb-491b-9ff0-b6ad65566f76 service swift] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 17:05:17.030579 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232305b3-acdb-491b-9ff0-b6ad65566f76 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 17:05:17.038655 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-232305b3-acdb-491b-9ff0-b6ad65566f76 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 17:05:17.040243 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-232305b3-acdb-491b-9ff0-b6ad65566f76 service swift] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 17:05:17.048833 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 1240/2492] 10.222.0.20 () {60 vars in 1384 bytes} [Mon Aug 1 17:05:16 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 01 17:05:26.784634 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG dbcounter [-] [87058] Writing DB stats keystone:SELECT=25 {{(pid=87058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 17:05:27.045751 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG dbcounter [-] [87057] Writing DB stats keystone:SELECT=32 {{(pid=87057) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 17:05:29.633650 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9653d35-fe12-4305-9287-66cc45636905 None None] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 17:05:29.643546 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9653d35-fe12-4305-9287-66cc45636905 None None] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 17:05:29.647133 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09551bc2-2c71-4275-9d7a-9dd19e638a75 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d9bd8d649154c979be3744bfd016f95', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b992d6a65df4bf9b27c13192d57c5f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 17:05:29.647746 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09551bc2-2c71-4275-9d7a-9dd19e638a75 service neutron] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 17:05:29.648058 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09551bc2-2c71-4275-9d7a-9dd19e638a75 service neutron] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 17:05:29.648334 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09551bc2-2c71-4275-9d7a-9dd19e638a75 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 17:05:29.650123 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.fernet_utils [None req-09551bc2-2c71-4275-9d7a-9dd19e638a75 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 17:05:29.661245 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09551bc2-2c71-4275-9d7a-9dd19e638a75 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 17:05:29.662730 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09551bc2-2c71-4275-9d7a-9dd19e638a75 service neutron] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 17:05:29.671813 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 1253/2493] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 1 17:05:29 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 17:05:35.595905 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b07203a-29af-4dce-9534-5931122558f9 None None] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 17:05:35.596443 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b07203a-29af-4dce-9534-5931122558f9 None None] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 17:05:35.596443 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b07203a-29af-4dce-9534-5931122558f9 None None] PATH_INFO: `/` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 17:05:35.597857 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 1241/2494] 10.222.0.20 () {54 vars in 921 bytes} [Mon Aug 1 17:05:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 17:05:35.604261 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f314aa-9083-49cb-a018-a2d0b236fd29 None None] REQUEST_METHOD: `POST` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 17:05:35.605374 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f314aa-9083-49cb-a018-a2d0b236fd29 None None] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 17:05:35.605477 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f314aa-9083-49cb-a018-a2d0b236fd29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 17:05:35.655218 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.auth.core [None req-f8f314aa-9083-49cb-a018-a2d0b236fd29 None None] MFA Rules not processed for user `6533106d03eb4ca185a8839336d2aff9`. Rule list: `[]` (Enabled: `True`). {{(pid=87058) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 17:05:35.656708 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.fernet_utils [None req-f8f314aa-9083-49cb-a018-a2d0b236fd29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 17:05:35.667797 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 1254/2495] 10.222.0.20 () {58 vars in 1018 bytes} [Mon Aug 1 17:05:35 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 64 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 17:05:35.682757 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b07203a-29af-4dce-9534-5931122558f9 None None] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 17:05:35.684621 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.fernet_utils [None req-9b07203a-29af-4dce-9534-5931122558f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87057) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 17:05:35.744568 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b07203a-29af-4dce-9534-5931122558f9 None None] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 17:05:35.748276 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c7fd13-f056-4854-9525-b3acc1c4db65 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd80694e439e2401d9491712666409016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b992d6a65df4bf9b27c13192d57c5f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 17:05:35.748910 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c7fd13-f056-4854-9525-b3acc1c4db65 service glance] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 17:05:35.749197 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c7fd13-f056-4854-9525-b3acc1c4db65 service glance] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 17:05:35.749472 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c7fd13-f056-4854-9525-b3acc1c4db65 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 17:05:35.759998 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39c7fd13-f056-4854-9525-b3acc1c4db65 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 17:05:35.762171 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39c7fd13-f056-4854-9525-b3acc1c4db65 service glance] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 17:05:35.773034 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 1242/2496] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 1 17:05:35 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 17:05:35.954724 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d399bbb-48b1-4d1e-ba2d-7f6e1a2cd5d4 None None] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 17:05:35.954950 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d399bbb-48b1-4d1e-ba2d-7f6e1a2cd5d4 None None] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 17:05:35.955229 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d399bbb-48b1-4d1e-ba2d-7f6e1a2cd5d4 None None] PATH_INFO: `/` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 17:05:35.956596 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 1255/2497] 10.222.0.20 () {54 vars in 921 bytes} [Mon Aug 1 17:05:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 17:05:35.961247 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa16f0cb-3be5-4d0e-8afb-6653d66c00e7 None None] REQUEST_METHOD: `POST` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 17:05:35.961557 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa16f0cb-3be5-4d0e-8afb-6653d66c00e7 None None] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 17:05:35.961835 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa16f0cb-3be5-4d0e-8afb-6653d66c00e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 17:05:36.001690 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.auth.core [None req-aa16f0cb-3be5-4d0e-8afb-6653d66c00e7 None None] MFA Rules not processed for user `3d9bd8d649154c979be3744bfd016f95`. Rule list: `[]` (Enabled: `True`). {{(pid=87057) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 17:05:36.002894 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.fernet_utils [None req-aa16f0cb-3be5-4d0e-8afb-6653d66c00e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87057) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 17:05:36.009307 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 1243/2498] 10.222.0.20 () {58 vars in 1018 bytes} [Mon Aug 1 17:05:35 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 17:05:36.021840 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d399bbb-48b1-4d1e-ba2d-7f6e1a2cd5d4 None None] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 17:05:36.033450 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d399bbb-48b1-4d1e-ba2d-7f6e1a2cd5d4 None None] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 17:05:36.036935 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a41a5687-ab35-4612-a763-bc4894c38cb9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d9bd8d649154c979be3744bfd016f95', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b992d6a65df4bf9b27c13192d57c5f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 17:05:36.037536 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41a5687-ab35-4612-a763-bc4894c38cb9 service neutron] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 17:05:36.037844 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41a5687-ab35-4612-a763-bc4894c38cb9 service neutron] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 17:05:36.038153 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41a5687-ab35-4612-a763-bc4894c38cb9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 17:05:36.045226 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a41a5687-ab35-4612-a763-bc4894c38cb9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 17:05:36.046682 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a41a5687-ab35-4612-a763-bc4894c38cb9 service neutron] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 17:05:36.054480 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 1256/2499] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 1 17:05:36 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 33 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 17:05:36.136744 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa02c893-0b5a-440c-9404-4e96f4172ef3 None None] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 17:05:36.146225 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa02c893-0b5a-440c-9404-4e96f4172ef3 None None] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 17:05:36.146225 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa02c893-0b5a-440c-9404-4e96f4172ef3 None None] PATH_INFO: `/` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 17:05:36.146225 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 1244/2500] 10.222.0.20 () {54 vars in 921 bytes} [Mon Aug 1 17:05:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 17:05:36.146537 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c2827a-20ad-40df-9e16-7b75d413ae3b None None] REQUEST_METHOD: `POST` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 17:05:36.146537 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c2827a-20ad-40df-9e16-7b75d413ae3b None None] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 17:05:36.146537 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c2827a-20ad-40df-9e16-7b75d413ae3b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 17:05:36.165387 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.auth.core [None req-13c2827a-20ad-40df-9e16-7b75d413ae3b None None] MFA Rules not processed for user `6533106d03eb4ca185a8839336d2aff9`. Rule list: `[]` (Enabled: `True`). {{(pid=87058) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 17:05:36.166756 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.fernet_utils [None req-13c2827a-20ad-40df-9e16-7b75d413ae3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 17:05:36.177006 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 1257/2501] 10.222.0.20 () {58 vars in 1018 bytes} [Mon Aug 1 17:05:36 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 17:05:36.188396 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa02c893-0b5a-440c-9404-4e96f4172ef3 None None] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 17:05:36.199568 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa02c893-0b5a-440c-9404-4e96f4172ef3 None None] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 17:05:36.203309 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc9087fb-59c1-425f-97ce-1232e9ea3661 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d9bd8d649154c979be3744bfd016f95', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b992d6a65df4bf9b27c13192d57c5f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 17:05:36.204031 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9087fb-59c1-425f-97ce-1232e9ea3661 service neutron] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 17:05:36.204445 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9087fb-59c1-425f-97ce-1232e9ea3661 service neutron] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 17:05:36.204718 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9087fb-59c1-425f-97ce-1232e9ea3661 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 17:05:36.215847 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc9087fb-59c1-425f-97ce-1232e9ea3661 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 17:05:36.217212 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc9087fb-59c1-425f-97ce-1232e9ea3661 service neutron] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 17:05:36.227970 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 1245/2502] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 1 17:05:36 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 17:05:36.949542 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13c2827a-20ad-40df-9e16-7b75d413ae3b None None] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 17:05:36.959236 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13c2827a-20ad-40df-9e16-7b75d413ae3b None None] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 17:05:36.963201 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a16a95b4-ad01-4002-b048-560c713cb41d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d9bd8d649154c979be3744bfd016f95', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b992d6a65df4bf9b27c13192d57c5f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 17:05:36.963812 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16a95b4-ad01-4002-b048-560c713cb41d service neutron] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 17:05:36.964659 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16a95b4-ad01-4002-b048-560c713cb41d service neutron] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 17:05:36.965628 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16a95b4-ad01-4002-b048-560c713cb41d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 17:05:36.967477 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.fernet_utils [None req-a16a95b4-ad01-4002-b048-560c713cb41d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 17:05:37.154989 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a16a95b4-ad01-4002-b048-560c713cb41d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 17:05:37.156437 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a16a95b4-ad01-4002-b048-560c713cb41d service neutron] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 17:05:37.165755 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 1258/2503] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 1 17:05:36 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 217 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 17:05:38.850011 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc9087fb-59c1-425f-97ce-1232e9ea3661 service neutron] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 17:05:38.853520 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.fernet_utils [None req-bc9087fb-59c1-425f-97ce-1232e9ea3661 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87057) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 17:05:38.886073 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc9087fb-59c1-425f-97ce-1232e9ea3661 service neutron] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 17:05:38.893832 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1372d9d-450f-4eb0-9480-cf15782ec0a2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6533106d03eb4ca185a8839336d2aff9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b992d6a65df4bf9b27c13192d57c5f0', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 17:05:38.894683 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1372d9d-450f-4eb0-9480-cf15782ec0a2 service nova] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 17:05:38.894956 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1372d9d-450f-4eb0-9480-cf15782ec0a2 service nova] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 17:05:38.895228 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1372d9d-450f-4eb0-9480-cf15782ec0a2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 17:05:38.910647 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.fernet_utils [None req-f1372d9d-450f-4eb0-9480-cf15782ec0a2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87057) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 17:05:38.926136 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1372d9d-450f-4eb0-9480-cf15782ec0a2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 17:05:38.927867 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1372d9d-450f-4eb0-9480-cf15782ec0a2 service nova] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 17:05:38.940008 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 1246/2504] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 1 17:05:38 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 17:05:39.227168 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b7c4d6-2a58-4eef-a948-ffd432e006f1 None None] REQUEST_METHOD: `POST` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 17:05:39.228115 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b7c4d6-2a58-4eef-a948-ffd432e006f1 None None] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 17:05:39.229132 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b7c4d6-2a58-4eef-a948-ffd432e006f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 17:05:39.314922 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.auth.core [None req-26b7c4d6-2a58-4eef-a948-ffd432e006f1 None None] MFA Rules not processed for user `b6a93d1c6c82489099adc8b413280fc0`. Rule list: `[]` (Enabled: `True`). {{(pid=87058) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 17:05:39.320353 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.fernet_utils [None req-26b7c4d6-2a58-4eef-a948-ffd432e006f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 17:05:39.343577 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 1259/2505] 10.222.0.20 () {58 vars in 1009 bytes} [Mon Aug 1 17:05:39 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 17:05:39.356834 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1372d9d-450f-4eb0-9480-cf15782ec0a2 service nova] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 17:05:39.387004 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1372d9d-450f-4eb0-9480-cf15782ec0a2 service nova] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 17:05:39.399905 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-408ca8d3-481b-4e3e-9466-c63d54e06f9c service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ab4add1faa04faa811ef7a9341cc35c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b992d6a65df4bf9b27c13192d57c5f0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 17:05:39.400182 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408ca8d3-481b-4e3e-9466-c63d54e06f9c service swift] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 17:05:39.400466 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408ca8d3-481b-4e3e-9466-c63d54e06f9c service swift] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 17:05:39.400740 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408ca8d3-481b-4e3e-9466-c63d54e06f9c service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 17:05:39.414705 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-408ca8d3-481b-4e3e-9466-c63d54e06f9c service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 17:05:39.416674 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-408ca8d3-481b-4e3e-9466-c63d54e06f9c service swift] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 17:05:39.426610 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 1247/2506] 10.222.0.20 () {60 vars in 1384 bytes} [Mon Aug 1 17:05:39 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 70 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 01 17:05:49.338288 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG dbcounter [-] [87058] Writing DB stats keystone:SELECT=70 {{(pid=87058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 17:05:49.423049 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG dbcounter [-] [87057] Writing DB stats keystone:SELECT=57 {{(pid=87057) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 17:06:01.197317 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b7c4d6-2a58-4eef-a948-ffd432e006f1 None None] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 17:06:01.207241 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b7c4d6-2a58-4eef-a948-ffd432e006f1 None None] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 17:06:01.211279 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0240999-b623-48a1-81e5-80f994a60641 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d9bd8d649154c979be3744bfd016f95', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b992d6a65df4bf9b27c13192d57c5f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 17:06:01.211888 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0240999-b623-48a1-81e5-80f994a60641 service neutron] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 17:06:01.212165 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0240999-b623-48a1-81e5-80f994a60641 service neutron] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 17:06:01.212435 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0240999-b623-48a1-81e5-80f994a60641 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 17:06:01.214311 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.fernet_utils [None req-a0240999-b623-48a1-81e5-80f994a60641 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 17:06:01.227142 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0240999-b623-48a1-81e5-80f994a60641 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 17:06:01.228562 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0240999-b623-48a1-81e5-80f994a60641 service neutron] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 17:06:01.238332 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 1260/2507] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 1 17:06:01 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 17:06:01.481842 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-408ca8d3-481b-4e3e-9466-c63d54e06f9c service swift] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 17:06:01.495429 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-408ca8d3-481b-4e3e-9466-c63d54e06f9c service swift] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 17:06:01.504683 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8743223c-4883-42f2-aec1-147dd4a0ee95 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6533106d03eb4ca185a8839336d2aff9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b992d6a65df4bf9b27c13192d57c5f0', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 17:06:01.504683 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8743223c-4883-42f2-aec1-147dd4a0ee95 service nova] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 17:06:01.504683 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8743223c-4883-42f2-aec1-147dd4a0ee95 service nova] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 17:06:01.505980 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8743223c-4883-42f2-aec1-147dd4a0ee95 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 17:06:01.508968 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.fernet_utils [None req-8743223c-4883-42f2-aec1-147dd4a0ee95 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87057) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 17:06:01.525817 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8743223c-4883-42f2-aec1-147dd4a0ee95 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 17:06:01.527539 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8743223c-4883-42f2-aec1-147dd4a0ee95 service nova] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 17:06:01.546215 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 1248/2508] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 1 17:06:01 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 58 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 17:06:04.281128 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13f1afb-7dd1-48e1-a255-d4e1c9634305 None None] REQUEST_METHOD: `POST` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 17:06:04.281624 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13f1afb-7dd1-48e1-a255-d4e1c9634305 None None] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 17:06:04.281924 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13f1afb-7dd1-48e1-a255-d4e1c9634305 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 17:06:04.305296 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.auth.core [None req-a13f1afb-7dd1-48e1-a255-d4e1c9634305 None None] MFA Rules not processed for user `b6a93d1c6c82489099adc8b413280fc0`. Rule list: `[]` (Enabled: `True`). {{(pid=87058) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 17:06:04.306546 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.fernet_utils [None req-a13f1afb-7dd1-48e1-a255-d4e1c9634305 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 17:06:04.319623 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 1261/2509] 10.222.0.20 () {58 vars in 967 bytes} [Mon Aug 1 17:06:04 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 17:06:04.335641 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8743223c-4883-42f2-aec1-147dd4a0ee95 service nova] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 17:06:04.347638 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8743223c-4883-42f2-aec1-147dd4a0ee95 service nova] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 17:06:04.352831 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e74391-6c9c-45f3-90b4-9a4cd85237ed service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ab4add1faa04faa811ef7a9341cc35c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b992d6a65df4bf9b27c13192d57c5f0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 17:06:04.353455 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e74391-6c9c-45f3-90b4-9a4cd85237ed service swift] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 17:06:04.353736 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e74391-6c9c-45f3-90b4-9a4cd85237ed service swift] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 17:06:04.354010 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e74391-6c9c-45f3-90b4-9a4cd85237ed service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 17:06:04.364585 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9e74391-6c9c-45f3-90b4-9a4cd85237ed service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 17:06:04.370795 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9e74391-6c9c-45f3-90b4-9a4cd85237ed service swift] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 17:06:04.380811 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 1249/2510] 10.222.0.20 () {60 vars in 1384 bytes} [Mon Aug 1 17:06:04 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 01 17:06:09.215540 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a13f1afb-7dd1-48e1-a255-d4e1c9634305 None None] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 17:06:09.216942 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.fernet_utils [None req-a13f1afb-7dd1-48e1-a255-d4e1c9634305 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 17:06:09.231686 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a13f1afb-7dd1-48e1-a255-d4e1c9634305 None None] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 17:06:09.236275 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c51dd9-f880-476a-8a34-fb4dbf423fec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 17:06:09.237013 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c51dd9-f880-476a-8a34-fb4dbf423fec admin admin] REQUEST_METHOD: `DELETE` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 17:06:09.237856 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c51dd9-f880-476a-8a34-fb4dbf423fec admin admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 17:06:09.238211 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c51dd9-f880-476a-8a34-fb4dbf423fec admin admin] PATH_INFO: `/v3/users/8c4a2a503fa14203bcf5b98bfbcccbf8` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 17:06:09.239638 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5c51dd9-f880-476a-8a34-fb4dbf423fec admin admin] RBAC: Authorizing `identity:delete_user(user_id=8c4a2a503fa14203bcf5b98bfbcccbf8)` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 17:06:09.241535 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5c51dd9-f880-476a-8a34-fb4dbf423fec admin admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 17:06:09.362389 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.notifications [None req-f5c51dd9-f880-476a-8a34-fb4dbf423fec admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8c4a2a503fa14203bcf5b98bfbcccbf8'} {{(pid=87058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 17:06:09.365063 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.notifications [None req-f5c51dd9-f880-476a-8a34-fb4dbf423fec admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8c4a2a503fa14203bcf5b98bfbcccbf8'} {{(pid=87058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 17:06:09.365063 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.notifications [None req-f5c51dd9-f880-476a-8a34-fb4dbf423fec admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8c4a2a503fa14203bcf5b98bfbcccbf8'} {{(pid=87058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 17:06:09.376680 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.notifications [None req-f5c51dd9-f880-476a-8a34-fb4dbf423fec admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8c4a2a503fa14203bcf5b98bfbcccbf8'} {{(pid=87058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 17:06:09.383876 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.notifications [None req-f5c51dd9-f880-476a-8a34-fb4dbf423fec admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8c4a2a503fa14203bcf5b98bfbcccbf8'} {{(pid=87058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 17:06:09.427347 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 1262/2511] 10.222.0.20 () {58 vars in 1224 bytes} [Mon Aug 1 17:06:09 2022] DELETE /identity/v3/users/8c4a2a503fa14203bcf5b98bfbcccbf8 => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 17:06:09.668645 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e74391-6c9c-45f3-90b4-9a4cd85237ed service swift] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 17:06:09.680078 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e74391-6c9c-45f3-90b4-9a4cd85237ed service swift] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 17:06:09.737206 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09eae297-2241-49e5-8a72-3e461e9e2ba7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 17:06:09.737851 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09eae297-2241-49e5-8a72-3e461e9e2ba7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 17:06:09.738161 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09eae297-2241-49e5-8a72-3e461e9e2ba7 admin admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 17:06:09.738461 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09eae297-2241-49e5-8a72-3e461e9e2ba7 admin admin] PATH_INFO: `/v3/projects/26251afa3c5e43839015776368defa1d` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 17:06:09.739905 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09eae297-2241-49e5-8a72-3e461e9e2ba7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=26251afa3c5e43839015776368defa1d)` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 17:06:09.741827 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09eae297-2241-49e5-8a72-3e461e9e2ba7 admin admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 17:06:09.856017 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.notifications [None req-09eae297-2241-49e5-8a72-3e461e9e2ba7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '26251afa3c5e43839015776368defa1d', 'request_id': 'req-09eae297-2241-49e5-8a72-3e461e9e2ba7'} {{(pid=87057) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 17:06:09.862069 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.notifications [None req-09eae297-2241-49e5-8a72-3e461e9e2ba7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '26251afa3c5e43839015776368defa1d', 'request_id': 'req-09eae297-2241-49e5-8a72-3e461e9e2ba7'} {{(pid=87057) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 17:06:09.867752 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.notifications [None req-09eae297-2241-49e5-8a72-3e461e9e2ba7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '26251afa3c5e43839015776368defa1d', 'request_id': 'req-09eae297-2241-49e5-8a72-3e461e9e2ba7'} {{(pid=87057) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 17:06:09.917665 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.notifications [None req-09eae297-2241-49e5-8a72-3e461e9e2ba7 admin admin] The token cache is being invalidate because project 26251afa3c5e43839015776368defa1d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87057) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 17:06:09.917955 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.notifications [None req-09eae297-2241-49e5-8a72-3e461e9e2ba7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87057) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 17:06:09.918596 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.notifications [None req-09eae297-2241-49e5-8a72-3e461e9e2ba7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87057) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 17:06:09.920142 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 1250/2512] 10.222.0.20 () {58 vars in 1233 bytes} [Mon Aug 1 17:06:09 2022] DELETE /identity/v3/projects/26251afa3c5e43839015776368defa1d => generated 0 bytes in 252 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 17:06:09.924202 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c51dd9-f880-476a-8a34-fb4dbf423fec admin admin] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 17:06:09.925632 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.fernet_utils [None req-f5c51dd9-f880-476a-8a34-fb4dbf423fec admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 17:06:09.938693 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c51dd9-f880-476a-8a34-fb4dbf423fec admin admin] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 17:06:09.994694 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe4326e-cea8-40a5-a445-8eff41ef8fec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 17:06:09.995333 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe4326e-cea8-40a5-a445-8eff41ef8fec admin admin] REQUEST_METHOD: `DELETE` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 17:06:09.995610 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe4326e-cea8-40a5-a445-8eff41ef8fec admin admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 17:06:09.995881 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe4326e-cea8-40a5-a445-8eff41ef8fec admin admin] PATH_INFO: `/v3/users/27ba54a036294a119db116883672fea9` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 17:06:09.997005 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfe4326e-cea8-40a5-a445-8eff41ef8fec admin admin] RBAC: Authorizing `identity:delete_user(user_id=27ba54a036294a119db116883672fea9)` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 17:06:09.998985 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfe4326e-cea8-40a5-a445-8eff41ef8fec admin admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 17:06:10.161438 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.notifications [None req-dfe4326e-cea8-40a5-a445-8eff41ef8fec admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '27ba54a036294a119db116883672fea9'} {{(pid=87058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 17:06:10.162283 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.notifications [None req-dfe4326e-cea8-40a5-a445-8eff41ef8fec admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '27ba54a036294a119db116883672fea9'} {{(pid=87058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 17:06:10.162822 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.notifications [None req-dfe4326e-cea8-40a5-a445-8eff41ef8fec admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '27ba54a036294a119db116883672fea9'} {{(pid=87058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 17:06:10.168207 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG dbcounter [-] [87058] Writing DB stats keystone:SELECT=81,keystone:DELETE=18,keystone:INSERT=1 {{(pid=87058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 17:06:10.174888 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.notifications [None req-dfe4326e-cea8-40a5-a445-8eff41ef8fec admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '27ba54a036294a119db116883672fea9'} {{(pid=87058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 17:06:10.181826 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.notifications [None req-dfe4326e-cea8-40a5-a445-8eff41ef8fec admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '27ba54a036294a119db116883672fea9'} {{(pid=87058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 17:06:10.244781 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 1263/2513] 10.222.0.20 () {58 vars in 1224 bytes} [Mon Aug 1 17:06:09 2022] DELETE /identity/v3/users/27ba54a036294a119db116883672fea9 => generated 0 bytes in 321 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 17:06:10.509557 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09eae297-2241-49e5-8a72-3e461e9e2ba7 admin admin] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 17:06:10.511069 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.fernet_utils [None req-09eae297-2241-49e5-8a72-3e461e9e2ba7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87057) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 17:06:10.524689 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09eae297-2241-49e5-8a72-3e461e9e2ba7 admin admin] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 17:06:10.589476 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5129641f-622e-4559-aca7-8167f0c1f406 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 17:06:10.590152 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5129641f-622e-4559-aca7-8167f0c1f406 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 17:06:10.590438 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5129641f-622e-4559-aca7-8167f0c1f406 admin admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 17:06:10.590698 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5129641f-622e-4559-aca7-8167f0c1f406 admin admin] PATH_INFO: `/v3/projects/71157b72fafd4c8dbb3b1681d56992c0` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 17:06:10.591888 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5129641f-622e-4559-aca7-8167f0c1f406 admin admin] RBAC: Authorizing `identity:delete_project(project_id=71157b72fafd4c8dbb3b1681d56992c0)` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 17:06:10.593816 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5129641f-622e-4559-aca7-8167f0c1f406 admin admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 17:06:10.600437 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG dbcounter [-] [87057] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=87057) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 17:06:10.742562 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.notifications [None req-5129641f-622e-4559-aca7-8167f0c1f406 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '71157b72fafd4c8dbb3b1681d56992c0', 'request_id': 'req-5129641f-622e-4559-aca7-8167f0c1f406'} {{(pid=87057) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 17:06:10.748880 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.notifications [None req-5129641f-622e-4559-aca7-8167f0c1f406 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '71157b72fafd4c8dbb3b1681d56992c0', 'request_id': 'req-5129641f-622e-4559-aca7-8167f0c1f406'} {{(pid=87057) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 17:06:10.755212 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.notifications [None req-5129641f-622e-4559-aca7-8167f0c1f406 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '71157b72fafd4c8dbb3b1681d56992c0', 'request_id': 'req-5129641f-622e-4559-aca7-8167f0c1f406'} {{(pid=87057) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 17:06:10.820554 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.notifications [None req-5129641f-622e-4559-aca7-8167f0c1f406 admin admin] The token cache is being invalidate because project 71157b72fafd4c8dbb3b1681d56992c0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87057) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 17:06:10.820866 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.notifications [None req-5129641f-622e-4559-aca7-8167f0c1f406 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87057) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 17:06:10.821488 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.notifications [None req-5129641f-622e-4559-aca7-8167f0c1f406 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87057) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 17:06:10.823051 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 1251/2514] 10.222.0.20 () {58 vars in 1233 bytes} [Mon Aug 1 17:06:10 2022] DELETE /identity/v3/projects/71157b72fafd4c8dbb3b1681d56992c0 => generated 0 bytes in 314 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 17:06:20.314653 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f7259a-7e35-40a5-aa73-1b99ccee9266 None None] REQUEST_METHOD: `POST` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 17:06:20.315163 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f7259a-7e35-40a5-aa73-1b99ccee9266 None None] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 17:06:20.315263 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f7259a-7e35-40a5-aa73-1b99ccee9266 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 17:06:20.334703 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.auth.core [None req-e1f7259a-7e35-40a5-aa73-1b99ccee9266 None None] MFA Rules not processed for user `f3484bf0457a407d954de53bf9bc8a91`. Rule list: `[]` (Enabled: `True`). {{(pid=87058) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 17:06:20.335932 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.fernet_utils [None req-e1f7259a-7e35-40a5-aa73-1b99ccee9266 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 17:06:20.392192 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 1264/2515] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 1 17:06:20 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 17:06:20.710821 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5129641f-622e-4559-aca7-8167f0c1f406 admin admin] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 17:06:20.712283 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.fernet_utils [None req-5129641f-622e-4559-aca7-8167f0c1f406 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87057) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 17:06:20.772396 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5129641f-622e-4559-aca7-8167f0c1f406 admin admin] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 17:06:20.776012 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c3ef932-7970-4f96-bbd5-a916b7893877 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d9bd8d649154c979be3744bfd016f95', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b992d6a65df4bf9b27c13192d57c5f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 17:06:20.776630 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3ef932-7970-4f96-bbd5-a916b7893877 service neutron] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 17:06:20.776911 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3ef932-7970-4f96-bbd5-a916b7893877 service neutron] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 17:06:20.777193 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3ef932-7970-4f96-bbd5-a916b7893877 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 17:06:20.779026 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.fernet_utils [None req-3c3ef932-7970-4f96-bbd5-a916b7893877 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87057) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 17:06:20.788935 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c3ef932-7970-4f96-bbd5-a916b7893877 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 17:06:20.790750 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c3ef932-7970-4f96-bbd5-a916b7893877 service neutron] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 17:06:20.798891 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 1252/2516] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 1 17:06:20 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 17:06:22.227648 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f7259a-7e35-40a5-aa73-1b99ccee9266 None None] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 17:06:22.229095 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.fernet_utils [None req-e1f7259a-7e35-40a5-aa73-1b99ccee9266 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 17:06:22.301156 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f7259a-7e35-40a5-aa73-1b99ccee9266 None None] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 17:06:22.305946 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a60c792-0fa1-4e92-a1b6-384332661edd service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8362293261614623bcdabc4120a97a9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b992d6a65df4bf9b27c13192d57c5f0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 17:06:22.306585 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a60c792-0fa1-4e92-a1b6-384332661edd service placement] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 17:06:22.306859 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a60c792-0fa1-4e92-a1b6-384332661edd service placement] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 17:06:22.307126 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a60c792-0fa1-4e92-a1b6-384332661edd service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 17:06:22.308837 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.fernet_utils [None req-4a60c792-0fa1-4e92-a1b6-384332661edd service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 17:06:22.320418 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a60c792-0fa1-4e92-a1b6-384332661edd service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 17:06:22.321977 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a60c792-0fa1-4e92-a1b6-384332661edd service placement] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 17:06:22.331594 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 1265/2517] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 1 17:06:22 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 17:06:25.956231 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c3ef932-7970-4f96-bbd5-a916b7893877 service neutron] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 17:06:25.958117 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.fernet_utils [None req-3c3ef932-7970-4f96-bbd5-a916b7893877 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87057) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 17:06:26.048598 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c3ef932-7970-4f96-bbd5-a916b7893877 service neutron] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 17:06:26.055178 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc8e966-8586-4258-be5b-41346dbf94e3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6533106d03eb4ca185a8839336d2aff9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b992d6a65df4bf9b27c13192d57c5f0', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 17:06:26.055805 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc8e966-8586-4258-be5b-41346dbf94e3 service nova] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 17:06:26.056082 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc8e966-8586-4258-be5b-41346dbf94e3 service nova] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 17:06:26.056355 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc8e966-8586-4258-be5b-41346dbf94e3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 17:06:26.058172 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.fernet_utils [None req-4dc8e966-8586-4258-be5b-41346dbf94e3 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87057) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 17:06:26.071448 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dc8e966-8586-4258-be5b-41346dbf94e3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 17:06:26.073060 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dc8e966-8586-4258-be5b-41346dbf94e3 service nova] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 17:06:26.084576 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 1253/2518] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 1 17:06:25 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 17:06:31.310494 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a60c792-0fa1-4e92-a1b6-384332661edd service placement] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 17:06:31.319950 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a60c792-0fa1-4e92-a1b6-384332661edd service placement] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 17:06:31.323563 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-656b39a7-dde9-4d51-91d6-342490dc918a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d9bd8d649154c979be3744bfd016f95', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b992d6a65df4bf9b27c13192d57c5f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 17:06:31.324163 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656b39a7-dde9-4d51-91d6-342490dc918a service neutron] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 17:06:31.324443 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656b39a7-dde9-4d51-91d6-342490dc918a service neutron] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 17:06:31.324714 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656b39a7-dde9-4d51-91d6-342490dc918a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 17:06:31.326509 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.fernet_utils [None req-656b39a7-dde9-4d51-91d6-342490dc918a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 17:06:31.390960 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-656b39a7-dde9-4d51-91d6-342490dc918a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 17:06:31.392391 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-656b39a7-dde9-4d51-91d6-342490dc918a service neutron] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 17:06:31.397996 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG dbcounter [-] [87058] Writing DB stats keystone:SELECT=97,keystone:DELETE=2,keystone:INSERT=1 {{(pid=87058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 17:06:31.403149 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 1266/2519] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 1 17:06:31 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 17:06:35.169714 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc8e966-8586-4258-be5b-41346dbf94e3 service nova] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 17:06:35.169714 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.fernet_utils [None req-4dc8e966-8586-4258-be5b-41346dbf94e3 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87057) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 17:06:35.180092 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc8e966-8586-4258-be5b-41346dbf94e3 service nova] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 17:06:35.184819 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd6e0ff4-a4ed-46ee-bb3d-fba83a93b29f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 17:06:35.185453 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6e0ff4-a4ed-46ee-bb3d-fba83a93b29f admin admin] REQUEST_METHOD: `DELETE` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 17:06:35.185901 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6e0ff4-a4ed-46ee-bb3d-fba83a93b29f admin admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 17:06:35.186209 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6e0ff4-a4ed-46ee-bb3d-fba83a93b29f admin admin] PATH_INFO: `/v3/users/c2079b0fbf7b4a8c88d2522da9f2c73b` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 17:06:35.200000 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd6e0ff4-a4ed-46ee-bb3d-fba83a93b29f admin admin] RBAC: Authorizing `identity:delete_user(user_id=c2079b0fbf7b4a8c88d2522da9f2c73b)` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 17:06:35.201982 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd6e0ff4-a4ed-46ee-bb3d-fba83a93b29f admin admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 17:06:35.206414 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG dbcounter [-] [87057] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=87057) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 17:06:35.345250 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.notifications [None req-dd6e0ff4-a4ed-46ee-bb3d-fba83a93b29f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c2079b0fbf7b4a8c88d2522da9f2c73b'} {{(pid=87057) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 17:06:35.346259 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.notifications [None req-dd6e0ff4-a4ed-46ee-bb3d-fba83a93b29f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c2079b0fbf7b4a8c88d2522da9f2c73b'} {{(pid=87057) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 17:06:35.403171 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.notifications [None req-dd6e0ff4-a4ed-46ee-bb3d-fba83a93b29f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c2079b0fbf7b4a8c88d2522da9f2c73b'} {{(pid=87057) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 17:06:35.403739 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.notifications [None req-dd6e0ff4-a4ed-46ee-bb3d-fba83a93b29f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c2079b0fbf7b4a8c88d2522da9f2c73b'} {{(pid=87057) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 17:06:35.411961 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.notifications [None req-dd6e0ff4-a4ed-46ee-bb3d-fba83a93b29f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c2079b0fbf7b4a8c88d2522da9f2c73b'} {{(pid=87057) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 17:06:35.430175 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 1254/2520] 10.222.0.20 () {58 vars in 1224 bytes} [Mon Aug 1 17:06:35 2022] DELETE /identity/v3/users/c2079b0fbf7b4a8c88d2522da9f2c73b => generated 0 bytes in 266 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 17:06:35.690707 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-656b39a7-dde9-4d51-91d6-342490dc918a service neutron] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 17:06:35.701648 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-656b39a7-dde9-4d51-91d6-342490dc918a service neutron] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 17:06:35.764172 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f04e99c-d20e-4539-8813-81b4881c3827 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 17:06:35.764811 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f04e99c-d20e-4539-8813-81b4881c3827 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 17:06:35.765085 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f04e99c-d20e-4539-8813-81b4881c3827 admin admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 17:06:35.765355 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f04e99c-d20e-4539-8813-81b4881c3827 admin admin] PATH_INFO: `/v3/projects/ee3d55c8b96e4821bd2592a3ee665533` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 17:06:35.776500 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f04e99c-d20e-4539-8813-81b4881c3827 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ee3d55c8b96e4821bd2592a3ee665533)` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 17:06:35.778486 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f04e99c-d20e-4539-8813-81b4881c3827 admin admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 17:06:35.873097 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.notifications [None req-0f04e99c-d20e-4539-8813-81b4881c3827 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ee3d55c8b96e4821bd2592a3ee665533', 'request_id': 'req-0f04e99c-d20e-4539-8813-81b4881c3827'} {{(pid=87058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 17:06:35.879939 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.notifications [None req-0f04e99c-d20e-4539-8813-81b4881c3827 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ee3d55c8b96e4821bd2592a3ee665533', 'request_id': 'req-0f04e99c-d20e-4539-8813-81b4881c3827'} {{(pid=87058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 17:06:35.893289 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.notifications [None req-0f04e99c-d20e-4539-8813-81b4881c3827 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ee3d55c8b96e4821bd2592a3ee665533', 'request_id': 'req-0f04e99c-d20e-4539-8813-81b4881c3827'} {{(pid=87058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 17:06:35.981095 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.notifications [None req-0f04e99c-d20e-4539-8813-81b4881c3827 admin admin] The token cache is being invalidate because project ee3d55c8b96e4821bd2592a3ee665533 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87058) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 17:06:35.981290 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.notifications [None req-0f04e99c-d20e-4539-8813-81b4881c3827 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 17:06:35.981898 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.notifications [None req-0f04e99c-d20e-4539-8813-81b4881c3827 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 17:06:35.984277 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 1267/2521] 10.222.0.20 () {58 vars in 1233 bytes} [Mon Aug 1 17:06:35 2022] DELETE /identity/v3/projects/ee3d55c8b96e4821bd2592a3ee665533 => generated 0 bytes in 294 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 17:06:35.990121 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58b4355-fd75-4a58-a8be-7eff4da069bc None None] REQUEST_METHOD: `POST` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 17:06:35.990544 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58b4355-fd75-4a58-a8be-7eff4da069bc None None] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 17:06:35.991091 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58b4355-fd75-4a58-a8be-7eff4da069bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 17:06:36.011287 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.auth.core [None req-a58b4355-fd75-4a58-a8be-7eff4da069bc None None] MFA Rules not processed for user `1007f8e379b74e169b98fb7c3c0ba5b4`. Rule list: `[]` (Enabled: `True`). {{(pid=87057) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 17:06:36.012499 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.fernet_utils [None req-a58b4355-fd75-4a58-a8be-7eff4da069bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87057) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 17:06:36.071772 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 1255/2522] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 1 17:06:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 17:06:36.080450 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27298822-b3a8-4a69-97c4-8b113d9c8dd1 None None] REQUEST_METHOD: `POST` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 17:06:36.080730 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27298822-b3a8-4a69-97c4-8b113d9c8dd1 None None] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 17:06:36.080997 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27298822-b3a8-4a69-97c4-8b113d9c8dd1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 17:06:36.100380 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.auth.core [None req-27298822-b3a8-4a69-97c4-8b113d9c8dd1 None None] MFA Rules not processed for user `1007f8e379b74e169b98fb7c3c0ba5b4`. Rule list: `[]` (Enabled: `True`). {{(pid=87058) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 17:06:36.101578 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.fernet_utils [None req-27298822-b3a8-4a69-97c4-8b113d9c8dd1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 17:06:36.110172 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 1268/2523] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 1 17:06:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 17:06:36.115236 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a58b4355-fd75-4a58-a8be-7eff4da069bc None None] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 17:06:36.125195 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a58b4355-fd75-4a58-a8be-7eff4da069bc None None] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 17:06:36.129867 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0737714-0256-42e3-9c5b-1309d7941078 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 17:06:36.131241 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0737714-0256-42e3-9c5b-1309d7941078 admin admin] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 17:06:36.131636 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0737714-0256-42e3-9c5b-1309d7941078 admin admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 17:06:36.131912 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0737714-0256-42e3-9c5b-1309d7941078 admin admin] PATH_INFO: `/v3/domains` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 17:06:36.132617 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0737714-0256-42e3-9c5b-1309d7941078 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87057) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 17:06:36.132924 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0737714-0256-42e3-9c5b-1309d7941078 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 17:06:36.134554 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0737714-0256-42e3-9c5b-1309d7941078 admin admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 17:06:36.146251 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 1256/2524] 10.222.0.20 () {58 vars in 1153 bytes} [Mon Aug 1 17:06:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 17:06:36.150462 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27298822-b3a8-4a69-97c4-8b113d9c8dd1 None None] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 17:06:36.159755 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27298822-b3a8-4a69-97c4-8b113d9c8dd1 None None] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 17:06:36.165021 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f164ba84-1238-49df-98f6-c616b1f27a3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 17:06:36.165641 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f164ba84-1238-49df-98f6-c616b1f27a3f admin admin] REQUEST_METHOD: `POST` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 17:06:36.165908 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f164ba84-1238-49df-98f6-c616b1f27a3f admin admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 17:06:36.166206 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f164ba84-1238-49df-98f6-c616b1f27a3f admin admin] PATH_INFO: `/v3/projects` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 17:06:36.167000 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f164ba84-1238-49df-98f6-c616b1f27a3f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 17:06:36.168952 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f164ba84-1238-49df-98f6-c616b1f27a3f admin admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 17:06:36.245251 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 1269/2525] 10.222.0.20 () {60 vars in 1153 bytes} [Mon Aug 1 17:06:36 2022] POST /identity/v3/projects => generated 365 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 17:06:36.249345 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0737714-0256-42e3-9c5b-1309d7941078 admin admin] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 17:06:36.259795 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0737714-0256-42e3-9c5b-1309d7941078 admin admin] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 17:06:36.318099 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b615bbe6-c17b-46fd-9758-ee3682097eb5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 17:06:36.318789 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b615bbe6-c17b-46fd-9758-ee3682097eb5 admin admin] REQUEST_METHOD: `POST` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 17:06:36.319074 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b615bbe6-c17b-46fd-9758-ee3682097eb5 admin admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 17:06:36.319351 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b615bbe6-c17b-46fd-9758-ee3682097eb5 admin admin] PATH_INFO: `/v3/users` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 17:06:36.320221 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b615bbe6-c17b-46fd-9758-ee3682097eb5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 17:06:36.322180 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b615bbe6-c17b-46fd-9758-ee3682097eb5 admin admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 17:06:36.398053 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG dbcounter [-] [87057] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=87057) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 17:06:36.403041 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 1257/2526] 10.222.0.20 () {60 vars in 1144 bytes} [Mon Aug 1 17:06:36 2022] POST /identity/v3/users => generated 283 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 17:06:36.407341 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f164ba84-1238-49df-98f6-c616b1f27a3f admin admin] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 17:06:36.418312 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f164ba84-1238-49df-98f6-c616b1f27a3f admin admin] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 17:06:36.422907 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4887b4f6-d22f-411a-9d19-0e358e9394a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 17:06:36.423511 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4887b4f6-d22f-411a-9d19-0e358e9394a1 admin admin] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 17:06:36.423798 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4887b4f6-d22f-411a-9d19-0e358e9394a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 17:06:36.424077 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4887b4f6-d22f-411a-9d19-0e358e9394a1 admin admin] PATH_INFO: `/v3/roles` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 17:06:36.424835 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4887b4f6-d22f-411a-9d19-0e358e9394a1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 17:06:36.426456 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4887b4f6-d22f-411a-9d19-0e358e9394a1 admin admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 17:06:36.437418 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 1270/2527] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Aug 1 17:06:36 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 17:06:36.441224 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b615bbe6-c17b-46fd-9758-ee3682097eb5 admin admin] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 17:06:36.451140 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b615bbe6-c17b-46fd-9758-ee3682097eb5 admin admin] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 17:06:36.456322 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f16e1aec-9a85-4d5c-a637-88908690cc89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 17:06:36.456918 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16e1aec-9a85-4d5c-a637-88908690cc89 admin admin] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 17:06:36.457194 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16e1aec-9a85-4d5c-a637-88908690cc89 admin admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 17:06:36.457465 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16e1aec-9a85-4d5c-a637-88908690cc89 admin admin] PATH_INFO: `/v3/roles` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 17:06:36.458265 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f16e1aec-9a85-4d5c-a637-88908690cc89 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 17:06:36.463690 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f16e1aec-9a85-4d5c-a637-88908690cc89 admin admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 17:06:36.470880 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 1258/2528] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Aug 1 17:06:36 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 01 17:06:36.475729 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4887b4f6-d22f-411a-9d19-0e358e9394a1 admin admin] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 17:06:36.485809 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4887b4f6-d22f-411a-9d19-0e358e9394a1 admin admin] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 17:06:36.491605 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45676ea5-99df-4518-b445-220d39f0839d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 17:06:36.492291 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45676ea5-99df-4518-b445-220d39f0839d admin admin] REQUEST_METHOD: `PUT` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 17:06:36.492568 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45676ea5-99df-4518-b445-220d39f0839d admin admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 17:06:36.492840 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45676ea5-99df-4518-b445-220d39f0839d admin admin] PATH_INFO: `/v3/projects/b3210ac64b7e45e0889f7a006e61f1ed/users/270b3170bdd24e0192972c63b53f68ea/roles/0999d3da63a545f18ff3252afdc90518` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 17:06:36.508873 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45676ea5-99df-4518-b445-220d39f0839d admin admin] RBAC: Authorizing `identity:create_grant(project_id=b3210ac64b7e45e0889f7a006e61f1ed, user_id=270b3170bdd24e0192972c63b53f68ea, role_id=0999d3da63a545f18ff3252afdc90518)` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 17:06:36.511615 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45676ea5-99df-4518-b445-220d39f0839d admin admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 17:06:36.598312 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 1271/2529] 10.222.0.20 () {60 vars in 1483 bytes} [Mon Aug 1 17:06:36 2022] PUT /identity/v3/projects/b3210ac64b7e45e0889f7a006e61f1ed/users/270b3170bdd24e0192972c63b53f68ea/roles/0999d3da63a545f18ff3252afdc90518 => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 17:06:36.609180 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2accd44b-3810-4f89-a086-3c8ef628563b None None] REQUEST_METHOD: `POST` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 17:06:36.609466 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2accd44b-3810-4f89-a086-3c8ef628563b None None] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 17:06:36.609742 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2accd44b-3810-4f89-a086-3c8ef628563b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 17:06:36.638435 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.auth.core [None req-2accd44b-3810-4f89-a086-3c8ef628563b None None] MFA Rules not processed for user `270b3170bdd24e0192972c63b53f68ea`. Rule list: `[]` (Enabled: `True`). {{(pid=87057) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 17:06:36.639671 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.fernet_utils [None req-2accd44b-3810-4f89-a086-3c8ef628563b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87057) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 17:06:36.707008 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 1259/2530] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 1 17:06:36 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 17:06:36.718542 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45676ea5-99df-4518-b445-220d39f0839d admin admin] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 17:06:36.720247 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.fernet_utils [None req-45676ea5-99df-4518-b445-220d39f0839d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 17:06:36.734200 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG dbcounter [-] [87058] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=87058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 17:06:36.803937 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45676ea5-99df-4518-b445-220d39f0839d admin admin] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 17:06:36.807669 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-843efbe5-d8ef-4eaa-aac4-5284a0304688 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25970b52ca15448fa1abb1f974d64038', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b992d6a65df4bf9b27c13192d57c5f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 17:06:36.808475 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843efbe5-d8ef-4eaa-aac4-5284a0304688 service cinder] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 17:06:36.808917 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843efbe5-d8ef-4eaa-aac4-5284a0304688 service cinder] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 17:06:36.809589 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843efbe5-d8ef-4eaa-aac4-5284a0304688 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 17:06:36.818694 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-843efbe5-d8ef-4eaa-aac4-5284a0304688 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 17:06:36.820134 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-843efbe5-d8ef-4eaa-aac4-5284a0304688 service cinder] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 17:06:36.829353 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 1272/2531] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 1 17:06:36 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 17:06:46.492596 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2accd44b-3810-4f89-a086-3c8ef628563b None None] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 17:06:46.555755 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2accd44b-3810-4f89-a086-3c8ef628563b None None] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 17:06:46.560273 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ed4791-3159-4ffd-b5ca-fffc6a184e29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 17:06:46.560939 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ed4791-3159-4ffd-b5ca-fffc6a184e29 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 17:06:46.561195 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ed4791-3159-4ffd-b5ca-fffc6a184e29 admin admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 17:06:46.561485 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ed4791-3159-4ffd-b5ca-fffc6a184e29 admin admin] PATH_INFO: `/v3/users/270b3170bdd24e0192972c63b53f68ea` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 17:06:46.562650 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24ed4791-3159-4ffd-b5ca-fffc6a184e29 admin admin] RBAC: Authorizing `identity:delete_user(user_id=270b3170bdd24e0192972c63b53f68ea)` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 17:06:46.564571 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24ed4791-3159-4ffd-b5ca-fffc6a184e29 admin admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 17:06:46.678236 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.notifications [None req-24ed4791-3159-4ffd-b5ca-fffc6a184e29 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '270b3170bdd24e0192972c63b53f68ea'} {{(pid=87057) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 17:06:46.679046 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.notifications [None req-24ed4791-3159-4ffd-b5ca-fffc6a184e29 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '270b3170bdd24e0192972c63b53f68ea'} {{(pid=87057) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 17:06:46.722711 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.notifications [None req-24ed4791-3159-4ffd-b5ca-fffc6a184e29 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '270b3170bdd24e0192972c63b53f68ea'} {{(pid=87057) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 17:06:46.723300 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.notifications [None req-24ed4791-3159-4ffd-b5ca-fffc6a184e29 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '270b3170bdd24e0192972c63b53f68ea'} {{(pid=87057) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 17:06:46.730458 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.notifications [None req-24ed4791-3159-4ffd-b5ca-fffc6a184e29 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '270b3170bdd24e0192972c63b53f68ea'} {{(pid=87057) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 17:06:46.743628 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG dbcounter [-] [87057] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87057) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 17:06:46.748576 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 1260/2532] 10.222.0.20 () {58 vars in 1224 bytes} [Mon Aug 1 17:06:46 2022] DELETE /identity/v3/users/270b3170bdd24e0192972c63b53f68ea => generated 0 bytes in 257 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 17:06:46.753085 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ecb167d-1069-4215-b3fd-a165b3e7a593 None None] REQUEST_METHOD: `POST` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 17:06:46.753700 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ecb167d-1069-4215-b3fd-a165b3e7a593 None None] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 17:06:46.753700 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ecb167d-1069-4215-b3fd-a165b3e7a593 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 17:06:46.773831 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.auth.core [None req-7ecb167d-1069-4215-b3fd-a165b3e7a593 None None] MFA Rules not processed for user `1007f8e379b74e169b98fb7c3c0ba5b4`. Rule list: `[]` (Enabled: `True`). {{(pid=87058) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 17:06:46.775159 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.fernet_utils [None req-7ecb167d-1069-4215-b3fd-a165b3e7a593 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 17:06:46.834613 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 1273/2533] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 1 17:06:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 17:06:46.844237 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ed4791-3159-4ffd-b5ca-fffc6a184e29 admin admin] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 17:06:46.845659 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.fernet_utils [None req-24ed4791-3159-4ffd-b5ca-fffc6a184e29 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87057) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 17:06:46.902756 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ed4791-3159-4ffd-b5ca-fffc6a184e29 admin admin] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 17:06:46.906394 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e25177c6-a121-42fd-80c4-1652be9f275d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d9bd8d649154c979be3744bfd016f95', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b992d6a65df4bf9b27c13192d57c5f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 17:06:46.907014 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25177c6-a121-42fd-80c4-1652be9f275d service neutron] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 17:06:46.907289 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25177c6-a121-42fd-80c4-1652be9f275d service neutron] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 17:06:46.907559 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25177c6-a121-42fd-80c4-1652be9f275d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 17:06:46.916805 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e25177c6-a121-42fd-80c4-1652be9f275d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 17:06:46.918250 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e25177c6-a121-42fd-80c4-1652be9f275d service neutron] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 17:06:46.927577 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 1261/2534] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 1 17:06:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 01 17:06:47.325407 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ecb167d-1069-4215-b3fd-a165b3e7a593 None None] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 17:06:47.326881 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.fernet_utils [None req-7ecb167d-1069-4215-b3fd-a165b3e7a593 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 17:06:47.339893 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ecb167d-1069-4215-b3fd-a165b3e7a593 None None] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 17:06:47.344772 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06aa73b2-ac2a-44ab-b352-de52cb598187 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 17:06:47.345422 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06aa73b2-ac2a-44ab-b352-de52cb598187 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 17:06:47.345685 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06aa73b2-ac2a-44ab-b352-de52cb598187 admin admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 17:06:47.345954 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06aa73b2-ac2a-44ab-b352-de52cb598187 admin admin] PATH_INFO: `/v3/projects/b3210ac64b7e45e0889f7a006e61f1ed` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 17:06:47.347150 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06aa73b2-ac2a-44ab-b352-de52cb598187 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b3210ac64b7e45e0889f7a006e61f1ed)` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 17:06:47.349211 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06aa73b2-ac2a-44ab-b352-de52cb598187 admin admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 17:06:47.489238 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.notifications [None req-06aa73b2-ac2a-44ab-b352-de52cb598187 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b3210ac64b7e45e0889f7a006e61f1ed', 'request_id': 'req-06aa73b2-ac2a-44ab-b352-de52cb598187'} {{(pid=87058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 17:06:47.496305 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.notifications [None req-06aa73b2-ac2a-44ab-b352-de52cb598187 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b3210ac64b7e45e0889f7a006e61f1ed', 'request_id': 'req-06aa73b2-ac2a-44ab-b352-de52cb598187'} {{(pid=87058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 17:06:47.502191 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.notifications [None req-06aa73b2-ac2a-44ab-b352-de52cb598187 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b3210ac64b7e45e0889f7a006e61f1ed', 'request_id': 'req-06aa73b2-ac2a-44ab-b352-de52cb598187'} {{(pid=87058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 17:06:47.550608 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.notifications [None req-06aa73b2-ac2a-44ab-b352-de52cb598187 admin admin] The token cache is being invalidate because project b3210ac64b7e45e0889f7a006e61f1ed was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87058) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 17:06:47.550899 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.notifications [None req-06aa73b2-ac2a-44ab-b352-de52cb598187 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 17:06:47.551532 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.notifications [None req-06aa73b2-ac2a-44ab-b352-de52cb598187 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 17:06:47.553129 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 1274/2535] 10.222.0.20 () {58 vars in 1233 bytes} [Mon Aug 1 17:06:47 2022] DELETE /identity/v3/projects/b3210ac64b7e45e0889f7a006e61f1ed => generated 0 bytes in 228 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 17:06:47.557941 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1ba978-d14d-4d36-ba01-66da525b5242 None None] REQUEST_METHOD: `POST` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 17:06:47.558474 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1ba978-d14d-4d36-ba01-66da525b5242 None None] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 17:06:47.558651 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1ba978-d14d-4d36-ba01-66da525b5242 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 17:06:47.579068 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.auth.core [None req-dd1ba978-d14d-4d36-ba01-66da525b5242 None None] MFA Rules not processed for user `1007f8e379b74e169b98fb7c3c0ba5b4`. Rule list: `[]` (Enabled: `True`). {{(pid=87057) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 17:06:47.580217 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.fernet_utils [None req-dd1ba978-d14d-4d36-ba01-66da525b5242 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87057) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 17:06:47.638558 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 1262/2536] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 1 17:06:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 17:06:47.648437 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66165a0-dd3f-4482-a198-38af8db1f288 None None] REQUEST_METHOD: `POST` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 17:06:47.648723 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66165a0-dd3f-4482-a198-38af8db1f288 None None] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 17:06:47.648997 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66165a0-dd3f-4482-a198-38af8db1f288 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 17:06:47.668196 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.auth.core [None req-d66165a0-dd3f-4482-a198-38af8db1f288 None None] MFA Rules not processed for user `1007f8e379b74e169b98fb7c3c0ba5b4`. Rule list: `[]` (Enabled: `True`). {{(pid=87058) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 17:06:47.669394 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.fernet_utils [None req-d66165a0-dd3f-4482-a198-38af8db1f288 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 17:06:47.677233 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 1275/2537] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 1 17:06:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 17:06:47.681491 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd1ba978-d14d-4d36-ba01-66da525b5242 None None] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 17:06:47.691202 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd1ba978-d14d-4d36-ba01-66da525b5242 None None] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 17:06:47.696478 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e878dee4-35ea-43ae-8ac2-656de0a7a115 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 17:06:47.697087 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e878dee4-35ea-43ae-8ac2-656de0a7a115 admin admin] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 17:06:47.697365 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e878dee4-35ea-43ae-8ac2-656de0a7a115 admin admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 17:06:47.697640 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e878dee4-35ea-43ae-8ac2-656de0a7a115 admin admin] PATH_INFO: `/v3/domains` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 17:06:47.698367 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e878dee4-35ea-43ae-8ac2-656de0a7a115 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87057) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 17:06:47.698671 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e878dee4-35ea-43ae-8ac2-656de0a7a115 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 17:06:47.700281 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e878dee4-35ea-43ae-8ac2-656de0a7a115 admin admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 17:06:47.712444 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 1263/2538] 10.222.0.20 () {58 vars in 1153 bytes} [Mon Aug 1 17:06:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 17:06:47.716530 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d66165a0-dd3f-4482-a198-38af8db1f288 None None] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 17:06:47.725666 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d66165a0-dd3f-4482-a198-38af8db1f288 None None] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 17:06:47.730293 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d9d708-8f92-4524-9e82-76421ec36f14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 17:06:47.730885 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d9d708-8f92-4524-9e82-76421ec36f14 admin admin] REQUEST_METHOD: `POST` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 17:06:47.731161 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d9d708-8f92-4524-9e82-76421ec36f14 admin admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 17:06:47.731432 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d9d708-8f92-4524-9e82-76421ec36f14 admin admin] PATH_INFO: `/v3/projects` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 17:06:47.732096 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8d9d708-8f92-4524-9e82-76421ec36f14 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 17:06:47.733968 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8d9d708-8f92-4524-9e82-76421ec36f14 admin admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 17:06:47.739541 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG dbcounter [-] [87058] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=87058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 17:06:47.815748 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 1276/2539] 10.222.0.20 () {60 vars in 1153 bytes} [Mon Aug 1 17:06:47 2022] POST /identity/v3/projects => generated 355 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 17:06:47.820473 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e878dee4-35ea-43ae-8ac2-656de0a7a115 admin admin] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 17:06:47.830497 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e878dee4-35ea-43ae-8ac2-656de0a7a115 admin admin] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 17:06:47.887660 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0934b0b0-ffe7-4e4f-93ea-84a618f48bf7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 17:06:47.888278 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0934b0b0-ffe7-4e4f-93ea-84a618f48bf7 admin admin] REQUEST_METHOD: `POST` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 17:06:47.888557 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0934b0b0-ffe7-4e4f-93ea-84a618f48bf7 admin admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 17:06:47.888832 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0934b0b0-ffe7-4e4f-93ea-84a618f48bf7 admin admin] PATH_INFO: `/v3/users` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 17:06:47.889489 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0934b0b0-ffe7-4e4f-93ea-84a618f48bf7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 17:06:47.891452 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0934b0b0-ffe7-4e4f-93ea-84a618f48bf7 admin admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 17:06:47.951432 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 1264/2540] 10.222.0.20 () {60 vars in 1144 bytes} [Mon Aug 1 17:06:47 2022] POST /identity/v3/users => generated 278 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 17:06:47.960975 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d9d708-8f92-4524-9e82-76421ec36f14 admin admin] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 17:06:47.971607 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d9d708-8f92-4524-9e82-76421ec36f14 admin admin] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 17:06:47.976306 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba595eb-4593-42ff-8ea4-93ecf55d4ce6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 17:06:47.976904 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba595eb-4593-42ff-8ea4-93ecf55d4ce6 admin admin] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 17:06:47.977176 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba595eb-4593-42ff-8ea4-93ecf55d4ce6 admin admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 17:06:47.977442 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba595eb-4593-42ff-8ea4-93ecf55d4ce6 admin admin] PATH_INFO: `/v3/roles` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 17:06:47.978234 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ba595eb-4593-42ff-8ea4-93ecf55d4ce6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 17:06:47.979805 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ba595eb-4593-42ff-8ea4-93ecf55d4ce6 admin admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 17:06:47.990544 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 1277/2541] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Aug 1 17:06:47 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 17:06:47.995705 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0934b0b0-ffe7-4e4f-93ea-84a618f48bf7 admin admin] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 17:06:48.006667 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0934b0b0-ffe7-4e4f-93ea-84a618f48bf7 admin admin] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 17:06:48.011781 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-963ede4c-a194-4e75-8ad1-024aba4413c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 17:06:48.012434 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-963ede4c-a194-4e75-8ad1-024aba4413c9 admin admin] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 17:06:48.012781 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-963ede4c-a194-4e75-8ad1-024aba4413c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 17:06:48.013112 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-963ede4c-a194-4e75-8ad1-024aba4413c9 admin admin] PATH_INFO: `/v3/roles` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 17:06:48.013956 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-963ede4c-a194-4e75-8ad1-024aba4413c9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 17:06:48.015647 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-963ede4c-a194-4e75-8ad1-024aba4413c9 admin admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 17:06:48.026424 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 1265/2542] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Aug 1 17:06:47 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 17:06:48.030398 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba595eb-4593-42ff-8ea4-93ecf55d4ce6 admin admin] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 17:06:48.040689 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba595eb-4593-42ff-8ea4-93ecf55d4ce6 admin admin] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 17:06:48.045203 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a091ee56-d6f8-49bb-a4aa-168bd7a003da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 17:06:48.045846 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a091ee56-d6f8-49bb-a4aa-168bd7a003da admin admin] REQUEST_METHOD: `PUT` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 17:06:48.046158 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a091ee56-d6f8-49bb-a4aa-168bd7a003da admin admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 17:06:48.046429 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a091ee56-d6f8-49bb-a4aa-168bd7a003da admin admin] PATH_INFO: `/v3/projects/d671f9b1b857437dae478305c0d70886/users/a20fb41d233e41e19a3cbcefbc9a5f8c/roles/0999d3da63a545f18ff3252afdc90518` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 17:06:48.061545 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a091ee56-d6f8-49bb-a4aa-168bd7a003da admin admin] RBAC: Authorizing `identity:create_grant(project_id=d671f9b1b857437dae478305c0d70886, user_id=a20fb41d233e41e19a3cbcefbc9a5f8c, role_id=0999d3da63a545f18ff3252afdc90518)` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 17:06:48.064355 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a091ee56-d6f8-49bb-a4aa-168bd7a003da admin admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 17:06:48.146658 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 1278/2543] 10.222.0.20 () {60 vars in 1483 bytes} [Mon Aug 1 17:06:48 2022] PUT /identity/v3/projects/d671f9b1b857437dae478305c0d70886/users/a20fb41d233e41e19a3cbcefbc9a5f8c/roles/0999d3da63a545f18ff3252afdc90518 => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 17:06:48.156969 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e9c8a9-e89f-4c3c-81f1-7e0e1267296e None None] REQUEST_METHOD: `POST` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 17:06:48.157256 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e9c8a9-e89f-4c3c-81f1-7e0e1267296e None None] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 17:06:48.157537 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e9c8a9-e89f-4c3c-81f1-7e0e1267296e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 17:06:48.170926 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG dbcounter [-] [87057] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87057) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 17:06:48.185501 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.auth.core [None req-49e9c8a9-e89f-4c3c-81f1-7e0e1267296e None None] MFA Rules not processed for user `a20fb41d233e41e19a3cbcefbc9a5f8c`. Rule list: `[]` (Enabled: `True`). {{(pid=87057) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 17:06:48.186759 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.fernet_utils [None req-49e9c8a9-e89f-4c3c-81f1-7e0e1267296e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87057) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 17:06:48.252087 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 1266/2544] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 1 17:06:48 2022] POST /identity/v3/auth/tokens => generated 3222 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 17:06:48.256788 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a091ee56-d6f8-49bb-a4aa-168bd7a003da admin admin] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 17:06:48.316960 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a091ee56-d6f8-49bb-a4aa-168bd7a003da admin admin] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 17:06:48.322523 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e3f09e0-e6ce-491b-87ac-1806cdf37ca9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 17:06:48.323151 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e3f09e0-e6ce-491b-87ac-1806cdf37ca9 admin admin] REQUEST_METHOD: `POST` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 17:06:48.323426 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e3f09e0-e6ce-491b-87ac-1806cdf37ca9 admin admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 17:06:48.323692 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e3f09e0-e6ce-491b-87ac-1806cdf37ca9 admin admin] PATH_INFO: `/v3/projects` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 17:06:48.324351 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e3f09e0-e6ce-491b-87ac-1806cdf37ca9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 17:06:48.326267 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e3f09e0-e6ce-491b-87ac-1806cdf37ca9 admin admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 17:06:48.415605 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 1279/2545] 10.222.0.20 () {60 vars in 1153 bytes} [Mon Aug 1 17:06:48 2022] POST /identity/v3/projects => generated 357 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 17:06:48.419628 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e9c8a9-e89f-4c3c-81f1-7e0e1267296e None None] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 17:06:48.430846 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e9c8a9-e89f-4c3c-81f1-7e0e1267296e None None] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 17:06:48.486950 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c1ef9a1-f814-49f8-b66f-f2cb4aed1cd6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 17:06:48.487578 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1ef9a1-f814-49f8-b66f-f2cb4aed1cd6 admin admin] REQUEST_METHOD: `POST` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 17:06:48.487859 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1ef9a1-f814-49f8-b66f-f2cb4aed1cd6 admin admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 17:06:48.488134 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1ef9a1-f814-49f8-b66f-f2cb4aed1cd6 admin admin] PATH_INFO: `/v3/users` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 17:06:48.489067 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c1ef9a1-f814-49f8-b66f-f2cb4aed1cd6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 17:06:48.491205 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c1ef9a1-f814-49f8-b66f-f2cb4aed1cd6 admin admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 17:06:48.568809 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 1267/2546] 10.222.0.20 () {60 vars in 1144 bytes} [Mon Aug 1 17:06:48 2022] POST /identity/v3/users => generated 285 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 17:06:48.573217 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e3f09e0-e6ce-491b-87ac-1806cdf37ca9 admin admin] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 17:06:48.584307 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e3f09e0-e6ce-491b-87ac-1806cdf37ca9 admin admin] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 17:06:48.589102 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f01236a-db6f-48e2-8ebf-b9bf332bb255 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 17:06:48.589706 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f01236a-db6f-48e2-8ebf-b9bf332bb255 admin admin] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 17:06:48.589987 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f01236a-db6f-48e2-8ebf-b9bf332bb255 admin admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 17:06:48.590318 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f01236a-db6f-48e2-8ebf-b9bf332bb255 admin admin] PATH_INFO: `/v3/roles` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 17:06:48.591008 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f01236a-db6f-48e2-8ebf-b9bf332bb255 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 17:06:48.592589 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f01236a-db6f-48e2-8ebf-b9bf332bb255 admin admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 17:06:48.604038 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 1280/2547] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Aug 1 17:06:48 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 17:06:48.608185 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c1ef9a1-f814-49f8-b66f-f2cb4aed1cd6 admin admin] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 17:06:48.619669 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c1ef9a1-f814-49f8-b66f-f2cb4aed1cd6 admin admin] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 17:06:48.624328 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bffa2a7-d3d2-4e92-91d6-38f4802f6978 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 17:06:48.625017 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bffa2a7-d3d2-4e92-91d6-38f4802f6978 admin admin] REQUEST_METHOD: `PUT` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 17:06:48.625296 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bffa2a7-d3d2-4e92-91d6-38f4802f6978 admin admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 17:06:48.625571 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bffa2a7-d3d2-4e92-91d6-38f4802f6978 admin admin] PATH_INFO: `/v3/projects/fbebb9ecbeb74914aa6dfe04aa28dcab/users/0c1674e38aac42fba3376338173756fc/roles/440bb943dd5b46fea038b903b6eaaf81` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 17:06:48.640961 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bffa2a7-d3d2-4e92-91d6-38f4802f6978 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fbebb9ecbeb74914aa6dfe04aa28dcab, user_id=0c1674e38aac42fba3376338173756fc, role_id=440bb943dd5b46fea038b903b6eaaf81)` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 17:06:48.643986 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bffa2a7-d3d2-4e92-91d6-38f4802f6978 admin admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 17:06:48.745978 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 1268/2548] 10.222.0.20 () {60 vars in 1483 bytes} [Mon Aug 1 17:06:48 2022] PUT /identity/v3/projects/fbebb9ecbeb74914aa6dfe04aa28dcab/users/0c1674e38aac42fba3376338173756fc/roles/440bb943dd5b46fea038b903b6eaaf81 => generated 0 bytes in 138 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 17:06:48.750015 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f01236a-db6f-48e2-8ebf-b9bf332bb255 admin admin] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 17:06:48.760005 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f01236a-db6f-48e2-8ebf-b9bf332bb255 admin admin] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 17:06:48.816112 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6383da10-9b26-44ad-8479-f9471bc11f43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 17:06:48.816715 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6383da10-9b26-44ad-8479-f9471bc11f43 admin admin] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 17:06:48.816990 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6383da10-9b26-44ad-8479-f9471bc11f43 admin admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 17:06:48.817266 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6383da10-9b26-44ad-8479-f9471bc11f43 admin admin] PATH_INFO: `/v3/roles` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 17:06:48.818624 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6383da10-9b26-44ad-8479-f9471bc11f43 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 17:06:48.820211 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6383da10-9b26-44ad-8479-f9471bc11f43 admin admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 17:06:48.829603 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 1281/2549] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Aug 1 17:06:48 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 17:06:48.833839 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bffa2a7-d3d2-4e92-91d6-38f4802f6978 admin admin] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 17:06:48.845067 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bffa2a7-d3d2-4e92-91d6-38f4802f6978 admin admin] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 17:06:48.850431 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56bf0f18-0936-432a-820f-6c2eb820fa99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 17:06:48.851085 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56bf0f18-0936-432a-820f-6c2eb820fa99 admin admin] REQUEST_METHOD: `PUT` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 17:06:48.851367 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56bf0f18-0936-432a-820f-6c2eb820fa99 admin admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 17:06:48.851636 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56bf0f18-0936-432a-820f-6c2eb820fa99 admin admin] PATH_INFO: `/v3/projects/fbebb9ecbeb74914aa6dfe04aa28dcab/users/0c1674e38aac42fba3376338173756fc/roles/440bb943dd5b46fea038b903b6eaaf81` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 17:06:48.853565 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56bf0f18-0936-432a-820f-6c2eb820fa99 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fbebb9ecbeb74914aa6dfe04aa28dcab, user_id=0c1674e38aac42fba3376338173756fc, role_id=440bb943dd5b46fea038b903b6eaaf81)` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 17:06:48.856214 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56bf0f18-0936-432a-820f-6c2eb820fa99 admin admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 17:06:48.867433 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 1269/2550] 10.222.0.20 () {60 vars in 1483 bytes} [Mon Aug 1 17:06:48 2022] PUT /identity/v3/projects/fbebb9ecbeb74914aa6dfe04aa28dcab/users/0c1674e38aac42fba3376338173756fc/roles/440bb943dd5b46fea038b903b6eaaf81 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 17:06:48.877733 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621a93dd-945e-44b6-83c1-7e409bc95208 None None] REQUEST_METHOD: `POST` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 17:06:48.878016 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621a93dd-945e-44b6-83c1-7e409bc95208 None None] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 17:06:48.878316 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621a93dd-945e-44b6-83c1-7e409bc95208 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 17:06:48.906822 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.auth.core [None req-621a93dd-945e-44b6-83c1-7e409bc95208 None None] MFA Rules not processed for user `0c1674e38aac42fba3376338173756fc`. Rule list: `[]` (Enabled: `True`). {{(pid=87058) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 17:06:48.908113 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.fernet_utils [None req-621a93dd-945e-44b6-83c1-7e409bc95208 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 17:06:48.917326 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG dbcounter [-] [87058] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 17:06:48.976633 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 1282/2551] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 1 17:06:48 2022] POST /identity/v3/auth/tokens => generated 3291 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 17:06:48.988285 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56bf0f18-0936-432a-820f-6c2eb820fa99 admin admin] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 17:06:48.990987 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.fernet_utils [None req-56bf0f18-0936-432a-820f-6c2eb820fa99 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87057) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 17:06:49.043877 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG dbcounter [-] [87057] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=87057) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 17:06:49.047528 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56bf0f18-0936-432a-820f-6c2eb820fa99 admin admin] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 17:06:49.051164 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea46dab-53df-496d-b48b-3092fb54aad9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25970b52ca15448fa1abb1f974d64038', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b992d6a65df4bf9b27c13192d57c5f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 17:06:49.051790 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea46dab-53df-496d-b48b-3092fb54aad9 service cinder] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 17:06:49.052922 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea46dab-53df-496d-b48b-3092fb54aad9 service cinder] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 17:06:49.053207 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea46dab-53df-496d-b48b-3092fb54aad9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 17:06:49.062489 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ea46dab-53df-496d-b48b-3092fb54aad9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 17:06:49.066647 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ea46dab-53df-496d-b48b-3092fb54aad9 service cinder] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 17:06:49.073376 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 1270/2552] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 1 17:06:48 2022] GET /identity/v3/auth/tokens => generated 3291 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 17:06:49.251044 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-621a93dd-945e-44b6-83c1-7e409bc95208 None None] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 17:06:49.260723 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-621a93dd-945e-44b6-83c1-7e409bc95208 None None] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 17:06:49.264199 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d3835e-08d4-4a3b-898f-6580fafa8892 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25970b52ca15448fa1abb1f974d64038', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b992d6a65df4bf9b27c13192d57c5f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 17:06:49.264798 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d3835e-08d4-4a3b-898f-6580fafa8892 service cinder] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 17:06:49.265074 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d3835e-08d4-4a3b-898f-6580fafa8892 service cinder] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 17:06:49.265343 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d3835e-08d4-4a3b-898f-6580fafa8892 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 17:06:49.321198 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14d3835e-08d4-4a3b-898f-6580fafa8892 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 17:06:49.322659 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14d3835e-08d4-4a3b-898f-6580fafa8892 service cinder] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 17:06:49.331432 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 1283/2553] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 1 17:06:49 2022] GET /identity/v3/auth/tokens => generated 3222 bytes in 81 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 17:06:59.156764 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG dbcounter [-] [87057] Writing DB stats keystone:SELECT=4 {{(pid=87057) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 17:06:59.327480 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG dbcounter [-] [87058] Writing DB stats keystone:SELECT=52 {{(pid=87058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 17:07:06.291962 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea46dab-53df-496d-b48b-3092fb54aad9 service cinder] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 17:07:06.354317 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea46dab-53df-496d-b48b-3092fb54aad9 service cinder] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 17:07:06.358817 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1b959aa-5c55-41c6-8cd8-676de1bef5f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 17:07:06.359453 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b959aa-5c55-41c6-8cd8-676de1bef5f2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 17:07:06.359735 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b959aa-5c55-41c6-8cd8-676de1bef5f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 17:07:06.360030 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b959aa-5c55-41c6-8cd8-676de1bef5f2 admin admin] PATH_INFO: `/v3/users/a20fb41d233e41e19a3cbcefbc9a5f8c` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 17:07:06.361193 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1b959aa-5c55-41c6-8cd8-676de1bef5f2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a20fb41d233e41e19a3cbcefbc9a5f8c)` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 17:07:06.363425 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1b959aa-5c55-41c6-8cd8-676de1bef5f2 admin admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 17:07:06.513596 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.notifications [None req-c1b959aa-5c55-41c6-8cd8-676de1bef5f2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a20fb41d233e41e19a3cbcefbc9a5f8c'} {{(pid=87057) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 17:07:06.514272 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.notifications [None req-c1b959aa-5c55-41c6-8cd8-676de1bef5f2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a20fb41d233e41e19a3cbcefbc9a5f8c'} {{(pid=87057) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 17:07:06.590793 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.notifications [None req-c1b959aa-5c55-41c6-8cd8-676de1bef5f2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a20fb41d233e41e19a3cbcefbc9a5f8c'} {{(pid=87057) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 17:07:06.591415 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.notifications [None req-c1b959aa-5c55-41c6-8cd8-676de1bef5f2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a20fb41d233e41e19a3cbcefbc9a5f8c'} {{(pid=87057) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 17:07:06.598567 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.notifications [None req-c1b959aa-5c55-41c6-8cd8-676de1bef5f2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a20fb41d233e41e19a3cbcefbc9a5f8c'} {{(pid=87057) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 17:07:06.616902 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 1271/2554] 10.222.0.20 () {58 vars in 1224 bytes} [Mon Aug 1 17:07:06 2022] DELETE /identity/v3/users/a20fb41d233e41e19a3cbcefbc9a5f8c => generated 0 bytes in 325 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 17:07:06.621356 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089e06fb-cf94-418c-8caf-d0b52ccb22a3 None None] REQUEST_METHOD: `POST` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 17:07:06.621851 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089e06fb-cf94-418c-8caf-d0b52ccb22a3 None None] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 17:07:06.621974 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089e06fb-cf94-418c-8caf-d0b52ccb22a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 17:07:06.641351 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.auth.core [None req-089e06fb-cf94-418c-8caf-d0b52ccb22a3 None None] MFA Rules not processed for user `1007f8e379b74e169b98fb7c3c0ba5b4`. Rule list: `[]` (Enabled: `True`). {{(pid=87058) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 17:07:06.642576 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.fernet_utils [None req-089e06fb-cf94-418c-8caf-d0b52ccb22a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 17:07:06.700559 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 1284/2555] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 1 17:07:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 17:07:06.710781 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1b959aa-5c55-41c6-8cd8-676de1bef5f2 admin admin] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 17:07:06.712229 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.fernet_utils [None req-c1b959aa-5c55-41c6-8cd8-676de1bef5f2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87057) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 17:07:06.769284 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1b959aa-5c55-41c6-8cd8-676de1bef5f2 admin admin] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 17:07:06.773530 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a9711c-882a-4ba9-820f-440fdf7b3f40 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d9bd8d649154c979be3744bfd016f95', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b992d6a65df4bf9b27c13192d57c5f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 17:07:06.774173 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a9711c-882a-4ba9-820f-440fdf7b3f40 service neutron] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 17:07:06.774456 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a9711c-882a-4ba9-820f-440fdf7b3f40 service neutron] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 17:07:06.774863 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a9711c-882a-4ba9-820f-440fdf7b3f40 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 17:07:06.783891 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9a9711c-882a-4ba9-820f-440fdf7b3f40 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 17:07:06.785345 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9a9711c-882a-4ba9-820f-440fdf7b3f40 service neutron] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 17:07:06.795610 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 1272/2556] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 1 17:07:06 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 17:07:07.203895 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-089e06fb-cf94-418c-8caf-d0b52ccb22a3 None None] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 17:07:07.205352 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.fernet_utils [None req-089e06fb-cf94-418c-8caf-d0b52ccb22a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 17:07:07.219671 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-089e06fb-cf94-418c-8caf-d0b52ccb22a3 None None] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 17:07:07.224772 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df9efd05-0190-4dc6-9321-eeb5575ca733 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 17:07:07.225117 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9efd05-0190-4dc6-9321-eeb5575ca733 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 17:07:07.225909 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9efd05-0190-4dc6-9321-eeb5575ca733 admin admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 17:07:07.226228 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9efd05-0190-4dc6-9321-eeb5575ca733 admin admin] PATH_INFO: `/v3/projects/d671f9b1b857437dae478305c0d70886` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 17:07:07.232301 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df9efd05-0190-4dc6-9321-eeb5575ca733 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d671f9b1b857437dae478305c0d70886)` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 17:07:07.232301 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df9efd05-0190-4dc6-9321-eeb5575ca733 admin admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 17:07:07.316270 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.notifications [None req-df9efd05-0190-4dc6-9321-eeb5575ca733 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd671f9b1b857437dae478305c0d70886', 'request_id': 'req-df9efd05-0190-4dc6-9321-eeb5575ca733'} {{(pid=87058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 17:07:07.322761 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.notifications [None req-df9efd05-0190-4dc6-9321-eeb5575ca733 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd671f9b1b857437dae478305c0d70886', 'request_id': 'req-df9efd05-0190-4dc6-9321-eeb5575ca733'} {{(pid=87058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 17:07:07.328333 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.notifications [None req-df9efd05-0190-4dc6-9321-eeb5575ca733 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd671f9b1b857437dae478305c0d70886', 'request_id': 'req-df9efd05-0190-4dc6-9321-eeb5575ca733'} {{(pid=87058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 17:07:07.407086 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.notifications [None req-df9efd05-0190-4dc6-9321-eeb5575ca733 admin admin] The token cache is being invalidate because project d671f9b1b857437dae478305c0d70886 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87058) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 17:07:07.407381 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.notifications [None req-df9efd05-0190-4dc6-9321-eeb5575ca733 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 17:07:07.408074 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.notifications [None req-df9efd05-0190-4dc6-9321-eeb5575ca733 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 17:07:07.409646 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 1285/2557] 10.222.0.20 () {58 vars in 1233 bytes} [Mon Aug 1 17:07:07 2022] DELETE /identity/v3/projects/d671f9b1b857437dae478305c0d70886 => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 17:07:07.414632 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a9711c-882a-4ba9-820f-440fdf7b3f40 service neutron] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 17:07:07.424466 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a9711c-882a-4ba9-820f-440fdf7b3f40 service neutron] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 17:07:07.442589 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG dbcounter [-] [87057] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87057) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 17:07:07.480939 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74569ff3-cbff-4b87-9ddc-390556b7b383 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 17:07:07.481510 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74569ff3-cbff-4b87-9ddc-390556b7b383 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 17:07:07.481791 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74569ff3-cbff-4b87-9ddc-390556b7b383 admin admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 17:07:07.482099 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74569ff3-cbff-4b87-9ddc-390556b7b383 admin admin] PATH_INFO: `/v3/users/0c1674e38aac42fba3376338173756fc` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 17:07:07.483274 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74569ff3-cbff-4b87-9ddc-390556b7b383 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0c1674e38aac42fba3376338173756fc)` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 17:07:07.485201 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74569ff3-cbff-4b87-9ddc-390556b7b383 admin admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 17:07:07.701203 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.notifications [None req-74569ff3-cbff-4b87-9ddc-390556b7b383 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0c1674e38aac42fba3376338173756fc'} {{(pid=87057) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 17:07:07.701842 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.notifications [None req-74569ff3-cbff-4b87-9ddc-390556b7b383 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0c1674e38aac42fba3376338173756fc'} {{(pid=87057) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 17:07:07.762039 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.notifications [None req-74569ff3-cbff-4b87-9ddc-390556b7b383 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0c1674e38aac42fba3376338173756fc'} {{(pid=87057) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 17:07:07.762675 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.notifications [None req-74569ff3-cbff-4b87-9ddc-390556b7b383 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0c1674e38aac42fba3376338173756fc'} {{(pid=87057) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 17:07:07.769685 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.notifications [None req-74569ff3-cbff-4b87-9ddc-390556b7b383 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0c1674e38aac42fba3376338173756fc'} {{(pid=87057) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 17:07:07.787251 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 1273/2558] 10.222.0.20 () {58 vars in 1224 bytes} [Mon Aug 1 17:07:07 2022] DELETE /identity/v3/users/0c1674e38aac42fba3376338173756fc => generated 0 bytes in 374 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 17:07:08.181722 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df9efd05-0190-4dc6-9321-eeb5575ca733 admin admin] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 17:07:08.183395 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.fernet_utils [None req-df9efd05-0190-4dc6-9321-eeb5575ca733 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 17:07:08.197683 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df9efd05-0190-4dc6-9321-eeb5575ca733 admin admin] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 17:07:08.255850 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b0cb9dd-b04f-46f6-a9fb-934633a17a12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 17:07:08.257148 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0cb9dd-b04f-46f6-a9fb-934633a17a12 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 17:07:08.257591 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0cb9dd-b04f-46f6-a9fb-934633a17a12 admin admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 17:07:08.257872 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0cb9dd-b04f-46f6-a9fb-934633a17a12 admin admin] PATH_INFO: `/v3/projects/fbebb9ecbeb74914aa6dfe04aa28dcab` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 17:07:08.259069 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b0cb9dd-b04f-46f6-a9fb-934633a17a12 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fbebb9ecbeb74914aa6dfe04aa28dcab)` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 17:07:08.260965 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b0cb9dd-b04f-46f6-a9fb-934633a17a12 admin admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 17:07:08.346489 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG dbcounter [-] [87058] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=87058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 17:07:08.362011 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.notifications [None req-8b0cb9dd-b04f-46f6-a9fb-934633a17a12 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fbebb9ecbeb74914aa6dfe04aa28dcab', 'request_id': 'req-8b0cb9dd-b04f-46f6-a9fb-934633a17a12'} {{(pid=87058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 17:07:08.370048 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.notifications [None req-8b0cb9dd-b04f-46f6-a9fb-934633a17a12 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fbebb9ecbeb74914aa6dfe04aa28dcab', 'request_id': 'req-8b0cb9dd-b04f-46f6-a9fb-934633a17a12'} {{(pid=87058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 17:07:08.375571 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.notifications [None req-8b0cb9dd-b04f-46f6-a9fb-934633a17a12 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fbebb9ecbeb74914aa6dfe04aa28dcab', 'request_id': 'req-8b0cb9dd-b04f-46f6-a9fb-934633a17a12'} {{(pid=87058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 17:07:08.441609 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.notifications [None req-8b0cb9dd-b04f-46f6-a9fb-934633a17a12 admin admin] The token cache is being invalidate because project fbebb9ecbeb74914aa6dfe04aa28dcab was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87058) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 17:07:08.441896 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.notifications [None req-8b0cb9dd-b04f-46f6-a9fb-934633a17a12 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 17:07:08.442543 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.notifications [None req-8b0cb9dd-b04f-46f6-a9fb-934633a17a12 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 17:07:08.444117 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 1286/2559] 10.222.0.20 () {58 vars in 1233 bytes} [Mon Aug 1 17:07:08 2022] DELETE /identity/v3/projects/fbebb9ecbeb74914aa6dfe04aa28dcab => generated 0 bytes in 263 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 17:07:08.449127 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63aeb801-b479-4a78-ab28-447609b67276 None None] REQUEST_METHOD: `POST` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 17:07:08.449640 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63aeb801-b479-4a78-ab28-447609b67276 None None] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 17:07:08.449748 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63aeb801-b479-4a78-ab28-447609b67276 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 17:07:08.468989 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.auth.core [None req-63aeb801-b479-4a78-ab28-447609b67276 None None] MFA Rules not processed for user `1007f8e379b74e169b98fb7c3c0ba5b4`. Rule list: `[]` (Enabled: `True`). {{(pid=87057) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 17:07:08.470240 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.fernet_utils [None req-63aeb801-b479-4a78-ab28-447609b67276 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87057) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 17:07:08.529016 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 1274/2560] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 1 17:07:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 17:07:08.536750 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e36a2f-1cb9-480e-8514-9f9f5bbd1db0 None None] REQUEST_METHOD: `POST` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 17:07:08.537032 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e36a2f-1cb9-480e-8514-9f9f5bbd1db0 None None] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 17:07:08.537302 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e36a2f-1cb9-480e-8514-9f9f5bbd1db0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 17:07:08.558221 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.auth.core [None req-46e36a2f-1cb9-480e-8514-9f9f5bbd1db0 None None] MFA Rules not processed for user `1007f8e379b74e169b98fb7c3c0ba5b4`. Rule list: `[]` (Enabled: `True`). {{(pid=87058) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 17:07:08.559420 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.fernet_utils [None req-46e36a2f-1cb9-480e-8514-9f9f5bbd1db0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 17:07:08.567351 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 1287/2561] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 1 17:07:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 17:07:08.571739 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63aeb801-b479-4a78-ab28-447609b67276 None None] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 17:07:08.581383 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63aeb801-b479-4a78-ab28-447609b67276 None None] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 17:07:08.586174 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-568f7cf3-9f61-49f8-bd92-0f923ddf9884 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 17:07:08.586784 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568f7cf3-9f61-49f8-bd92-0f923ddf9884 admin admin] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 17:07:08.587067 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568f7cf3-9f61-49f8-bd92-0f923ddf9884 admin admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 17:07:08.587344 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568f7cf3-9f61-49f8-bd92-0f923ddf9884 admin admin] PATH_INFO: `/v3/domains` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 17:07:08.588041 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-568f7cf3-9f61-49f8-bd92-0f923ddf9884 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87057) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 17:07:08.588347 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-568f7cf3-9f61-49f8-bd92-0f923ddf9884 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 17:07:08.589955 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-568f7cf3-9f61-49f8-bd92-0f923ddf9884 admin admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 17:07:08.601916 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 1275/2562] 10.222.0.20 () {58 vars in 1153 bytes} [Mon Aug 1 17:07:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 17:07:08.605919 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e36a2f-1cb9-480e-8514-9f9f5bbd1db0 None None] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 17:07:08.615294 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e36a2f-1cb9-480e-8514-9f9f5bbd1db0 None None] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 17:07:08.620443 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c672f516-7ba4-4d3c-bfff-63ccbaecc2c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 17:07:08.621035 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c672f516-7ba4-4d3c-bfff-63ccbaecc2c7 admin admin] REQUEST_METHOD: `POST` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 17:07:08.621313 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c672f516-7ba4-4d3c-bfff-63ccbaecc2c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 17:07:08.621585 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c672f516-7ba4-4d3c-bfff-63ccbaecc2c7 admin admin] PATH_INFO: `/v3/projects` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 17:07:08.622277 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c672f516-7ba4-4d3c-bfff-63ccbaecc2c7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 17:07:08.624156 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c672f516-7ba4-4d3c-bfff-63ccbaecc2c7 admin admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 17:07:08.712322 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 1288/2563] 10.222.0.20 () {60 vars in 1153 bytes} [Mon Aug 1 17:07:08 2022] POST /identity/v3/projects => generated 387 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 17:07:08.716488 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-568f7cf3-9f61-49f8-bd92-0f923ddf9884 admin admin] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 17:07:08.727245 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-568f7cf3-9f61-49f8-bd92-0f923ddf9884 admin admin] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 17:07:08.753619 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG dbcounter [-] [87057] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87057) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 17:07:08.785638 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f85b53-4dfa-45ba-88c5-d89d4b7acf41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 17:07:08.786310 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f85b53-4dfa-45ba-88c5-d89d4b7acf41 admin admin] REQUEST_METHOD: `POST` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 17:07:08.786597 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f85b53-4dfa-45ba-88c5-d89d4b7acf41 admin admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 17:07:08.786872 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f85b53-4dfa-45ba-88c5-d89d4b7acf41 admin admin] PATH_INFO: `/v3/users` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 17:07:08.787529 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1f85b53-4dfa-45ba-88c5-d89d4b7acf41 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 17:07:08.790135 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1f85b53-4dfa-45ba-88c5-d89d4b7acf41 admin admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 17:07:08.859539 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 1276/2564] 10.222.0.20 () {60 vars in 1144 bytes} [Mon Aug 1 17:07:08 2022] POST /identity/v3/users => generated 294 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 17:07:08.863913 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c672f516-7ba4-4d3c-bfff-63ccbaecc2c7 admin admin] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 17:07:08.874023 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c672f516-7ba4-4d3c-bfff-63ccbaecc2c7 admin admin] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 17:07:08.878544 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-571f7990-7c35-4115-991f-3f0021f2bcc9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 17:07:08.879137 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571f7990-7c35-4115-991f-3f0021f2bcc9 admin admin] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 17:07:08.879410 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571f7990-7c35-4115-991f-3f0021f2bcc9 admin admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 17:07:08.879677 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571f7990-7c35-4115-991f-3f0021f2bcc9 admin admin] PATH_INFO: `/v3/roles` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 17:07:08.880381 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-571f7990-7c35-4115-991f-3f0021f2bcc9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 17:07:08.881932 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-571f7990-7c35-4115-991f-3f0021f2bcc9 admin admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 17:07:08.891638 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 1289/2565] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Aug 1 17:07:08 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 17:07:08.895418 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f85b53-4dfa-45ba-88c5-d89d4b7acf41 admin admin] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 17:07:08.906324 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f85b53-4dfa-45ba-88c5-d89d4b7acf41 admin admin] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 17:07:08.911201 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2e5c9b3-201a-4407-a50a-f3da1eb8b9aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 17:07:08.911847 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e5c9b3-201a-4407-a50a-f3da1eb8b9aa admin admin] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 17:07:08.912124 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e5c9b3-201a-4407-a50a-f3da1eb8b9aa admin admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 17:07:08.912414 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e5c9b3-201a-4407-a50a-f3da1eb8b9aa admin admin] PATH_INFO: `/v3/roles` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 17:07:08.913755 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2e5c9b3-201a-4407-a50a-f3da1eb8b9aa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 17:07:08.915411 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2e5c9b3-201a-4407-a50a-f3da1eb8b9aa admin admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 17:07:08.925206 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 1277/2566] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Aug 1 17:07:08 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 01 17:07:08.929817 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-571f7990-7c35-4115-991f-3f0021f2bcc9 admin admin] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 17:07:08.939602 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-571f7990-7c35-4115-991f-3f0021f2bcc9 admin admin] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 17:07:08.944723 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a74d8d40-63b5-484c-bfb1-a463319dc07a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 17:07:08.945361 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74d8d40-63b5-484c-bfb1-a463319dc07a admin admin] REQUEST_METHOD: `PUT` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 17:07:08.945634 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74d8d40-63b5-484c-bfb1-a463319dc07a admin admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 17:07:08.945900 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74d8d40-63b5-484c-bfb1-a463319dc07a admin admin] PATH_INFO: `/v3/projects/307709da045b45769d6fbb086235ce33/users/67ff6961730049e2b1e8a905b75d14ae/roles/0999d3da63a545f18ff3252afdc90518` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 17:07:08.960075 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a74d8d40-63b5-484c-bfb1-a463319dc07a admin admin] RBAC: Authorizing `identity:create_grant(project_id=307709da045b45769d6fbb086235ce33, user_id=67ff6961730049e2b1e8a905b75d14ae, role_id=0999d3da63a545f18ff3252afdc90518)` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 17:07:08.963121 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a74d8d40-63b5-484c-bfb1-a463319dc07a admin admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 17:07:09.047999 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 1290/2567] 10.222.0.20 () {60 vars in 1483 bytes} [Mon Aug 1 17:07:08 2022] PUT /identity/v3/projects/307709da045b45769d6fbb086235ce33/users/67ff6961730049e2b1e8a905b75d14ae/roles/0999d3da63a545f18ff3252afdc90518 => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 17:07:09.058955 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08027a0-406f-41e5-b218-3f36e5d735e2 None None] REQUEST_METHOD: `POST` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 17:07:09.059243 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08027a0-406f-41e5-b218-3f36e5d735e2 None None] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 17:07:09.059521 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08027a0-406f-41e5-b218-3f36e5d735e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 17:07:09.087732 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.auth.core [None req-e08027a0-406f-41e5-b218-3f36e5d735e2 None None] MFA Rules not processed for user `67ff6961730049e2b1e8a905b75d14ae`. Rule list: `[]` (Enabled: `True`). {{(pid=87057) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 17:07:09.088954 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.fernet_utils [None req-e08027a0-406f-41e5-b218-3f36e5d735e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87057) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 17:07:09.152323 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 1278/2568] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 1 17:07:09 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 17:07:09.156971 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a74d8d40-63b5-484c-bfb1-a463319dc07a admin admin] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 17:07:09.218012 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a74d8d40-63b5-484c-bfb1-a463319dc07a admin admin] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 17:07:09.222702 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-377e7d57-acf9-44c2-b127-9927804de36a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 17:07:09.223306 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377e7d57-acf9-44c2-b127-9927804de36a admin admin] REQUEST_METHOD: `POST` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 17:07:09.223585 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377e7d57-acf9-44c2-b127-9927804de36a admin admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 17:07:09.223855 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377e7d57-acf9-44c2-b127-9927804de36a admin admin] PATH_INFO: `/v3/projects` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 17:07:09.224517 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-377e7d57-acf9-44c2-b127-9927804de36a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 17:07:09.227070 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-377e7d57-acf9-44c2-b127-9927804de36a admin admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 17:07:09.312056 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 1291/2569] 10.222.0.20 () {60 vars in 1153 bytes} [Mon Aug 1 17:07:09 2022] POST /identity/v3/projects => generated 383 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 17:07:09.316137 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e08027a0-406f-41e5-b218-3f36e5d735e2 None None] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 17:07:09.327173 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e08027a0-406f-41e5-b218-3f36e5d735e2 None None] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 17:07:09.383739 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc55176-8062-4004-b4aa-1bb0b6f38a0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 17:07:09.384358 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc55176-8062-4004-b4aa-1bb0b6f38a0a admin admin] REQUEST_METHOD: `POST` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 17:07:09.384642 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc55176-8062-4004-b4aa-1bb0b6f38a0a admin admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 17:07:09.384920 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc55176-8062-4004-b4aa-1bb0b6f38a0a admin admin] PATH_INFO: `/v3/users` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 17:07:09.385593 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bc55176-8062-4004-b4aa-1bb0b6f38a0a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 17:07:09.387562 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bc55176-8062-4004-b4aa-1bb0b6f38a0a admin admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 17:07:09.495757 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 1279/2570] 10.222.0.20 () {60 vars in 1144 bytes} [Mon Aug 1 17:07:09 2022] POST /identity/v3/users => generated 298 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 17:07:09.499746 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-377e7d57-acf9-44c2-b127-9927804de36a admin admin] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 17:07:09.510466 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-377e7d57-acf9-44c2-b127-9927804de36a admin admin] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 17:07:09.515068 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f14f36-ea2a-432f-8f2c-027c0043492b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 17:07:09.515701 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f14f36-ea2a-432f-8f2c-027c0043492b admin admin] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 17:07:09.515976 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f14f36-ea2a-432f-8f2c-027c0043492b admin admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 17:07:09.516243 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f14f36-ea2a-432f-8f2c-027c0043492b admin admin] PATH_INFO: `/v3/roles` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 17:07:09.516928 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5f14f36-ea2a-432f-8f2c-027c0043492b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 17:07:09.518562 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5f14f36-ea2a-432f-8f2c-027c0043492b admin admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 17:07:09.529506 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 1292/2571] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Aug 1 17:07:09 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 17:07:09.533596 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc55176-8062-4004-b4aa-1bb0b6f38a0a admin admin] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 17:07:09.544168 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc55176-8062-4004-b4aa-1bb0b6f38a0a admin admin] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 17:07:09.548819 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d17eb067-c07f-4a3c-aafc-f29234d6b663 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 17:07:09.549476 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17eb067-c07f-4a3c-aafc-f29234d6b663 admin admin] REQUEST_METHOD: `PUT` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 17:07:09.549758 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17eb067-c07f-4a3c-aafc-f29234d6b663 admin admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 17:07:09.550057 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17eb067-c07f-4a3c-aafc-f29234d6b663 admin admin] PATH_INFO: `/v3/projects/f4606f3ad6bb4b9fa42305644dd956ee/users/7df8393559f44efd9596535a35c53e4a/roles/440bb943dd5b46fea038b903b6eaaf81` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 17:07:09.564982 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d17eb067-c07f-4a3c-aafc-f29234d6b663 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f4606f3ad6bb4b9fa42305644dd956ee, user_id=7df8393559f44efd9596535a35c53e4a, role_id=440bb943dd5b46fea038b903b6eaaf81)` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 17:07:09.567758 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d17eb067-c07f-4a3c-aafc-f29234d6b663 admin admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 17:07:09.630827 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 1280/2572] 10.222.0.20 () {60 vars in 1483 bytes} [Mon Aug 1 17:07:09 2022] PUT /identity/v3/projects/f4606f3ad6bb4b9fa42305644dd956ee/users/7df8393559f44efd9596535a35c53e4a/roles/440bb943dd5b46fea038b903b6eaaf81 => generated 0 bytes in 98 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 17:07:09.634829 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f14f36-ea2a-432f-8f2c-027c0043492b admin admin] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 17:07:09.644708 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f14f36-ea2a-432f-8f2c-027c0043492b admin admin] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 17:07:09.665053 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG dbcounter [-] [87058] Writing DB stats keystone:DELETE=6,keystone:SELECT=90,keystone:INSERT=4 {{(pid=87058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 17:07:09.701994 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-879b9ecb-f0c8-451f-8e1c-1c644515476a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 17:07:09.702568 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879b9ecb-f0c8-451f-8e1c-1c644515476a admin admin] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 17:07:09.702857 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879b9ecb-f0c8-451f-8e1c-1c644515476a admin admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 17:07:09.703132 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879b9ecb-f0c8-451f-8e1c-1c644515476a admin admin] PATH_INFO: `/v3/roles` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 17:07:09.703962 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-879b9ecb-f0c8-451f-8e1c-1c644515476a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 17:07:09.705553 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-879b9ecb-f0c8-451f-8e1c-1c644515476a admin admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 17:07:09.716226 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 1293/2573] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Aug 1 17:07:09 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 17:07:09.720378 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d17eb067-c07f-4a3c-aafc-f29234d6b663 admin admin] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 17:07:09.725966 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG dbcounter [-] [87057] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=87057) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 17:07:09.731267 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d17eb067-c07f-4a3c-aafc-f29234d6b663 admin admin] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 17:07:09.735912 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79691bc2-fb9f-49ab-ab60-a6e178d27bdc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 17:07:09.736570 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79691bc2-fb9f-49ab-ab60-a6e178d27bdc admin admin] REQUEST_METHOD: `PUT` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 17:07:09.736853 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79691bc2-fb9f-49ab-ab60-a6e178d27bdc admin admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 17:07:09.737130 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79691bc2-fb9f-49ab-ab60-a6e178d27bdc admin admin] PATH_INFO: `/v3/projects/f4606f3ad6bb4b9fa42305644dd956ee/users/7df8393559f44efd9596535a35c53e4a/roles/440bb943dd5b46fea038b903b6eaaf81` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 17:07:09.739657 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79691bc2-fb9f-49ab-ab60-a6e178d27bdc admin admin] RBAC: Authorizing `identity:create_grant(project_id=f4606f3ad6bb4b9fa42305644dd956ee, user_id=7df8393559f44efd9596535a35c53e4a, role_id=440bb943dd5b46fea038b903b6eaaf81)` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 17:07:09.743168 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79691bc2-fb9f-49ab-ab60-a6e178d27bdc admin admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 17:07:09.753511 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 1281/2574] 10.222.0.20 () {60 vars in 1483 bytes} [Mon Aug 1 17:07:09 2022] PUT /identity/v3/projects/f4606f3ad6bb4b9fa42305644dd956ee/users/7df8393559f44efd9596535a35c53e4a/roles/440bb943dd5b46fea038b903b6eaaf81 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 17:07:09.764732 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e9e5be-9271-4dcb-87ed-379b09018ac5 None None] REQUEST_METHOD: `POST` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 17:07:09.765017 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e9e5be-9271-4dcb-87ed-379b09018ac5 None None] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 17:07:09.765294 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e9e5be-9271-4dcb-87ed-379b09018ac5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 17:07:09.793457 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.auth.core [None req-54e9e5be-9271-4dcb-87ed-379b09018ac5 None None] MFA Rules not processed for user `7df8393559f44efd9596535a35c53e4a`. Rule list: `[]` (Enabled: `True`). {{(pid=87058) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 17:07:09.794739 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.fernet_utils [None req-54e9e5be-9271-4dcb-87ed-379b09018ac5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 17:07:09.861789 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 1294/2575] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 1 17:07:09 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 17:07:09.873101 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79691bc2-fb9f-49ab-ab60-a6e178d27bdc admin admin] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 17:07:09.874786 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.fernet_utils [None req-79691bc2-fb9f-49ab-ab60-a6e178d27bdc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87057) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 17:07:09.934408 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79691bc2-fb9f-49ab-ab60-a6e178d27bdc admin admin] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 17:07:09.938085 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c42c697-f913-475f-ae95-d0181b4e775e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25970b52ca15448fa1abb1f974d64038', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b992d6a65df4bf9b27c13192d57c5f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 17:07:09.938700 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c42c697-f913-475f-ae95-d0181b4e775e service cinder] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 17:07:09.938979 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c42c697-f913-475f-ae95-d0181b4e775e service cinder] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 17:07:09.939254 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c42c697-f913-475f-ae95-d0181b4e775e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 17:07:09.948815 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c42c697-f913-475f-ae95-d0181b4e775e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 17:07:09.950282 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c42c697-f913-475f-ae95-d0181b4e775e service cinder] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 17:07:09.960671 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 1282/2576] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 1 17:07:09 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 17:07:09.988873 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54e9e5be-9271-4dcb-87ed-379b09018ac5 None None] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 17:07:10.049924 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54e9e5be-9271-4dcb-87ed-379b09018ac5 None None] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 17:07:10.054790 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d225de3-a713-4121-a77e-e84764940e74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 17:07:10.055459 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d225de3-a713-4121-a77e-e84764940e74 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 17:07:10.055738 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d225de3-a713-4121-a77e-e84764940e74 admin admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 17:07:10.056011 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d225de3-a713-4121-a77e-e84764940e74 admin admin] PATH_INFO: `/v3/users/67ff6961730049e2b1e8a905b75d14ae` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 17:07:10.057190 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d225de3-a713-4121-a77e-e84764940e74 admin admin] RBAC: Authorizing `identity:delete_user(user_id=67ff6961730049e2b1e8a905b75d14ae)` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 17:07:10.059153 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d225de3-a713-4121-a77e-e84764940e74 admin admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 17:07:10.257230 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.notifications [None req-0d225de3-a713-4121-a77e-e84764940e74 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '67ff6961730049e2b1e8a905b75d14ae'} {{(pid=87058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 17:07:10.257913 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.notifications [None req-0d225de3-a713-4121-a77e-e84764940e74 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '67ff6961730049e2b1e8a905b75d14ae'} {{(pid=87058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 17:07:10.258573 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.notifications [None req-0d225de3-a713-4121-a77e-e84764940e74 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '67ff6961730049e2b1e8a905b75d14ae'} {{(pid=87058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 17:07:10.259904 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG dbcounter [-] [87058] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=87058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 17:07:10.271127 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.notifications [None req-0d225de3-a713-4121-a77e-e84764940e74 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '67ff6961730049e2b1e8a905b75d14ae'} {{(pid=87058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 17:07:10.278892 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.notifications [None req-0d225de3-a713-4121-a77e-e84764940e74 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '67ff6961730049e2b1e8a905b75d14ae'} {{(pid=87058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 17:07:10.357992 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 1295/2577] 10.222.0.20 () {58 vars in 1224 bytes} [Mon Aug 1 17:07:09 2022] DELETE /identity/v3/users/67ff6961730049e2b1e8a905b75d14ae => generated 0 bytes in 370 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 17:07:10.363491 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a3ebc4-df3e-42b4-ac07-95a2aab44d04 None None] REQUEST_METHOD: `POST` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 17:07:10.363776 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a3ebc4-df3e-42b4-ac07-95a2aab44d04 None None] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 17:07:10.364049 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a3ebc4-df3e-42b4-ac07-95a2aab44d04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 17:07:10.385140 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.auth.core [None req-99a3ebc4-df3e-42b4-ac07-95a2aab44d04 None None] MFA Rules not processed for user `1007f8e379b74e169b98fb7c3c0ba5b4`. Rule list: `[]` (Enabled: `True`). {{(pid=87057) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 17:07:10.386429 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.fernet_utils [None req-99a3ebc4-df3e-42b4-ac07-95a2aab44d04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87057) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 17:07:10.452756 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 1283/2578] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 1 17:07:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 17:07:10.463663 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d225de3-a713-4121-a77e-e84764940e74 admin admin] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 17:07:10.465196 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.fernet_utils [None req-0d225de3-a713-4121-a77e-e84764940e74 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 17:07:10.528296 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d225de3-a713-4121-a77e-e84764940e74 admin admin] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 17:07:10.531905 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c7754e-e159-4db9-b645-bab9fb8e68fa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d9bd8d649154c979be3744bfd016f95', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b992d6a65df4bf9b27c13192d57c5f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 17:07:10.532523 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c7754e-e159-4db9-b645-bab9fb8e68fa service neutron] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 17:07:10.532810 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c7754e-e159-4db9-b645-bab9fb8e68fa service neutron] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 17:07:10.533084 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c7754e-e159-4db9-b645-bab9fb8e68fa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 17:07:10.543175 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0c7754e-e159-4db9-b645-bab9fb8e68fa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 17:07:10.545679 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0c7754e-e159-4db9-b645-bab9fb8e68fa service neutron] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 17:07:10.554483 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 1296/2579] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 1 17:07:10 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 17:07:11.012051 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a3ebc4-df3e-42b4-ac07-95a2aab44d04 None None] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 17:07:11.012051 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.fernet_utils [None req-99a3ebc4-df3e-42b4-ac07-95a2aab44d04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87057) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 17:07:11.018618 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a3ebc4-df3e-42b4-ac07-95a2aab44d04 None None] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 17:07:11.023689 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c6472c-e1d6-472a-9ea9-1fbbe5613794 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 17:07:11.024386 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c6472c-e1d6-472a-9ea9-1fbbe5613794 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 17:07:11.024662 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c6472c-e1d6-472a-9ea9-1fbbe5613794 admin admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 17:07:11.024990 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c6472c-e1d6-472a-9ea9-1fbbe5613794 admin admin] PATH_INFO: `/v3/projects/307709da045b45769d6fbb086235ce33` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 17:07:11.026280 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39c6472c-e1d6-472a-9ea9-1fbbe5613794 admin admin] RBAC: Authorizing `identity:delete_project(project_id=307709da045b45769d6fbb086235ce33)` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 17:07:11.028281 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39c6472c-e1d6-472a-9ea9-1fbbe5613794 admin admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 17:07:11.134487 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.notifications [None req-39c6472c-e1d6-472a-9ea9-1fbbe5613794 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '307709da045b45769d6fbb086235ce33', 'request_id': 'req-39c6472c-e1d6-472a-9ea9-1fbbe5613794'} {{(pid=87057) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 17:07:11.140952 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.notifications [None req-39c6472c-e1d6-472a-9ea9-1fbbe5613794 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '307709da045b45769d6fbb086235ce33', 'request_id': 'req-39c6472c-e1d6-472a-9ea9-1fbbe5613794'} {{(pid=87057) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 17:07:11.146769 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.notifications [None req-39c6472c-e1d6-472a-9ea9-1fbbe5613794 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '307709da045b45769d6fbb086235ce33', 'request_id': 'req-39c6472c-e1d6-472a-9ea9-1fbbe5613794'} {{(pid=87057) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 17:07:11.231295 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.notifications [None req-39c6472c-e1d6-472a-9ea9-1fbbe5613794 admin admin] The token cache is being invalidate because project 307709da045b45769d6fbb086235ce33 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87057) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 17:07:11.231613 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.notifications [None req-39c6472c-e1d6-472a-9ea9-1fbbe5613794 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87057) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 17:07:11.232242 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.notifications [None req-39c6472c-e1d6-472a-9ea9-1fbbe5613794 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87057) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 17:07:11.233762 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 1284/2580] 10.222.0.20 () {58 vars in 1233 bytes} [Mon Aug 1 17:07:11 2022] DELETE /identity/v3/projects/307709da045b45769d6fbb086235ce33 => generated 0 bytes in 230 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 17:07:11.237766 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c7754e-e159-4db9-b645-bab9fb8e68fa service neutron] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 17:07:11.248178 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c7754e-e159-4db9-b645-bab9fb8e68fa service neutron] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 17:07:11.304365 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b3beeb-ee2a-475f-8eaf-78d2de080cc1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 17:07:11.304998 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b3beeb-ee2a-475f-8eaf-78d2de080cc1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 17:07:11.305179 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b3beeb-ee2a-475f-8eaf-78d2de080cc1 admin admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 17:07:11.305446 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b3beeb-ee2a-475f-8eaf-78d2de080cc1 admin admin] PATH_INFO: `/v3/users/7df8393559f44efd9596535a35c53e4a` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 17:07:11.306623 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63b3beeb-ee2a-475f-8eaf-78d2de080cc1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7df8393559f44efd9596535a35c53e4a)` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 17:07:11.308518 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63b3beeb-ee2a-475f-8eaf-78d2de080cc1 admin admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 17:07:11.437935 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.notifications [None req-63b3beeb-ee2a-475f-8eaf-78d2de080cc1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7df8393559f44efd9596535a35c53e4a'} {{(pid=87058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 17:07:11.438621 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.notifications [None req-63b3beeb-ee2a-475f-8eaf-78d2de080cc1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7df8393559f44efd9596535a35c53e4a'} {{(pid=87058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 17:07:11.439305 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.notifications [None req-63b3beeb-ee2a-475f-8eaf-78d2de080cc1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7df8393559f44efd9596535a35c53e4a'} {{(pid=87058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 17:07:11.452349 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.notifications [None req-63b3beeb-ee2a-475f-8eaf-78d2de080cc1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7df8393559f44efd9596535a35c53e4a'} {{(pid=87058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 17:07:11.456878 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG dbcounter [-] [87058] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=87058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 17:07:11.459878 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.notifications [None req-63b3beeb-ee2a-475f-8eaf-78d2de080cc1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7df8393559f44efd9596535a35c53e4a'} {{(pid=87058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 17:07:11.528107 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 1297/2581] 10.222.0.20 () {58 vars in 1224 bytes} [Mon Aug 1 17:07:11 2022] DELETE /identity/v3/users/7df8393559f44efd9596535a35c53e4a => generated 0 bytes in 291 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 17:07:11.930636 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c6472c-e1d6-472a-9ea9-1fbbe5613794 admin admin] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 17:07:11.932190 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.fernet_utils [None req-39c6472c-e1d6-472a-9ea9-1fbbe5613794 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87057) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 17:07:11.946527 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c6472c-e1d6-472a-9ea9-1fbbe5613794 admin admin] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 17:07:11.959425 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG dbcounter [-] [87057] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=87057) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 17:07:12.006727 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21646084-4af0-4c08-9744-66a1f8c0584b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 17:07:12.008236 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21646084-4af0-4c08-9744-66a1f8c0584b admin admin] REQUEST_METHOD: `DELETE` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 17:07:12.008497 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21646084-4af0-4c08-9744-66a1f8c0584b admin admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 17:07:12.008773 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21646084-4af0-4c08-9744-66a1f8c0584b admin admin] PATH_INFO: `/v3/projects/f4606f3ad6bb4b9fa42305644dd956ee` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 17:07:12.009958 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21646084-4af0-4c08-9744-66a1f8c0584b admin admin] RBAC: Authorizing `identity:delete_project(project_id=f4606f3ad6bb4b9fa42305644dd956ee)` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 17:07:12.012075 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21646084-4af0-4c08-9744-66a1f8c0584b admin admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 17:07:12.152809 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.notifications [None req-21646084-4af0-4c08-9744-66a1f8c0584b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f4606f3ad6bb4b9fa42305644dd956ee', 'request_id': 'req-21646084-4af0-4c08-9744-66a1f8c0584b'} {{(pid=87057) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 17:07:12.159421 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.notifications [None req-21646084-4af0-4c08-9744-66a1f8c0584b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f4606f3ad6bb4b9fa42305644dd956ee', 'request_id': 'req-21646084-4af0-4c08-9744-66a1f8c0584b'} {{(pid=87057) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 17:07:12.165885 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.notifications [None req-21646084-4af0-4c08-9744-66a1f8c0584b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f4606f3ad6bb4b9fa42305644dd956ee', 'request_id': 'req-21646084-4af0-4c08-9744-66a1f8c0584b'} {{(pid=87057) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 17:07:12.253675 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.notifications [None req-21646084-4af0-4c08-9744-66a1f8c0584b admin admin] The token cache is being invalidate because project f4606f3ad6bb4b9fa42305644dd956ee was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87057) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 17:07:12.254084 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.notifications [None req-21646084-4af0-4c08-9744-66a1f8c0584b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87057) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 17:07:12.254772 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.notifications [None req-21646084-4af0-4c08-9744-66a1f8c0584b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87057) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 17:07:12.256377 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 1285/2582] 10.222.0.20 () {58 vars in 1233 bytes} [Mon Aug 1 17:07:11 2022] DELETE /identity/v3/projects/f4606f3ad6bb4b9fa42305644dd956ee => generated 0 bytes in 326 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 17:07:12.261282 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f475c21-b99d-4eed-99c5-b1fceb917748 None None] REQUEST_METHOD: `POST` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 17:07:12.261584 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f475c21-b99d-4eed-99c5-b1fceb917748 None None] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 17:07:12.261856 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f475c21-b99d-4eed-99c5-b1fceb917748 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 17:07:12.281924 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.auth.core [None req-4f475c21-b99d-4eed-99c5-b1fceb917748 None None] MFA Rules not processed for user `1007f8e379b74e169b98fb7c3c0ba5b4`. Rule list: `[]` (Enabled: `True`). {{(pid=87058) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 17:07:12.283163 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.fernet_utils [None req-4f475c21-b99d-4eed-99c5-b1fceb917748 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 17:07:12.342385 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 1298/2583] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 1 17:07:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 17:07:12.350313 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e2c85b-658c-4046-a987-01a5c3808494 None None] REQUEST_METHOD: `POST` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 17:07:12.351130 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e2c85b-658c-4046-a987-01a5c3808494 None None] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 17:07:12.351389 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e2c85b-658c-4046-a987-01a5c3808494 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 17:07:12.372827 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.auth.core [None req-32e2c85b-658c-4046-a987-01a5c3808494 None None] MFA Rules not processed for user `1007f8e379b74e169b98fb7c3c0ba5b4`. Rule list: `[]` (Enabled: `True`). {{(pid=87057) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 17:07:12.374077 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.fernet_utils [None req-32e2c85b-658c-4046-a987-01a5c3808494 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87057) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 17:07:12.382004 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 1286/2584] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 1 17:07:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 17:07:12.386368 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f475c21-b99d-4eed-99c5-b1fceb917748 None None] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 17:07:12.398365 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f475c21-b99d-4eed-99c5-b1fceb917748 None None] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 17:07:12.403076 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecedf1a3-2984-4d4d-94c2-2d5679fe18e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 17:07:12.403695 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecedf1a3-2984-4d4d-94c2-2d5679fe18e7 admin admin] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 17:07:12.403995 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecedf1a3-2984-4d4d-94c2-2d5679fe18e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 17:07:12.404273 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecedf1a3-2984-4d4d-94c2-2d5679fe18e7 admin admin] PATH_INFO: `/v3/domains` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 17:07:12.404976 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecedf1a3-2984-4d4d-94c2-2d5679fe18e7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87058) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 17:07:12.405280 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecedf1a3-2984-4d4d-94c2-2d5679fe18e7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 17:07:12.406937 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecedf1a3-2984-4d4d-94c2-2d5679fe18e7 admin admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 17:07:12.419485 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 1299/2585] 10.222.0.20 () {58 vars in 1153 bytes} [Mon Aug 1 17:07:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 17:07:12.423823 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32e2c85b-658c-4046-a987-01a5c3808494 None None] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 17:07:12.434184 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32e2c85b-658c-4046-a987-01a5c3808494 None None] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 17:07:12.438860 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d13c9aa-5a2c-4b96-95a6-2d0a1df60a19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 17:07:12.439473 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d13c9aa-5a2c-4b96-95a6-2d0a1df60a19 admin admin] REQUEST_METHOD: `POST` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 17:07:12.439752 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d13c9aa-5a2c-4b96-95a6-2d0a1df60a19 admin admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 17:07:12.440023 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d13c9aa-5a2c-4b96-95a6-2d0a1df60a19 admin admin] PATH_INFO: `/v3/projects` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 17:07:12.440687 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d13c9aa-5a2c-4b96-95a6-2d0a1df60a19 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 17:07:12.442628 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d13c9aa-5a2c-4b96-95a6-2d0a1df60a19 admin admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 17:07:12.530519 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 1287/2586] 10.222.0.20 () {60 vars in 1153 bytes} [Mon Aug 1 17:07:12 2022] POST /identity/v3/projects => generated 387 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 17:07:12.534826 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecedf1a3-2984-4d4d-94c2-2d5679fe18e7 admin admin] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 17:07:12.545473 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecedf1a3-2984-4d4d-94c2-2d5679fe18e7 admin admin] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 17:07:12.601617 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b86585-8ccf-4209-a888-95e473b0a1bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 17:07:12.602265 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b86585-8ccf-4209-a888-95e473b0a1bb admin admin] REQUEST_METHOD: `POST` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 17:07:12.602542 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b86585-8ccf-4209-a888-95e473b0a1bb admin admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 17:07:12.602810 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b86585-8ccf-4209-a888-95e473b0a1bb admin admin] PATH_INFO: `/v3/users` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 17:07:12.603478 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5b86585-8ccf-4209-a888-95e473b0a1bb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 17:07:12.605389 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5b86585-8ccf-4209-a888-95e473b0a1bb admin admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 17:07:12.697383 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 1300/2587] 10.222.0.20 () {60 vars in 1144 bytes} [Mon Aug 1 17:07:12 2022] POST /identity/v3/users => generated 294 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 17:07:12.698488 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d13c9aa-5a2c-4b96-95a6-2d0a1df60a19 admin admin] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 17:07:12.708925 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d13c9aa-5a2c-4b96-95a6-2d0a1df60a19 admin admin] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 17:07:12.714824 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd286b3-8a5b-4405-93ad-33850113695a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 17:07:12.715545 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd286b3-8a5b-4405-93ad-33850113695a admin admin] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 17:07:12.715828 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd286b3-8a5b-4405-93ad-33850113695a admin admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 17:07:12.716100 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd286b3-8a5b-4405-93ad-33850113695a admin admin] PATH_INFO: `/v3/roles` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 17:07:12.716794 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdd286b3-8a5b-4405-93ad-33850113695a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 17:07:12.718433 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdd286b3-8a5b-4405-93ad-33850113695a admin admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 17:07:12.729902 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 1288/2588] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Aug 1 17:07:12 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 01 17:07:12.733862 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b86585-8ccf-4209-a888-95e473b0a1bb admin admin] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 17:07:12.744499 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b86585-8ccf-4209-a888-95e473b0a1bb admin admin] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 17:07:12.749254 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce4b1e8e-899a-4759-81f4-285760c15aad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 17:07:12.749932 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4b1e8e-899a-4759-81f4-285760c15aad admin admin] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 17:07:12.750319 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4b1e8e-899a-4759-81f4-285760c15aad admin admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 17:07:12.750654 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4b1e8e-899a-4759-81f4-285760c15aad admin admin] PATH_INFO: `/v3/roles` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 17:07:12.751436 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce4b1e8e-899a-4759-81f4-285760c15aad admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 17:07:12.753080 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce4b1e8e-899a-4759-81f4-285760c15aad admin admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 17:07:12.764232 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 1301/2589] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Aug 1 17:07:12 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 17:07:12.768389 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd286b3-8a5b-4405-93ad-33850113695a admin admin] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 17:07:12.779121 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd286b3-8a5b-4405-93ad-33850113695a admin admin] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 17:07:12.783805 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2535a982-1764-4541-95a3-fbb69c5d0679 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 17:07:12.784473 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2535a982-1764-4541-95a3-fbb69c5d0679 admin admin] REQUEST_METHOD: `PUT` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 17:07:12.784750 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2535a982-1764-4541-95a3-fbb69c5d0679 admin admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 17:07:12.785028 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2535a982-1764-4541-95a3-fbb69c5d0679 admin admin] PATH_INFO: `/v3/projects/16123d96527740df98b034dd9cb9579c/users/0f5b31811d244ba1ac6a731c8384f586/roles/0999d3da63a545f18ff3252afdc90518` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 17:07:12.800397 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2535a982-1764-4541-95a3-fbb69c5d0679 admin admin] RBAC: Authorizing `identity:create_grant(project_id=16123d96527740df98b034dd9cb9579c, user_id=0f5b31811d244ba1ac6a731c8384f586, role_id=0999d3da63a545f18ff3252afdc90518)` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 17:07:12.803216 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2535a982-1764-4541-95a3-fbb69c5d0679 admin admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 17:07:12.889382 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 1289/2590] 10.222.0.20 () {60 vars in 1483 bytes} [Mon Aug 1 17:07:12 2022] PUT /identity/v3/projects/16123d96527740df98b034dd9cb9579c/users/0f5b31811d244ba1ac6a731c8384f586/roles/0999d3da63a545f18ff3252afdc90518 => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 17:07:12.902449 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197c9caa-ba45-4da7-a192-15998162f5e8 None None] REQUEST_METHOD: `POST` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 17:07:12.902449 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197c9caa-ba45-4da7-a192-15998162f5e8 None None] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 17:07:12.902449 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197c9caa-ba45-4da7-a192-15998162f5e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 17:07:12.928260 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.auth.core [None req-197c9caa-ba45-4da7-a192-15998162f5e8 None None] MFA Rules not processed for user `0f5b31811d244ba1ac6a731c8384f586`. Rule list: `[]` (Enabled: `True`). {{(pid=87058) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 17:07:12.929471 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.fernet_utils [None req-197c9caa-ba45-4da7-a192-15998162f5e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 17:07:12.971389 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG dbcounter [-] [87058] Writing DB stats keystone:SELECT=95,keystone:INSERT=4,keystone:DELETE=1 {{(pid=87058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 17:07:12.995862 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 1302/2591] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 1 17:07:12 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 17:07:13.000388 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2535a982-1764-4541-95a3-fbb69c5d0679 admin admin] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 17:07:13.049692 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG dbcounter [-] [87057] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=87057) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 17:07:13.066688 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2535a982-1764-4541-95a3-fbb69c5d0679 admin admin] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 17:07:13.072021 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e3504b7-6ca1-41fa-9d86-fee732740712 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 17:07:13.072640 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3504b7-6ca1-41fa-9d86-fee732740712 admin admin] REQUEST_METHOD: `POST` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 17:07:13.072920 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3504b7-6ca1-41fa-9d86-fee732740712 admin admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 17:07:13.073191 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3504b7-6ca1-41fa-9d86-fee732740712 admin admin] PATH_INFO: `/v3/projects` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 17:07:13.073854 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e3504b7-6ca1-41fa-9d86-fee732740712 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 17:07:13.075885 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e3504b7-6ca1-41fa-9d86-fee732740712 admin admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 17:07:13.175826 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 1290/2592] 10.222.0.20 () {60 vars in 1153 bytes} [Mon Aug 1 17:07:12 2022] POST /identity/v3/projects => generated 387 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 17:07:13.180428 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-197c9caa-ba45-4da7-a192-15998162f5e8 None None] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 17:07:13.191504 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-197c9caa-ba45-4da7-a192-15998162f5e8 None None] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 17:07:13.272065 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0338cc76-b080-4de1-9167-2f7574e5e6dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 17:07:13.275020 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0338cc76-b080-4de1-9167-2f7574e5e6dd admin admin] REQUEST_METHOD: `POST` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 17:07:13.275020 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0338cc76-b080-4de1-9167-2f7574e5e6dd admin admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 17:07:13.275020 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0338cc76-b080-4de1-9167-2f7574e5e6dd admin admin] PATH_INFO: `/v3/users` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 17:07:13.275620 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0338cc76-b080-4de1-9167-2f7574e5e6dd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 17:07:13.278550 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0338cc76-b080-4de1-9167-2f7574e5e6dd admin admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 17:07:13.398368 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 1303/2593] 10.222.0.20 () {60 vars in 1144 bytes} [Mon Aug 1 17:07:13 2022] POST /identity/v3/users => generated 300 bytes in 219 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 17:07:13.399379 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e3504b7-6ca1-41fa-9d86-fee732740712 admin admin] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 17:07:13.410922 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e3504b7-6ca1-41fa-9d86-fee732740712 admin admin] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 17:07:13.421990 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f763279-0c7e-493d-ba47-36e25c346251 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 17:07:13.422815 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f763279-0c7e-493d-ba47-36e25c346251 admin admin] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 17:07:13.423099 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f763279-0c7e-493d-ba47-36e25c346251 admin admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 17:07:13.423376 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f763279-0c7e-493d-ba47-36e25c346251 admin admin] PATH_INFO: `/v3/roles` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 17:07:13.424093 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f763279-0c7e-493d-ba47-36e25c346251 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 17:07:13.425934 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f763279-0c7e-493d-ba47-36e25c346251 admin admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 17:07:13.439645 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 1291/2594] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Aug 1 17:07:13 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 17:07:13.443624 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0338cc76-b080-4de1-9167-2f7574e5e6dd admin admin] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 17:07:13.454835 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0338cc76-b080-4de1-9167-2f7574e5e6dd admin admin] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 17:07:13.460507 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb495627-647e-4488-a644-79118302380c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 17:07:13.461084 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb495627-647e-4488-a644-79118302380c admin admin] REQUEST_METHOD: `PUT` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 17:07:13.461752 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb495627-647e-4488-a644-79118302380c admin admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 17:07:13.461898 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb495627-647e-4488-a644-79118302380c admin admin] PATH_INFO: `/v3/projects/3730bd32452e4643ac71926f2b24c36c/users/e4373fd23dbe430db3d162d59f98bd39/roles/440bb943dd5b46fea038b903b6eaaf81` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 17:07:13.484841 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb495627-647e-4488-a644-79118302380c admin admin] RBAC: Authorizing `identity:create_grant(project_id=3730bd32452e4643ac71926f2b24c36c, user_id=e4373fd23dbe430db3d162d59f98bd39, role_id=440bb943dd5b46fea038b903b6eaaf81)` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 17:07:13.490178 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb495627-647e-4488-a644-79118302380c admin admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 17:07:13.594954 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 1304/2595] 10.222.0.20 () {60 vars in 1483 bytes} [Mon Aug 1 17:07:13 2022] PUT /identity/v3/projects/3730bd32452e4643ac71926f2b24c36c/users/e4373fd23dbe430db3d162d59f98bd39/roles/440bb943dd5b46fea038b903b6eaaf81 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 17:07:13.598706 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f763279-0c7e-493d-ba47-36e25c346251 admin admin] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 17:07:13.610282 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f763279-0c7e-493d-ba47-36e25c346251 admin admin] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 17:07:13.677400 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e61e5a95-e331-471a-9809-ecbf3109c471 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 17:07:13.678594 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61e5a95-e331-471a-9809-ecbf3109c471 admin admin] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 17:07:13.679439 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61e5a95-e331-471a-9809-ecbf3109c471 admin admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 17:07:13.679727 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61e5a95-e331-471a-9809-ecbf3109c471 admin admin] PATH_INFO: `/v3/roles` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 17:07:13.680858 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e61e5a95-e331-471a-9809-ecbf3109c471 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 17:07:13.682976 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e61e5a95-e331-471a-9809-ecbf3109c471 admin admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 17:07:13.703537 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 1292/2596] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Aug 1 17:07:13 2022] GET /identity/v3/roles => generated 1420 bytes in 103 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 17:07:13.703660 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb495627-647e-4488-a644-79118302380c admin admin] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 17:07:13.713725 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb495627-647e-4488-a644-79118302380c admin admin] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 17:07:13.718499 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d24ad27-570a-47c1-b28d-ffd5c147f9ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 17:07:13.719162 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d24ad27-570a-47c1-b28d-ffd5c147f9ae admin admin] REQUEST_METHOD: `PUT` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 17:07:13.719445 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d24ad27-570a-47c1-b28d-ffd5c147f9ae admin admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 17:07:13.719723 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d24ad27-570a-47c1-b28d-ffd5c147f9ae admin admin] PATH_INFO: `/v3/projects/3730bd32452e4643ac71926f2b24c36c/users/e4373fd23dbe430db3d162d59f98bd39/roles/440bb943dd5b46fea038b903b6eaaf81` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 17:07:13.721733 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d24ad27-570a-47c1-b28d-ffd5c147f9ae admin admin] RBAC: Authorizing `identity:create_grant(project_id=3730bd32452e4643ac71926f2b24c36c, user_id=e4373fd23dbe430db3d162d59f98bd39, role_id=440bb943dd5b46fea038b903b6eaaf81)` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 17:07:13.724777 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d24ad27-570a-47c1-b28d-ffd5c147f9ae admin admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 17:07:13.737284 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 1305/2597] 10.222.0.20 () {60 vars in 1483 bytes} [Mon Aug 1 17:07:13 2022] PUT /identity/v3/projects/3730bd32452e4643ac71926f2b24c36c/users/e4373fd23dbe430db3d162d59f98bd39/roles/440bb943dd5b46fea038b903b6eaaf81 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 17:07:13.748364 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a651c40-cfbe-4ebd-8f82-bc004689aa8b None None] REQUEST_METHOD: `POST` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 17:07:13.748660 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a651c40-cfbe-4ebd-8f82-bc004689aa8b None None] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 17:07:13.748924 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a651c40-cfbe-4ebd-8f82-bc004689aa8b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 17:07:13.777705 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.auth.core [None req-4a651c40-cfbe-4ebd-8f82-bc004689aa8b None None] MFA Rules not processed for user `e4373fd23dbe430db3d162d59f98bd39`. Rule list: `[]` (Enabled: `True`). {{(pid=87057) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 17:07:13.779022 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.fernet_utils [None req-4a651c40-cfbe-4ebd-8f82-bc004689aa8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87057) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 17:07:13.848604 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 1293/2598] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 1 17:07:13 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 17:07:13.859975 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d24ad27-570a-47c1-b28d-ffd5c147f9ae admin admin] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 17:07:13.861702 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.fernet_utils [None req-2d24ad27-570a-47c1-b28d-ffd5c147f9ae admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 17:07:13.928108 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d24ad27-570a-47c1-b28d-ffd5c147f9ae admin admin] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 17:07:13.933172 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b86c5c-d4dd-4af9-9c9c-b281a49c9a9d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25970b52ca15448fa1abb1f974d64038', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b992d6a65df4bf9b27c13192d57c5f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 17:07:13.933848 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b86c5c-d4dd-4af9-9c9c-b281a49c9a9d service cinder] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 17:07:13.941998 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b86c5c-d4dd-4af9-9c9c-b281a49c9a9d service cinder] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 17:07:13.943799 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b86c5c-d4dd-4af9-9c9c-b281a49c9a9d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 17:07:13.958790 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99b86c5c-d4dd-4af9-9c9c-b281a49c9a9d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 17:07:13.960958 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99b86c5c-d4dd-4af9-9c9c-b281a49c9a9d service cinder] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 17:07:13.971910 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 1306/2599] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 1 17:07:13 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 17:07:15.120945 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a651c40-cfbe-4ebd-8f82-bc004689aa8b None None] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 17:07:15.169618 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG dbcounter [-] [87057] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87057) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 17:07:15.192400 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a651c40-cfbe-4ebd-8f82-bc004689aa8b None None] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 17:07:15.197770 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78fc2300-c514-4a9b-8521-081f01e6d35c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 17:07:15.198450 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78fc2300-c514-4a9b-8521-081f01e6d35c admin admin] REQUEST_METHOD: `DELETE` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 17:07:15.198733 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78fc2300-c514-4a9b-8521-081f01e6d35c admin admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 17:07:15.199010 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78fc2300-c514-4a9b-8521-081f01e6d35c admin admin] PATH_INFO: `/v3/users/0f5b31811d244ba1ac6a731c8384f586` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 17:07:15.200213 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78fc2300-c514-4a9b-8521-081f01e6d35c admin admin] RBAC: Authorizing `identity:delete_user(user_id=0f5b31811d244ba1ac6a731c8384f586)` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 17:07:15.202175 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78fc2300-c514-4a9b-8521-081f01e6d35c admin admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 17:07:15.370728 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.notifications [None req-78fc2300-c514-4a9b-8521-081f01e6d35c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0f5b31811d244ba1ac6a731c8384f586'} {{(pid=87057) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 17:07:15.371292 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.notifications [None req-78fc2300-c514-4a9b-8521-081f01e6d35c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0f5b31811d244ba1ac6a731c8384f586'} {{(pid=87057) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 17:07:15.436749 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.notifications [None req-78fc2300-c514-4a9b-8521-081f01e6d35c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0f5b31811d244ba1ac6a731c8384f586'} {{(pid=87057) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 17:07:15.437375 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.notifications [None req-78fc2300-c514-4a9b-8521-081f01e6d35c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0f5b31811d244ba1ac6a731c8384f586'} {{(pid=87057) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 17:07:15.446225 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.notifications [None req-78fc2300-c514-4a9b-8521-081f01e6d35c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0f5b31811d244ba1ac6a731c8384f586'} {{(pid=87057) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 17:07:15.464560 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 1294/2600] 10.222.0.20 () {58 vars in 1224 bytes} [Mon Aug 1 17:07:15 2022] DELETE /identity/v3/users/0f5b31811d244ba1ac6a731c8384f586 => generated 0 bytes in 344 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 17:07:15.469250 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c69ff71-59a0-493d-8d4c-a7cb659319d1 None None] REQUEST_METHOD: `POST` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 17:07:15.469738 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c69ff71-59a0-493d-8d4c-a7cb659319d1 None None] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 17:07:15.469846 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c69ff71-59a0-493d-8d4c-a7cb659319d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 17:07:15.489579 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.auth.core [None req-7c69ff71-59a0-493d-8d4c-a7cb659319d1 None None] MFA Rules not processed for user `1007f8e379b74e169b98fb7c3c0ba5b4`. Rule list: `[]` (Enabled: `True`). {{(pid=87058) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 17:07:15.490828 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.fernet_utils [None req-7c69ff71-59a0-493d-8d4c-a7cb659319d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 17:07:15.523588 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG dbcounter [-] [87058] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=87058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 17:07:15.550236 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 1307/2601] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 1 17:07:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 17:07:15.560541 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78fc2300-c514-4a9b-8521-081f01e6d35c admin admin] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 17:07:15.561965 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.fernet_utils [None req-78fc2300-c514-4a9b-8521-081f01e6d35c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87057) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 17:07:15.619479 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78fc2300-c514-4a9b-8521-081f01e6d35c admin admin] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 17:07:15.623336 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f0657a-e2ac-442e-94cb-d180d62a231c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d9bd8d649154c979be3744bfd016f95', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b992d6a65df4bf9b27c13192d57c5f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 17:07:15.623937 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f0657a-e2ac-442e-94cb-d180d62a231c service neutron] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 17:07:15.624222 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f0657a-e2ac-442e-94cb-d180d62a231c service neutron] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 17:07:15.624495 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f0657a-e2ac-442e-94cb-d180d62a231c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 17:07:15.634646 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33f0657a-e2ac-442e-94cb-d180d62a231c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 17:07:15.636296 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33f0657a-e2ac-442e-94cb-d180d62a231c service neutron] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 17:07:15.646409 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 1295/2602] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 1 17:07:15 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 17:07:16.062090 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c69ff71-59a0-493d-8d4c-a7cb659319d1 None None] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 17:07:16.063570 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.fernet_utils [None req-7c69ff71-59a0-493d-8d4c-a7cb659319d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 17:07:16.076588 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c69ff71-59a0-493d-8d4c-a7cb659319d1 None None] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 17:07:16.081293 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de267d5f-f47a-4f28-bc6b-36cbb0a9dda4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 17:07:16.081911 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de267d5f-f47a-4f28-bc6b-36cbb0a9dda4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 17:07:16.082229 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de267d5f-f47a-4f28-bc6b-36cbb0a9dda4 admin admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 17:07:16.082503 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de267d5f-f47a-4f28-bc6b-36cbb0a9dda4 admin admin] PATH_INFO: `/v3/projects/16123d96527740df98b034dd9cb9579c` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 17:07:16.083702 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de267d5f-f47a-4f28-bc6b-36cbb0a9dda4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=16123d96527740df98b034dd9cb9579c)` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 17:07:16.085804 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de267d5f-f47a-4f28-bc6b-36cbb0a9dda4 admin admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 17:07:16.232764 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.notifications [None req-de267d5f-f47a-4f28-bc6b-36cbb0a9dda4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '16123d96527740df98b034dd9cb9579c', 'request_id': 'req-de267d5f-f47a-4f28-bc6b-36cbb0a9dda4'} {{(pid=87058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 17:07:16.239277 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.notifications [None req-de267d5f-f47a-4f28-bc6b-36cbb0a9dda4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '16123d96527740df98b034dd9cb9579c', 'request_id': 'req-de267d5f-f47a-4f28-bc6b-36cbb0a9dda4'} {{(pid=87058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 17:07:16.245476 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.notifications [None req-de267d5f-f47a-4f28-bc6b-36cbb0a9dda4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '16123d96527740df98b034dd9cb9579c', 'request_id': 'req-de267d5f-f47a-4f28-bc6b-36cbb0a9dda4'} {{(pid=87058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 17:07:16.315735 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.notifications [None req-de267d5f-f47a-4f28-bc6b-36cbb0a9dda4 admin admin] The token cache is being invalidate because project 16123d96527740df98b034dd9cb9579c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87058) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 17:07:16.316022 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.notifications [None req-de267d5f-f47a-4f28-bc6b-36cbb0a9dda4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 17:07:16.316645 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.notifications [None req-de267d5f-f47a-4f28-bc6b-36cbb0a9dda4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 17:07:16.318240 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 1308/2603] 10.222.0.20 () {58 vars in 1233 bytes} [Mon Aug 1 17:07:16 2022] DELETE /identity/v3/projects/16123d96527740df98b034dd9cb9579c => generated 0 bytes in 257 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 17:07:16.322268 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f0657a-e2ac-442e-94cb-d180d62a231c service neutron] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 17:07:16.331951 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f0657a-e2ac-442e-94cb-d180d62a231c service neutron] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 17:07:16.387782 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79dd1d4b-5fb6-46d2-a19e-f62cf289bb21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 17:07:16.388445 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79dd1d4b-5fb6-46d2-a19e-f62cf289bb21 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 17:07:16.388722 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79dd1d4b-5fb6-46d2-a19e-f62cf289bb21 admin admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 17:07:16.388991 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79dd1d4b-5fb6-46d2-a19e-f62cf289bb21 admin admin] PATH_INFO: `/v3/users/e4373fd23dbe430db3d162d59f98bd39` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 17:07:16.390206 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79dd1d4b-5fb6-46d2-a19e-f62cf289bb21 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e4373fd23dbe430db3d162d59f98bd39)` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 17:07:16.392114 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79dd1d4b-5fb6-46d2-a19e-f62cf289bb21 admin admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 17:07:16.393557 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG dbcounter [-] [87057] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87057) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 17:07:16.611014 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.notifications [None req-79dd1d4b-5fb6-46d2-a19e-f62cf289bb21 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e4373fd23dbe430db3d162d59f98bd39'} {{(pid=87057) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 17:07:16.611743 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.notifications [None req-79dd1d4b-5fb6-46d2-a19e-f62cf289bb21 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e4373fd23dbe430db3d162d59f98bd39'} {{(pid=87057) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 17:07:16.722474 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.notifications [None req-79dd1d4b-5fb6-46d2-a19e-f62cf289bb21 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e4373fd23dbe430db3d162d59f98bd39'} {{(pid=87057) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 17:07:16.723120 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.notifications [None req-79dd1d4b-5fb6-46d2-a19e-f62cf289bb21 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e4373fd23dbe430db3d162d59f98bd39'} {{(pid=87057) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 17:07:16.731060 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.notifications [None req-79dd1d4b-5fb6-46d2-a19e-f62cf289bb21 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e4373fd23dbe430db3d162d59f98bd39'} {{(pid=87057) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 17:07:16.748820 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 1296/2604] 10.222.0.20 () {58 vars in 1224 bytes} [Mon Aug 1 17:07:16 2022] DELETE /identity/v3/users/e4373fd23dbe430db3d162d59f98bd39 => generated 0 bytes in 427 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 17:07:17.192961 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de267d5f-f47a-4f28-bc6b-36cbb0a9dda4 admin admin] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 17:07:17.195327 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.fernet_utils [None req-de267d5f-f47a-4f28-bc6b-36cbb0a9dda4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 17:07:17.208997 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de267d5f-f47a-4f28-bc6b-36cbb0a9dda4 admin admin] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 17:07:17.266889 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c78fb8-d5e7-4927-94cd-13cae776e547 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 17:07:17.267552 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c78fb8-d5e7-4927-94cd-13cae776e547 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 17:07:17.267806 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c78fb8-d5e7-4927-94cd-13cae776e547 admin admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 17:07:17.268114 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c78fb8-d5e7-4927-94cd-13cae776e547 admin admin] PATH_INFO: `/v3/projects/3730bd32452e4643ac71926f2b24c36c` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 17:07:17.269255 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61c78fb8-d5e7-4927-94cd-13cae776e547 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3730bd32452e4643ac71926f2b24c36c)` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 17:07:17.271385 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61c78fb8-d5e7-4927-94cd-13cae776e547 admin admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 17:07:17.380095 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.notifications [None req-61c78fb8-d5e7-4927-94cd-13cae776e547 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3730bd32452e4643ac71926f2b24c36c', 'request_id': 'req-61c78fb8-d5e7-4927-94cd-13cae776e547'} {{(pid=87058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 17:07:17.386979 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.notifications [None req-61c78fb8-d5e7-4927-94cd-13cae776e547 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3730bd32452e4643ac71926f2b24c36c', 'request_id': 'req-61c78fb8-d5e7-4927-94cd-13cae776e547'} {{(pid=87058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 17:07:17.392723 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.notifications [None req-61c78fb8-d5e7-4927-94cd-13cae776e547 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3730bd32452e4643ac71926f2b24c36c', 'request_id': 'req-61c78fb8-d5e7-4927-94cd-13cae776e547'} {{(pid=87058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 17:07:17.394447 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG dbcounter [-] [87058] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=87058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 17:07:17.487508 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.notifications [None req-61c78fb8-d5e7-4927-94cd-13cae776e547 admin admin] The token cache is being invalidate because project 3730bd32452e4643ac71926f2b24c36c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87058) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 17:07:17.487799 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.notifications [None req-61c78fb8-d5e7-4927-94cd-13cae776e547 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 17:07:17.488745 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.notifications [None req-61c78fb8-d5e7-4927-94cd-13cae776e547 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 17:07:17.490294 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 1309/2605] 10.222.0.20 () {58 vars in 1233 bytes} [Mon Aug 1 17:07:17 2022] DELETE /identity/v3/projects/3730bd32452e4643ac71926f2b24c36c => generated 0 bytes in 298 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 17:07:17.498017 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9a5674-5335-4556-a090-d3535cfd4cb3 None None] REQUEST_METHOD: `POST` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 17:07:17.498017 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9a5674-5335-4556-a090-d3535cfd4cb3 None None] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 17:07:17.498017 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9a5674-5335-4556-a090-d3535cfd4cb3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 17:07:17.516519 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.auth.core [None req-af9a5674-5335-4556-a090-d3535cfd4cb3 None None] MFA Rules not processed for user `1007f8e379b74e169b98fb7c3c0ba5b4`. Rule list: `[]` (Enabled: `True`). {{(pid=87057) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 17:07:17.517774 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.fernet_utils [None req-af9a5674-5335-4556-a090-d3535cfd4cb3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87057) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 17:07:17.576035 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 1297/2606] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 1 17:07:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 17:07:17.583576 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9bde35-673f-45a2-a8a0-855beaed40e3 None None] REQUEST_METHOD: `POST` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 17:07:17.583862 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9bde35-673f-45a2-a8a0-855beaed40e3 None None] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 17:07:17.584138 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9bde35-673f-45a2-a8a0-855beaed40e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 17:07:17.603865 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.auth.core [None req-6e9bde35-673f-45a2-a8a0-855beaed40e3 None None] MFA Rules not processed for user `1007f8e379b74e169b98fb7c3c0ba5b4`. Rule list: `[]` (Enabled: `True`). {{(pid=87058) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 17:07:17.605065 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.fernet_utils [None req-6e9bde35-673f-45a2-a8a0-855beaed40e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 17:07:17.612942 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 1310/2607] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 1 17:07:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 17:07:17.617422 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af9a5674-5335-4556-a090-d3535cfd4cb3 None None] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 17:07:17.626980 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af9a5674-5335-4556-a090-d3535cfd4cb3 None None] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 17:07:17.631503 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a3235f-4d34-4d3e-b921-ae6978c72a36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 17:07:17.632642 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a3235f-4d34-4d3e-b921-ae6978c72a36 admin admin] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 17:07:17.632919 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a3235f-4d34-4d3e-b921-ae6978c72a36 admin admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 17:07:17.633186 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a3235f-4d34-4d3e-b921-ae6978c72a36 admin admin] PATH_INFO: `/v3/domains` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 17:07:17.633878 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52a3235f-4d34-4d3e-b921-ae6978c72a36 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87057) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 17:07:17.634216 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52a3235f-4d34-4d3e-b921-ae6978c72a36 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 17:07:17.635809 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52a3235f-4d34-4d3e-b921-ae6978c72a36 admin admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 17:07:17.647062 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 1298/2608] 10.222.0.20 () {58 vars in 1153 bytes} [Mon Aug 1 17:07:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 17:07:17.651929 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e9bde35-673f-45a2-a8a0-855beaed40e3 None None] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 17:07:17.660970 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e9bde35-673f-45a2-a8a0-855beaed40e3 None None] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 17:07:17.665716 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b2a73b-12c9-4c8a-89b1-55176276d5da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 17:07:17.666346 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b2a73b-12c9-4c8a-89b1-55176276d5da admin admin] REQUEST_METHOD: `POST` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 17:07:17.666607 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b2a73b-12c9-4c8a-89b1-55176276d5da admin admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 17:07:17.666873 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b2a73b-12c9-4c8a-89b1-55176276d5da admin admin] PATH_INFO: `/v3/projects` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 17:07:17.667517 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99b2a73b-12c9-4c8a-89b1-55176276d5da admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 17:07:17.669389 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99b2a73b-12c9-4c8a-89b1-55176276d5da admin admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 17:07:17.804451 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 1311/2609] 10.222.0.20 () {60 vars in 1153 bytes} [Mon Aug 1 17:07:17 2022] POST /identity/v3/projects => generated 373 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 17:07:17.808529 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a3235f-4d34-4d3e-b921-ae6978c72a36 admin admin] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 17:07:17.818793 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a3235f-4d34-4d3e-b921-ae6978c72a36 admin admin] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 17:07:17.878905 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb623a87-9bcd-4d6e-90cb-36b4f4f5f53f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 17:07:17.879803 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb623a87-9bcd-4d6e-90cb-36b4f4f5f53f admin admin] REQUEST_METHOD: `POST` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 17:07:17.880087 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb623a87-9bcd-4d6e-90cb-36b4f4f5f53f admin admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 17:07:17.880364 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb623a87-9bcd-4d6e-90cb-36b4f4f5f53f admin admin] PATH_INFO: `/v3/users` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 17:07:17.883593 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb623a87-9bcd-4d6e-90cb-36b4f4f5f53f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 17:07:17.886234 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb623a87-9bcd-4d6e-90cb-36b4f4f5f53f admin admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 17:07:17.896205 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG dbcounter [-] [87057] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=87057) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 17:07:17.929375 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b2a73b-12c9-4c8a-89b1-55176276d5da admin admin] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 17:07:17.931187 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.fernet_utils [None req-99b2a73b-12c9-4c8a-89b1-55176276d5da admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 17:07:17.979609 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 1299/2610] 10.222.0.20 () {60 vars in 1144 bytes} [Mon Aug 1 17:07:17 2022] POST /identity/v3/users => generated 287 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 17:07:17.983694 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb623a87-9bcd-4d6e-90cb-36b4f4f5f53f admin admin] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 17:07:17.989340 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b2a73b-12c9-4c8a-89b1-55176276d5da admin admin] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 17:07:17.994156 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d821f22-abe7-4230-b45f-340a6101bf7e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d9bd8d649154c979be3744bfd016f95', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b992d6a65df4bf9b27c13192d57c5f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 17:07:17.994778 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d821f22-abe7-4230-b45f-340a6101bf7e service neutron] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 17:07:17.995089 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d821f22-abe7-4230-b45f-340a6101bf7e service neutron] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 17:07:17.995405 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d821f22-abe7-4230-b45f-340a6101bf7e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 17:07:17.995984 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb623a87-9bcd-4d6e-90cb-36b4f4f5f53f admin admin] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 17:07:17.997197 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.fernet_utils [None req-0d821f22-abe7-4230-b45f-340a6101bf7e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 17:07:18.000715 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f79533f9-3327-4e61-aade-66df629a7505 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 17:07:18.003861 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79533f9-3327-4e61-aade-66df629a7505 admin admin] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 17:07:18.003861 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79533f9-3327-4e61-aade-66df629a7505 admin admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 17:07:18.003861 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79533f9-3327-4e61-aade-66df629a7505 admin admin] PATH_INFO: `/v3/roles` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 17:07:18.003861 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f79533f9-3327-4e61-aade-66df629a7505 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 17:07:18.004255 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f79533f9-3327-4e61-aade-66df629a7505 admin admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 17:07:18.010986 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d821f22-abe7-4230-b45f-340a6101bf7e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 17:07:18.012448 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d821f22-abe7-4230-b45f-340a6101bf7e service neutron] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 17:07:18.018803 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 1300/2611] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Aug 1 17:07:17 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 01 17:07:18.019725 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f79533f9-3327-4e61-aade-66df629a7505 admin admin] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 17:07:18.022917 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 1312/2612] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 1 17:07:17 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 01 17:07:18.029287 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f79533f9-3327-4e61-aade-66df629a7505 admin admin] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 17:07:18.033892 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e79f84-c7cd-400b-8da5-9f4a567d2c74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 17:07:18.034529 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e79f84-c7cd-400b-8da5-9f4a567d2c74 admin admin] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 17:07:18.034813 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e79f84-c7cd-400b-8da5-9f4a567d2c74 admin admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 17:07:18.035086 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e79f84-c7cd-400b-8da5-9f4a567d2c74 admin admin] PATH_INFO: `/v3/roles` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 17:07:18.035773 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06e79f84-c7cd-400b-8da5-9f4a567d2c74 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 17:07:18.037354 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06e79f84-c7cd-400b-8da5-9f4a567d2c74 admin admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 17:07:18.047523 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 1301/2613] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Aug 1 17:07:18 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 17:07:18.051781 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d821f22-abe7-4230-b45f-340a6101bf7e service neutron] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 17:07:18.062450 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d821f22-abe7-4230-b45f-340a6101bf7e service neutron] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 17:07:18.067276 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0388ac7f-246e-409f-bbb8-47b60f1af6e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 17:07:18.067939 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0388ac7f-246e-409f-bbb8-47b60f1af6e1 admin admin] REQUEST_METHOD: `PUT` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 17:07:18.068221 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0388ac7f-246e-409f-bbb8-47b60f1af6e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 17:07:18.068494 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0388ac7f-246e-409f-bbb8-47b60f1af6e1 admin admin] PATH_INFO: `/v3/projects/66b02f044b38477ebeab310bcb212eb3/users/3ab5faa2a30e479ba62db97b14ad39d9/roles/0999d3da63a545f18ff3252afdc90518` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 17:07:18.084494 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0388ac7f-246e-409f-bbb8-47b60f1af6e1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=66b02f044b38477ebeab310bcb212eb3, user_id=3ab5faa2a30e479ba62db97b14ad39d9, role_id=0999d3da63a545f18ff3252afdc90518)` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 17:07:18.087498 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0388ac7f-246e-409f-bbb8-47b60f1af6e1 admin admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 17:07:18.140098 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 1313/2614] 10.222.0.20 () {60 vars in 1483 bytes} [Mon Aug 1 17:07:18 2022] PUT /identity/v3/projects/66b02f044b38477ebeab310bcb212eb3/users/3ab5faa2a30e479ba62db97b14ad39d9/roles/0999d3da63a545f18ff3252afdc90518 => generated 0 bytes in 89 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 17:07:18.152150 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7f79f3-e523-4f51-8155-d44ea6f414f4 None None] REQUEST_METHOD: `POST` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 17:07:18.152440 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7f79f3-e523-4f51-8155-d44ea6f414f4 None None] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 17:07:18.152820 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7f79f3-e523-4f51-8155-d44ea6f414f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 17:07:18.181886 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.auth.core [None req-8d7f79f3-e523-4f51-8155-d44ea6f414f4 None None] MFA Rules not processed for user `3ab5faa2a30e479ba62db97b14ad39d9`. Rule list: `[]` (Enabled: `True`). {{(pid=87057) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 17:07:18.183148 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.fernet_utils [None req-8d7f79f3-e523-4f51-8155-d44ea6f414f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87057) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 17:07:18.249493 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 1302/2615] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 1 17:07:18 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 17:07:18.261633 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0388ac7f-246e-409f-bbb8-47b60f1af6e1 admin admin] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 17:07:18.263542 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.fernet_utils [None req-0388ac7f-246e-409f-bbb8-47b60f1af6e1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 17:07:18.322737 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0388ac7f-246e-409f-bbb8-47b60f1af6e1 admin admin] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 17:07:18.326595 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43bb2aef-177a-40f4-a55a-d9bc87f11138 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25970b52ca15448fa1abb1f974d64038', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b992d6a65df4bf9b27c13192d57c5f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 17:07:18.327256 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43bb2aef-177a-40f4-a55a-d9bc87f11138 service cinder] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 17:07:18.327535 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43bb2aef-177a-40f4-a55a-d9bc87f11138 service cinder] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 17:07:18.327813 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43bb2aef-177a-40f4-a55a-d9bc87f11138 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 17:07:18.337198 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43bb2aef-177a-40f4-a55a-d9bc87f11138 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 17:07:18.338688 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43bb2aef-177a-40f4-a55a-d9bc87f11138 service cinder] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 17:07:18.348111 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 1314/2616] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 1 17:07:18 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 17:07:18.370329 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d7f79f3-e523-4f51-8155-d44ea6f414f4 None None] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 17:07:18.434747 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d7f79f3-e523-4f51-8155-d44ea6f414f4 None None] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 17:07:18.439409 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6b0b8ab-80f1-4291-8bdf-ff818647d834 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 17:07:18.440054 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b0b8ab-80f1-4291-8bdf-ff818647d834 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 17:07:18.440333 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b0b8ab-80f1-4291-8bdf-ff818647d834 admin admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 17:07:18.440643 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b0b8ab-80f1-4291-8bdf-ff818647d834 admin admin] PATH_INFO: `/v3/users/3ab5faa2a30e479ba62db97b14ad39d9` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 17:07:18.441840 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6b0b8ab-80f1-4291-8bdf-ff818647d834 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3ab5faa2a30e479ba62db97b14ad39d9)` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 17:07:18.444913 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6b0b8ab-80f1-4291-8bdf-ff818647d834 admin admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 17:07:18.645247 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.notifications [None req-e6b0b8ab-80f1-4291-8bdf-ff818647d834 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3ab5faa2a30e479ba62db97b14ad39d9'} {{(pid=87057) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 17:07:18.646268 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.notifications [None req-e6b0b8ab-80f1-4291-8bdf-ff818647d834 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3ab5faa2a30e479ba62db97b14ad39d9'} {{(pid=87057) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 17:07:18.711769 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.notifications [None req-e6b0b8ab-80f1-4291-8bdf-ff818647d834 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3ab5faa2a30e479ba62db97b14ad39d9'} {{(pid=87057) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 17:07:18.712522 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.notifications [None req-e6b0b8ab-80f1-4291-8bdf-ff818647d834 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3ab5faa2a30e479ba62db97b14ad39d9'} {{(pid=87057) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 17:07:18.720115 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.notifications [None req-e6b0b8ab-80f1-4291-8bdf-ff818647d834 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3ab5faa2a30e479ba62db97b14ad39d9'} {{(pid=87057) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 17:07:18.722092 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG dbcounter [-] [87057] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=87057) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 17:07:18.740020 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 1303/2617] 10.222.0.20 () {58 vars in 1224 bytes} [Mon Aug 1 17:07:18 2022] DELETE /identity/v3/users/3ab5faa2a30e479ba62db97b14ad39d9 => generated 0 bytes in 370 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 17:07:18.744525 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94a8102-81fc-4c23-a93d-53f219a9d626 None None] REQUEST_METHOD: `POST` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 17:07:18.744881 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94a8102-81fc-4c23-a93d-53f219a9d626 None None] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 17:07:18.745165 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94a8102-81fc-4c23-a93d-53f219a9d626 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 17:07:18.764262 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.auth.core [None req-c94a8102-81fc-4c23-a93d-53f219a9d626 None None] MFA Rules not processed for user `1007f8e379b74e169b98fb7c3c0ba5b4`. Rule list: `[]` (Enabled: `True`). {{(pid=87058) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 17:07:18.765489 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.fernet_utils [None req-c94a8102-81fc-4c23-a93d-53f219a9d626 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 17:07:18.809273 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG dbcounter [-] [87058] Writing DB stats keystone:INSERT=3,keystone:DELETE=1,keystone:SELECT=96 {{(pid=87058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 17:07:18.828255 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 1315/2618] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 1 17:07:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 17:07:18.838690 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6b0b8ab-80f1-4291-8bdf-ff818647d834 admin admin] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 17:07:18.840144 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.fernet_utils [None req-e6b0b8ab-80f1-4291-8bdf-ff818647d834 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87057) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 17:07:18.899303 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6b0b8ab-80f1-4291-8bdf-ff818647d834 admin admin] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 17:07:18.903215 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8058888f-9587-4d3c-920f-c5f8a0b0f608 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d9bd8d649154c979be3744bfd016f95', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b992d6a65df4bf9b27c13192d57c5f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 17:07:18.904001 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8058888f-9587-4d3c-920f-c5f8a0b0f608 service neutron] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 17:07:18.904280 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8058888f-9587-4d3c-920f-c5f8a0b0f608 service neutron] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 17:07:18.904554 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8058888f-9587-4d3c-920f-c5f8a0b0f608 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 17:07:18.915360 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8058888f-9587-4d3c-920f-c5f8a0b0f608 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 17:07:18.916832 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8058888f-9587-4d3c-920f-c5f8a0b0f608 service neutron] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 17:07:18.927326 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 1304/2619] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 1 17:07:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 17:07:19.415632 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c94a8102-81fc-4c23-a93d-53f219a9d626 None None] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 17:07:19.417820 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.fernet_utils [None req-c94a8102-81fc-4c23-a93d-53f219a9d626 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 17:07:19.436641 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c94a8102-81fc-4c23-a93d-53f219a9d626 None None] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 17:07:19.442538 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e90cb1c3-7aef-4ccc-a1f0-87215c65f2b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 17:07:19.443145 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90cb1c3-7aef-4ccc-a1f0-87215c65f2b9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 17:07:19.443422 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90cb1c3-7aef-4ccc-a1f0-87215c65f2b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 17:07:19.443699 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90cb1c3-7aef-4ccc-a1f0-87215c65f2b9 admin admin] PATH_INFO: `/v3/projects/66b02f044b38477ebeab310bcb212eb3` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 17:07:19.445389 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e90cb1c3-7aef-4ccc-a1f0-87215c65f2b9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=66b02f044b38477ebeab310bcb212eb3)` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 17:07:19.447541 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e90cb1c3-7aef-4ccc-a1f0-87215c65f2b9 admin admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 17:07:19.571142 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.notifications [None req-e90cb1c3-7aef-4ccc-a1f0-87215c65f2b9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '66b02f044b38477ebeab310bcb212eb3', 'request_id': 'req-e90cb1c3-7aef-4ccc-a1f0-87215c65f2b9'} {{(pid=87058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 17:07:19.577547 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.notifications [None req-e90cb1c3-7aef-4ccc-a1f0-87215c65f2b9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '66b02f044b38477ebeab310bcb212eb3', 'request_id': 'req-e90cb1c3-7aef-4ccc-a1f0-87215c65f2b9'} {{(pid=87058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 17:07:19.583135 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.notifications [None req-e90cb1c3-7aef-4ccc-a1f0-87215c65f2b9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '66b02f044b38477ebeab310bcb212eb3', 'request_id': 'req-e90cb1c3-7aef-4ccc-a1f0-87215c65f2b9'} {{(pid=87058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 17:07:19.658958 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.notifications [None req-e90cb1c3-7aef-4ccc-a1f0-87215c65f2b9 admin admin] The token cache is being invalidate because project 66b02f044b38477ebeab310bcb212eb3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87058) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 17:07:19.659158 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.notifications [None req-e90cb1c3-7aef-4ccc-a1f0-87215c65f2b9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 17:07:19.659795 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.notifications [None req-e90cb1c3-7aef-4ccc-a1f0-87215c65f2b9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 17:07:19.661394 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 1316/2620] 10.222.0.20 () {58 vars in 1233 bytes} [Mon Aug 1 17:07:19 2022] DELETE /identity/v3/projects/66b02f044b38477ebeab310bcb212eb3 => generated 0 bytes in 246 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 17:07:19.666830 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd30a7ad-d6da-4460-9203-0aac788eb64f None None] REQUEST_METHOD: `POST` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 17:07:19.666830 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd30a7ad-d6da-4460-9203-0aac788eb64f None None] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 17:07:19.667318 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd30a7ad-d6da-4460-9203-0aac788eb64f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 17:07:19.689194 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.auth.core [None req-dd30a7ad-d6da-4460-9203-0aac788eb64f None None] MFA Rules not processed for user `1007f8e379b74e169b98fb7c3c0ba5b4`. Rule list: `[]` (Enabled: `True`). {{(pid=87057) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 17:07:19.690610 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.fernet_utils [None req-dd30a7ad-d6da-4460-9203-0aac788eb64f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87057) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 17:07:19.760354 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 1305/2621] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 1 17:07:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 17:07:19.827527 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f4a257-1f8d-4a98-965d-cfc271da0901 None None] REQUEST_METHOD: `POST` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 17:07:19.827813 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f4a257-1f8d-4a98-965d-cfc271da0901 None None] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 17:07:19.828085 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f4a257-1f8d-4a98-965d-cfc271da0901 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 17:07:19.848622 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.auth.core [None req-b9f4a257-1f8d-4a98-965d-cfc271da0901 None None] MFA Rules not processed for user `1007f8e379b74e169b98fb7c3c0ba5b4`. Rule list: `[]` (Enabled: `True`). {{(pid=87058) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 17:07:19.849836 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.fernet_utils [None req-b9f4a257-1f8d-4a98-965d-cfc271da0901 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 17:07:19.862019 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 1317/2622] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 1 17:07:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 17:07:19.863585 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd30a7ad-d6da-4460-9203-0aac788eb64f None None] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 17:07:19.875123 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd30a7ad-d6da-4460-9203-0aac788eb64f None None] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 17:07:19.880692 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0308114-ec84-47f0-a174-8638be23b201 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 17:07:19.881323 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0308114-ec84-47f0-a174-8638be23b201 admin admin] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 17:07:19.881605 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0308114-ec84-47f0-a174-8638be23b201 admin admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 17:07:19.881925 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0308114-ec84-47f0-a174-8638be23b201 admin admin] PATH_INFO: `/v3/domains` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 17:07:19.883077 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0308114-ec84-47f0-a174-8638be23b201 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87057) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 17:07:19.883387 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0308114-ec84-47f0-a174-8638be23b201 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 17:07:19.885022 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0308114-ec84-47f0-a174-8638be23b201 admin admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 17:07:19.899095 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 1306/2623] 10.222.0.20 () {58 vars in 1153 bytes} [Mon Aug 1 17:07:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 17:07:19.904405 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f4a257-1f8d-4a98-965d-cfc271da0901 None None] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 17:07:19.916140 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f4a257-1f8d-4a98-965d-cfc271da0901 None None] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 17:07:19.921018 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfb2c4e8-38b2-47de-9272-4af260057ed9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 17:07:19.921659 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb2c4e8-38b2-47de-9272-4af260057ed9 admin admin] REQUEST_METHOD: `POST` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 17:07:19.921935 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb2c4e8-38b2-47de-9272-4af260057ed9 admin admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 17:07:19.922239 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb2c4e8-38b2-47de-9272-4af260057ed9 admin admin] PATH_INFO: `/v3/projects` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 17:07:19.922906 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfb2c4e8-38b2-47de-9272-4af260057ed9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 17:07:19.924899 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfb2c4e8-38b2-47de-9272-4af260057ed9 admin admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 17:07:20.007253 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 1318/2624] 10.222.0.20 () {60 vars in 1153 bytes} [Mon Aug 1 17:07:19 2022] POST /identity/v3/projects => generated 361 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 17:07:20.008538 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0308114-ec84-47f0-a174-8638be23b201 admin admin] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 17:07:20.018795 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0308114-ec84-47f0-a174-8638be23b201 admin admin] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 17:07:20.087983 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fe026be-c2f0-4920-842c-02f1224539c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 17:07:20.087983 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe026be-c2f0-4920-842c-02f1224539c5 admin admin] REQUEST_METHOD: `POST` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 17:07:20.087983 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe026be-c2f0-4920-842c-02f1224539c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 17:07:20.087983 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe026be-c2f0-4920-842c-02f1224539c5 admin admin] PATH_INFO: `/v3/users` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 17:07:20.088416 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fe026be-c2f0-4920-842c-02f1224539c5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 17:07:20.089702 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fe026be-c2f0-4920-842c-02f1224539c5 admin admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 17:07:20.187739 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 1307/2625] 10.222.0.20 () {60 vars in 1144 bytes} [Mon Aug 1 17:07:20 2022] POST /identity/v3/users => generated 281 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 17:07:20.191258 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfb2c4e8-38b2-47de-9272-4af260057ed9 admin admin] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 17:07:20.204017 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfb2c4e8-38b2-47de-9272-4af260057ed9 admin admin] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 17:07:20.209210 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cbc1191-f0de-4c2e-a7ec-2e629824f09a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 17:07:20.209713 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cbc1191-f0de-4c2e-a7ec-2e629824f09a admin admin] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 17:07:20.210671 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cbc1191-f0de-4c2e-a7ec-2e629824f09a admin admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 17:07:20.211305 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cbc1191-f0de-4c2e-a7ec-2e629824f09a admin admin] PATH_INFO: `/v3/roles` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 17:07:20.211732 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cbc1191-f0de-4c2e-a7ec-2e629824f09a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 17:07:20.213504 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cbc1191-f0de-4c2e-a7ec-2e629824f09a admin admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 17:07:20.230066 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 1319/2626] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Aug 1 17:07:20 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 17:07:20.233275 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fe026be-c2f0-4920-842c-02f1224539c5 admin admin] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 17:07:20.238932 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG dbcounter [-] [87057] Writing DB stats keystone:SELECT=95,keystone:DELETE=2,keystone:INSERT=3 {{(pid=87057) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 17:07:20.244739 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fe026be-c2f0-4920-842c-02f1224539c5 admin admin] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 17:07:20.249495 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd44a54b-110f-4505-9836-7f187fc134ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 17:07:20.250243 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd44a54b-110f-4505-9836-7f187fc134ab admin admin] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 17:07:20.250587 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd44a54b-110f-4505-9836-7f187fc134ab admin admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 17:07:20.250928 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd44a54b-110f-4505-9836-7f187fc134ab admin admin] PATH_INFO: `/v3/roles` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 17:07:20.251697 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd44a54b-110f-4505-9836-7f187fc134ab admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 17:07:20.253413 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd44a54b-110f-4505-9836-7f187fc134ab admin admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 17:07:20.264970 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 1308/2627] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Aug 1 17:07:20 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 17:07:20.269179 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cbc1191-f0de-4c2e-a7ec-2e629824f09a admin admin] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 17:07:20.279554 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cbc1191-f0de-4c2e-a7ec-2e629824f09a admin admin] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 17:07:20.284279 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3824736-2113-4d96-9ace-45f4409f5e45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 17:07:20.284934 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3824736-2113-4d96-9ace-45f4409f5e45 admin admin] REQUEST_METHOD: `PUT` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 17:07:20.285214 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3824736-2113-4d96-9ace-45f4409f5e45 admin admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 17:07:20.285516 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3824736-2113-4d96-9ace-45f4409f5e45 admin admin] PATH_INFO: `/v3/projects/861d785b831c46498161891b2a490c12/users/1699fa0f04d9411db9530380ac94978f/roles/0999d3da63a545f18ff3252afdc90518` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 17:07:20.299962 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3824736-2113-4d96-9ace-45f4409f5e45 admin admin] RBAC: Authorizing `identity:create_grant(project_id=861d785b831c46498161891b2a490c12, user_id=1699fa0f04d9411db9530380ac94978f, role_id=0999d3da63a545f18ff3252afdc90518)` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 17:07:20.304215 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3824736-2113-4d96-9ace-45f4409f5e45 admin admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 17:07:20.386322 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 1320/2628] 10.222.0.20 () {60 vars in 1483 bytes} [Mon Aug 1 17:07:20 2022] PUT /identity/v3/projects/861d785b831c46498161891b2a490c12/users/1699fa0f04d9411db9530380ac94978f/roles/0999d3da63a545f18ff3252afdc90518 => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 17:07:20.396307 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107d3ce6-afeb-40eb-8b98-582621eefd33 None None] REQUEST_METHOD: `POST` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 17:07:20.397028 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107d3ce6-afeb-40eb-8b98-582621eefd33 None None] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 17:07:20.397332 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107d3ce6-afeb-40eb-8b98-582621eefd33 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 17:07:20.426859 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.auth.core [None req-107d3ce6-afeb-40eb-8b98-582621eefd33 None None] MFA Rules not processed for user `1699fa0f04d9411db9530380ac94978f`. Rule list: `[]` (Enabled: `True`). {{(pid=87057) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 17:07:20.428100 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.fernet_utils [None req-107d3ce6-afeb-40eb-8b98-582621eefd33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87057) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 17:07:20.501020 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 1309/2629] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 1 17:07:20 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 17:07:20.532780 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3824736-2113-4d96-9ace-45f4409f5e45 admin admin] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 17:07:20.534657 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.fernet_utils [None req-d3824736-2113-4d96-9ace-45f4409f5e45 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 17:07:20.607780 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3824736-2113-4d96-9ace-45f4409f5e45 admin admin] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 17:07:20.612218 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c87bda7a-c4bb-4026-bbdb-5bb357c10037 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25970b52ca15448fa1abb1f974d64038', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b992d6a65df4bf9b27c13192d57c5f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 17:07:20.613405 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87bda7a-c4bb-4026-bbdb-5bb357c10037 service cinder] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 17:07:20.613526 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87bda7a-c4bb-4026-bbdb-5bb357c10037 service cinder] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 17:07:20.613764 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87bda7a-c4bb-4026-bbdb-5bb357c10037 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 17:07:20.625513 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c87bda7a-c4bb-4026-bbdb-5bb357c10037 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 17:07:20.627199 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c87bda7a-c4bb-4026-bbdb-5bb357c10037 service cinder] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 17:07:20.639466 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 1321/2630] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 1 17:07:20 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 17:07:20.658758 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-107d3ce6-afeb-40eb-8b98-582621eefd33 None None] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 17:07:20.727091 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-107d3ce6-afeb-40eb-8b98-582621eefd33 None None] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 17:07:20.732019 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1705270b-0e80-4b86-bca3-a28a2f56bac3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 17:07:20.732726 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1705270b-0e80-4b86-bca3-a28a2f56bac3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 17:07:20.733011 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1705270b-0e80-4b86-bca3-a28a2f56bac3 admin admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 17:07:20.733289 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1705270b-0e80-4b86-bca3-a28a2f56bac3 admin admin] PATH_INFO: `/v3/users/1699fa0f04d9411db9530380ac94978f` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 17:07:20.734597 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1705270b-0e80-4b86-bca3-a28a2f56bac3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1699fa0f04d9411db9530380ac94978f)` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 17:07:20.736590 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1705270b-0e80-4b86-bca3-a28a2f56bac3 admin admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 17:07:20.917031 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.notifications [None req-1705270b-0e80-4b86-bca3-a28a2f56bac3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1699fa0f04d9411db9530380ac94978f'} {{(pid=87057) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 17:07:20.917210 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.notifications [None req-1705270b-0e80-4b86-bca3-a28a2f56bac3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1699fa0f04d9411db9530380ac94978f'} {{(pid=87057) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 17:07:20.968225 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.notifications [None req-1705270b-0e80-4b86-bca3-a28a2f56bac3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1699fa0f04d9411db9530380ac94978f'} {{(pid=87057) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 17:07:20.969066 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.notifications [None req-1705270b-0e80-4b86-bca3-a28a2f56bac3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1699fa0f04d9411db9530380ac94978f'} {{(pid=87057) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 17:07:20.980745 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.notifications [None req-1705270b-0e80-4b86-bca3-a28a2f56bac3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1699fa0f04d9411db9530380ac94978f'} {{(pid=87057) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 17:07:21.006471 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG dbcounter [-] [87057] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87057) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 17:07:21.011623 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 1310/2631] 10.222.0.20 () {58 vars in 1224 bytes} [Mon Aug 1 17:07:20 2022] DELETE /identity/v3/users/1699fa0f04d9411db9530380ac94978f => generated 0 bytes in 354 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 17:07:21.016071 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a78291a-0c20-425d-bf48-ab309ad4f970 None None] REQUEST_METHOD: `POST` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 17:07:21.016433 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a78291a-0c20-425d-bf48-ab309ad4f970 None None] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 17:07:21.016772 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a78291a-0c20-425d-bf48-ab309ad4f970 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 17:07:21.030358 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG dbcounter [-] [87058] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=87058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 17:07:21.036808 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.auth.core [None req-2a78291a-0c20-425d-bf48-ab309ad4f970 None None] MFA Rules not processed for user `1007f8e379b74e169b98fb7c3c0ba5b4`. Rule list: `[]` (Enabled: `True`). {{(pid=87058) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 17:07:21.038164 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.fernet_utils [None req-2a78291a-0c20-425d-bf48-ab309ad4f970 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 17:07:21.108010 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 1322/2632] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 1 17:07:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 17:07:21.122880 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1705270b-0e80-4b86-bca3-a28a2f56bac3 admin admin] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 17:07:21.124324 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.fernet_utils [None req-1705270b-0e80-4b86-bca3-a28a2f56bac3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87057) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 17:07:21.188742 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1705270b-0e80-4b86-bca3-a28a2f56bac3 admin admin] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 17:07:21.192552 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cb1b095-1cb2-4b30-86c0-8566a08d55e1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d9bd8d649154c979be3744bfd016f95', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b992d6a65df4bf9b27c13192d57c5f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 17:07:21.193185 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb1b095-1cb2-4b30-86c0-8566a08d55e1 service neutron] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 17:07:21.193469 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb1b095-1cb2-4b30-86c0-8566a08d55e1 service neutron] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 17:07:21.193745 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb1b095-1cb2-4b30-86c0-8566a08d55e1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 17:07:21.204281 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cb1b095-1cb2-4b30-86c0-8566a08d55e1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 17:07:21.205749 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cb1b095-1cb2-4b30-86c0-8566a08d55e1 service neutron] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 17:07:21.217958 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 1311/2633] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 1 17:07:21 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 17:07:21.776474 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a78291a-0c20-425d-bf48-ab309ad4f970 None None] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 17:07:21.778525 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.fernet_utils [None req-2a78291a-0c20-425d-bf48-ab309ad4f970 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 17:07:21.791369 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a78291a-0c20-425d-bf48-ab309ad4f970 None None] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 17:07:21.796848 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1acef4a4-9ba7-4eee-ad46-1c27b6172506 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 17:07:21.797509 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1acef4a4-9ba7-4eee-ad46-1c27b6172506 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 17:07:21.797790 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1acef4a4-9ba7-4eee-ad46-1c27b6172506 admin admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 17:07:21.798086 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1acef4a4-9ba7-4eee-ad46-1c27b6172506 admin admin] PATH_INFO: `/v3/projects/861d785b831c46498161891b2a490c12` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 17:07:21.799265 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1acef4a4-9ba7-4eee-ad46-1c27b6172506 admin admin] RBAC: Authorizing `identity:delete_project(project_id=861d785b831c46498161891b2a490c12)` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 17:07:21.801178 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1acef4a4-9ba7-4eee-ad46-1c27b6172506 admin admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 17:07:21.916242 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.notifications [None req-1acef4a4-9ba7-4eee-ad46-1c27b6172506 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '861d785b831c46498161891b2a490c12', 'request_id': 'req-1acef4a4-9ba7-4eee-ad46-1c27b6172506'} {{(pid=87058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 17:07:21.924024 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.notifications [None req-1acef4a4-9ba7-4eee-ad46-1c27b6172506 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '861d785b831c46498161891b2a490c12', 'request_id': 'req-1acef4a4-9ba7-4eee-ad46-1c27b6172506'} {{(pid=87058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 17:07:21.931449 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.notifications [None req-1acef4a4-9ba7-4eee-ad46-1c27b6172506 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '861d785b831c46498161891b2a490c12', 'request_id': 'req-1acef4a4-9ba7-4eee-ad46-1c27b6172506'} {{(pid=87058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 17:07:22.031475 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.notifications [None req-1acef4a4-9ba7-4eee-ad46-1c27b6172506 admin admin] The token cache is being invalidate because project 861d785b831c46498161891b2a490c12 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87058) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 17:07:22.031682 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.notifications [None req-1acef4a4-9ba7-4eee-ad46-1c27b6172506 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 17:07:22.032442 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.notifications [None req-1acef4a4-9ba7-4eee-ad46-1c27b6172506 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 17:07:22.033903 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 1323/2634] 10.222.0.20 () {58 vars in 1233 bytes} [Mon Aug 1 17:07:21 2022] DELETE /identity/v3/projects/861d785b831c46498161891b2a490c12 => generated 0 bytes in 258 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 17:07:22.038599 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1acef4a4-9ba7-4eee-ad46-1c27b6172506 admin admin] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 17:07:22.039276 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d164e687-93fd-441d-9249-56486f4a5ea3 None None] REQUEST_METHOD: `POST` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 17:07:22.039729 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d164e687-93fd-441d-9249-56486f4a5ea3 None None] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 17:07:22.041021 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d164e687-93fd-441d-9249-56486f4a5ea3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 17:07:22.041145 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.fernet_utils [None req-1acef4a4-9ba7-4eee-ad46-1c27b6172506 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 17:07:22.061599 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.auth.core [None req-d164e687-93fd-441d-9249-56486f4a5ea3 None None] MFA Rules not processed for user `1007f8e379b74e169b98fb7c3c0ba5b4`. Rule list: `[]` (Enabled: `True`). {{(pid=87057) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 17:07:22.063377 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.fernet_utils [None req-d164e687-93fd-441d-9249-56486f4a5ea3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87057) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 17:07:22.111951 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1acef4a4-9ba7-4eee-ad46-1c27b6172506 admin admin] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 17:07:22.117004 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e41ee34-7ec7-4a12-9b57-4481f193fb3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 17:07:22.117648 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e41ee34-7ec7-4a12-9b57-4481f193fb3f admin admin] REQUEST_METHOD: `DELETE` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 17:07:22.117932 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e41ee34-7ec7-4a12-9b57-4481f193fb3f admin admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 17:07:22.118242 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e41ee34-7ec7-4a12-9b57-4481f193fb3f admin admin] PATH_INFO: `/v3/users/48cc0715e7b04f8a8c9f0d8c455bd4c6` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 17:07:22.126849 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 1312/2635] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 1 17:07:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 17:07:22.133411 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e41ee34-7ec7-4a12-9b57-4481f193fb3f admin admin] RBAC: Authorizing `identity:delete_user(user_id=48cc0715e7b04f8a8c9f0d8c455bd4c6)` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 17:07:22.134823 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036b2dfc-6887-4267-aa06-1447f27841a1 None None] REQUEST_METHOD: `POST` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 17:07:22.135115 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036b2dfc-6887-4267-aa06-1447f27841a1 None None] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 17:07:22.137694 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036b2dfc-6887-4267-aa06-1447f27841a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 17:07:22.137809 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e41ee34-7ec7-4a12-9b57-4481f193fb3f admin admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 17:07:22.155381 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.auth.core [None req-036b2dfc-6887-4267-aa06-1447f27841a1 None None] MFA Rules not processed for user `1007f8e379b74e169b98fb7c3c0ba5b4`. Rule list: `[]` (Enabled: `True`). {{(pid=87057) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 17:07:22.156630 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.fernet_utils [None req-036b2dfc-6887-4267-aa06-1447f27841a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87057) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 17:07:22.158024 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG dbcounter [-] [87058] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 17:07:22.165338 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 1313/2636] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 1 17:07:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 17:07:22.169519 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-036b2dfc-6887-4267-aa06-1447f27841a1 None None] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 17:07:22.178644 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-036b2dfc-6887-4267-aa06-1447f27841a1 None None] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 17:07:22.183826 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-972f28fd-5799-4156-8ae7-d30282e3fb06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 17:07:22.184417 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972f28fd-5799-4156-8ae7-d30282e3fb06 admin admin] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 17:07:22.184690 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972f28fd-5799-4156-8ae7-d30282e3fb06 admin admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 17:07:22.184972 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972f28fd-5799-4156-8ae7-d30282e3fb06 admin admin] PATH_INFO: `/v3/domains` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 17:07:22.185665 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-972f28fd-5799-4156-8ae7-d30282e3fb06 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87057) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 17:07:22.185971 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-972f28fd-5799-4156-8ae7-d30282e3fb06 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 17:07:22.187604 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-972f28fd-5799-4156-8ae7-d30282e3fb06 admin admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 17:07:22.199969 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 1314/2637] 10.222.0.20 () {58 vars in 1153 bytes} [Mon Aug 1 17:07:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 17:07:22.204210 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-972f28fd-5799-4156-8ae7-d30282e3fb06 admin admin] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 17:07:22.214320 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-972f28fd-5799-4156-8ae7-d30282e3fb06 admin admin] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 17:07:22.219060 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8318d7b1-f8dc-4b42-8ece-bf7a5d5ac88b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 17:07:22.219646 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8318d7b1-f8dc-4b42-8ece-bf7a5d5ac88b admin admin] REQUEST_METHOD: `POST` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 17:07:22.219931 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8318d7b1-f8dc-4b42-8ece-bf7a5d5ac88b admin admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 17:07:22.220205 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8318d7b1-f8dc-4b42-8ece-bf7a5d5ac88b admin admin] PATH_INFO: `/v3/projects` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 17:07:22.220858 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8318d7b1-f8dc-4b42-8ece-bf7a5d5ac88b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 17:07:22.222779 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8318d7b1-f8dc-4b42-8ece-bf7a5d5ac88b admin admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 17:07:22.289876 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.notifications [None req-5e41ee34-7ec7-4a12-9b57-4481f193fb3f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '48cc0715e7b04f8a8c9f0d8c455bd4c6'} {{(pid=87058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 17:07:22.290701 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.notifications [None req-5e41ee34-7ec7-4a12-9b57-4481f193fb3f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '48cc0715e7b04f8a8c9f0d8c455bd4c6'} {{(pid=87058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 17:07:22.291319 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.notifications [None req-5e41ee34-7ec7-4a12-9b57-4481f193fb3f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '48cc0715e7b04f8a8c9f0d8c455bd4c6'} {{(pid=87058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 17:07:22.300876 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 1315/2638] 10.222.0.20 () {60 vars in 1153 bytes} [Mon Aug 1 17:07:22 2022] POST /identity/v3/projects => generated 371 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 17:07:22.305687 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8318d7b1-f8dc-4b42-8ece-bf7a5d5ac88b admin admin] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 17:07:22.307329 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.fernet_utils [None req-8318d7b1-f8dc-4b42-8ece-bf7a5d5ac88b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87057) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 17:07:22.309017 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.notifications [None req-5e41ee34-7ec7-4a12-9b57-4481f193fb3f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '48cc0715e7b04f8a8c9f0d8c455bd4c6'} {{(pid=87058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 17:07:22.322315 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.notifications [None req-5e41ee34-7ec7-4a12-9b57-4481f193fb3f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '48cc0715e7b04f8a8c9f0d8c455bd4c6'} {{(pid=87058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 17:07:22.380950 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8318d7b1-f8dc-4b42-8ece-bf7a5d5ac88b admin admin] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 17:07:22.380950 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66422b05-2eac-45f5-b1ca-5675b3a581a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 17:07:22.380950 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66422b05-2eac-45f5-b1ca-5675b3a581a3 admin admin] REQUEST_METHOD: `POST` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 17:07:22.380950 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66422b05-2eac-45f5-b1ca-5675b3a581a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 17:07:22.383026 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66422b05-2eac-45f5-b1ca-5675b3a581a3 admin admin] PATH_INFO: `/v3/users` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 17:07:22.383026 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66422b05-2eac-45f5-b1ca-5675b3a581a3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 17:07:22.383697 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66422b05-2eac-45f5-b1ca-5675b3a581a3 admin admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 17:07:22.392723 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG dbcounter [-] [87057] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=87057) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 17:07:22.499347 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 1324/2639] 10.222.0.20 () {58 vars in 1224 bytes} [Mon Aug 1 17:07:22 2022] DELETE /identity/v3/users/48cc0715e7b04f8a8c9f0d8c455bd4c6 => generated 0 bytes in 461 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 17:07:22.512189 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 1316/2640] 10.222.0.20 () {60 vars in 1144 bytes} [Mon Aug 1 17:07:22 2022] POST /identity/v3/users => generated 286 bytes in 207 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 17:07:22.520063 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e41ee34-7ec7-4a12-9b57-4481f193fb3f admin admin] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 17:07:22.584016 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e41ee34-7ec7-4a12-9b57-4481f193fb3f admin admin] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 17:07:22.589444 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f330187-177f-457e-84c6-647617a36b9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 17:07:22.590088 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f330187-177f-457e-84c6-647617a36b9c admin admin] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 17:07:22.590370 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f330187-177f-457e-84c6-647617a36b9c admin admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 17:07:22.590642 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f330187-177f-457e-84c6-647617a36b9c admin admin] PATH_INFO: `/v3/roles` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 17:07:22.591333 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f330187-177f-457e-84c6-647617a36b9c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 17:07:22.592921 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f330187-177f-457e-84c6-647617a36b9c admin admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 17:07:22.604102 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 1325/2641] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Aug 1 17:07:22 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 17:07:22.608258 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66422b05-2eac-45f5-b1ca-5675b3a581a3 admin admin] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 17:07:22.618761 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66422b05-2eac-45f5-b1ca-5675b3a581a3 admin admin] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 17:07:22.623531 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35cbef09-d7d4-4d00-81fe-1f4393f88872 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 17:07:22.624156 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35cbef09-d7d4-4d00-81fe-1f4393f88872 admin admin] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 17:07:22.624443 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35cbef09-d7d4-4d00-81fe-1f4393f88872 admin admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 17:07:22.624715 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35cbef09-d7d4-4d00-81fe-1f4393f88872 admin admin] PATH_INFO: `/v3/roles` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 17:07:22.625413 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35cbef09-d7d4-4d00-81fe-1f4393f88872 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 17:07:22.627089 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35cbef09-d7d4-4d00-81fe-1f4393f88872 admin admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 17:07:22.643229 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 1317/2642] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Aug 1 17:07:22 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 17:07:22.644396 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f330187-177f-457e-84c6-647617a36b9c admin admin] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 17:07:22.655634 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f330187-177f-457e-84c6-647617a36b9c admin admin] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 17:07:22.660380 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f31b0b56-c293-41c6-a2ab-560740b2cad7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 17:07:22.661041 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31b0b56-c293-41c6-a2ab-560740b2cad7 admin admin] REQUEST_METHOD: `PUT` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 17:07:22.661324 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31b0b56-c293-41c6-a2ab-560740b2cad7 admin admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 17:07:22.661601 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31b0b56-c293-41c6-a2ab-560740b2cad7 admin admin] PATH_INFO: `/v3/projects/536501fe636446debe5097d1ea57b4dd/users/1b8fdb1c491f480eb0d9d2f39e0a3e12/roles/0999d3da63a545f18ff3252afdc90518` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 17:07:22.676488 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f31b0b56-c293-41c6-a2ab-560740b2cad7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=536501fe636446debe5097d1ea57b4dd, user_id=1b8fdb1c491f480eb0d9d2f39e0a3e12, role_id=0999d3da63a545f18ff3252afdc90518)` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 17:07:22.679229 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f31b0b56-c293-41c6-a2ab-560740b2cad7 admin admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 17:07:22.741469 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35cbef09-d7d4-4d00-81fe-1f4393f88872 admin admin] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 17:07:22.743276 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.fernet_utils [None req-35cbef09-d7d4-4d00-81fe-1f4393f88872 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87057) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 17:07:22.752378 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 1326/2643] 10.222.0.20 () {60 vars in 1483 bytes} [Mon Aug 1 17:07:22 2022] PUT /identity/v3/projects/536501fe636446debe5097d1ea57b4dd/users/1b8fdb1c491f480eb0d9d2f39e0a3e12/roles/0999d3da63a545f18ff3252afdc90518 => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 17:07:22.764168 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e63dbd3-7a91-45b9-ae06-3274f244353c None None] REQUEST_METHOD: `POST` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 17:07:22.769526 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e63dbd3-7a91-45b9-ae06-3274f244353c None None] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 17:07:22.769526 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e63dbd3-7a91-45b9-ae06-3274f244353c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 17:07:22.769793 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35cbef09-d7d4-4d00-81fe-1f4393f88872 admin admin] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 17:07:22.800288 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.auth.core [None req-0e63dbd3-7a91-45b9-ae06-3274f244353c None None] MFA Rules not processed for user `1b8fdb1c491f480eb0d9d2f39e0a3e12`. Rule list: `[]` (Enabled: `True`). {{(pid=87058) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 17:07:22.801602 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.fernet_utils [None req-0e63dbd3-7a91-45b9-ae06-3274f244353c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 17:07:22.830205 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc9900b4-e45b-45d1-bcdc-c5c33993e56c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 17:07:22.830855 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9900b4-e45b-45d1-bcdc-c5c33993e56c admin admin] REQUEST_METHOD: `DELETE` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 17:07:22.831249 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9900b4-e45b-45d1-bcdc-c5c33993e56c admin admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 17:07:22.831531 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9900b4-e45b-45d1-bcdc-c5c33993e56c admin admin] PATH_INFO: `/v3/projects/dfc1c3badb144d24a2b86341a155505a` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 17:07:22.842165 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc9900b4-e45b-45d1-bcdc-c5c33993e56c admin admin] RBAC: Authorizing `identity:delete_project(project_id=dfc1c3badb144d24a2b86341a155505a)` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 17:07:22.844098 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc9900b4-e45b-45d1-bcdc-c5c33993e56c admin admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 17:07:22.863647 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG dbcounter [-] [87058] Writing DB stats keystone:DELETE=7,keystone:SELECT=91,keystone:INSERT=2 {{(pid=87058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 17:07:22.867924 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 1327/2644] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 1 17:07:22 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 17:07:22.878900 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e63dbd3-7a91-45b9-ae06-3274f244353c None None] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 17:07:22.883890 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.fernet_utils [None req-0e63dbd3-7a91-45b9-ae06-3274f244353c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 17:07:22.939543 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e63dbd3-7a91-45b9-ae06-3274f244353c None None] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 17:07:22.955144 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.notifications [None req-dc9900b4-e45b-45d1-bcdc-c5c33993e56c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dfc1c3badb144d24a2b86341a155505a', 'request_id': 'req-dc9900b4-e45b-45d1-bcdc-c5c33993e56c'} {{(pid=87057) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 17:07:22.962255 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.notifications [None req-dc9900b4-e45b-45d1-bcdc-c5c33993e56c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dfc1c3badb144d24a2b86341a155505a', 'request_id': 'req-dc9900b4-e45b-45d1-bcdc-c5c33993e56c'} {{(pid=87057) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 17:07:22.969261 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.notifications [None req-dc9900b4-e45b-45d1-bcdc-c5c33993e56c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dfc1c3badb144d24a2b86341a155505a', 'request_id': 'req-dc9900b4-e45b-45d1-bcdc-c5c33993e56c'} {{(pid=87057) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 17:07:22.989531 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc8b32b5-130e-41f7-9bc7-37e457043477 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25970b52ca15448fa1abb1f974d64038', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b992d6a65df4bf9b27c13192d57c5f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 17:07:22.990164 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8b32b5-130e-41f7-9bc7-37e457043477 service cinder] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 17:07:22.990447 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8b32b5-130e-41f7-9bc7-37e457043477 service cinder] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 17:07:22.990723 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8b32b5-130e-41f7-9bc7-37e457043477 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 17:07:23.025612 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.notifications [None req-dc9900b4-e45b-45d1-bcdc-c5c33993e56c admin admin] The token cache is being invalidate because project dfc1c3badb144d24a2b86341a155505a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87057) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 17:07:23.025991 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.notifications [None req-dc9900b4-e45b-45d1-bcdc-c5c33993e56c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87057) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 17:07:23.026658 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.notifications [None req-dc9900b4-e45b-45d1-bcdc-c5c33993e56c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87057) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 17:07:23.028267 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 1318/2645] 10.222.0.20 () {58 vars in 1233 bytes} [Mon Aug 1 17:07:22 2022] DELETE /identity/v3/projects/dfc1c3badb144d24a2b86341a155505a => generated 0 bytes in 288 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 17:07:23.033336 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52bc2c15-4f6c-42d4-8848-eca2816fd7b4 None None] REQUEST_METHOD: `POST` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 17:07:23.033618 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52bc2c15-4f6c-42d4-8848-eca2816fd7b4 None None] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 17:07:23.033888 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52bc2c15-4f6c-42d4-8848-eca2816fd7b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 17:07:23.048058 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc8b32b5-130e-41f7-9bc7-37e457043477 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 17:07:23.049894 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc8b32b5-130e-41f7-9bc7-37e457043477 service cinder] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 17:07:23.055020 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.auth.core [None req-52bc2c15-4f6c-42d4-8848-eca2816fd7b4 None None] MFA Rules not processed for user `1007f8e379b74e169b98fb7c3c0ba5b4`. Rule list: `[]` (Enabled: `True`). {{(pid=87057) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 17:07:23.056392 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.fernet_utils [None req-52bc2c15-4f6c-42d4-8848-eca2816fd7b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87057) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 17:07:23.058732 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 1328/2646] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 1 17:07:22 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 180 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 01 17:07:23.083904 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc8b32b5-130e-41f7-9bc7-37e457043477 service cinder] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 17:07:23.119585 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 1319/2647] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 1 17:07:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 17:07:23.127062 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f643593-f5aa-447d-9eb8-40d6f764452c None None] REQUEST_METHOD: `POST` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 17:07:23.136419 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f643593-f5aa-447d-9eb8-40d6f764452c None None] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 17:07:23.136419 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f643593-f5aa-447d-9eb8-40d6f764452c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 17:07:23.136419 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG dbcounter [-] [87057] Writing DB stats keystone:INSERT=3,keystone:SELECT=90,keystone:DELETE=7 {{(pid=87057) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 17:07:23.148044 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.auth.core [None req-5f643593-f5aa-447d-9eb8-40d6f764452c None None] MFA Rules not processed for user `1007f8e379b74e169b98fb7c3c0ba5b4`. Rule list: `[]` (Enabled: `True`). {{(pid=87057) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 17:07:23.149269 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.fernet_utils [None req-5f643593-f5aa-447d-9eb8-40d6f764452c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87057) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 17:07:23.149655 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc8b32b5-130e-41f7-9bc7-37e457043477 service cinder] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 17:07:23.154296 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df82741e-d25b-4a28-bb65-8606806dc5c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 17:07:23.154986 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df82741e-d25b-4a28-bb65-8606806dc5c9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 17:07:23.155264 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df82741e-d25b-4a28-bb65-8606806dc5c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 17:07:23.155538 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df82741e-d25b-4a28-bb65-8606806dc5c9 admin admin] PATH_INFO: `/v3/users/1b8fdb1c491f480eb0d9d2f39e0a3e12` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 17:07:23.156702 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df82741e-d25b-4a28-bb65-8606806dc5c9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1b8fdb1c491f480eb0d9d2f39e0a3e12)` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 17:07:23.157402 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 1320/2648] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 1 17:07:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 17:07:23.158706 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df82741e-d25b-4a28-bb65-8606806dc5c9 admin admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 17:07:23.161380 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f643593-f5aa-447d-9eb8-40d6f764452c None None] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 17:07:23.171675 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f643593-f5aa-447d-9eb8-40d6f764452c None None] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 17:07:23.172316 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG dbcounter [-] [87058] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=87058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 17:07:23.177728 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3033adae-92ee-4cc1-9038-6fb07deae87f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 17:07:23.178451 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3033adae-92ee-4cc1-9038-6fb07deae87f admin admin] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 17:07:23.178729 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3033adae-92ee-4cc1-9038-6fb07deae87f admin admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 17:07:23.179002 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3033adae-92ee-4cc1-9038-6fb07deae87f admin admin] PATH_INFO: `/v3/domains` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 17:07:23.179698 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3033adae-92ee-4cc1-9038-6fb07deae87f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87057) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 17:07:23.179994 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3033adae-92ee-4cc1-9038-6fb07deae87f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 17:07:23.181594 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3033adae-92ee-4cc1-9038-6fb07deae87f admin admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 17:07:23.193234 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 1321/2649] 10.222.0.20 () {58 vars in 1153 bytes} [Mon Aug 1 17:07:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 17:07:23.202670 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3033adae-92ee-4cc1-9038-6fb07deae87f admin admin] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 17:07:23.212654 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3033adae-92ee-4cc1-9038-6fb07deae87f admin admin] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 17:07:23.217358 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fa51ad6-6a14-4b69-80b8-b3a256d42934 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 17:07:23.218012 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa51ad6-6a14-4b69-80b8-b3a256d42934 admin admin] REQUEST_METHOD: `POST` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 17:07:23.218315 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa51ad6-6a14-4b69-80b8-b3a256d42934 admin admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 17:07:23.218584 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa51ad6-6a14-4b69-80b8-b3a256d42934 admin admin] PATH_INFO: `/v3/projects` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 17:07:23.219228 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fa51ad6-6a14-4b69-80b8-b3a256d42934 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 17:07:23.221093 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fa51ad6-6a14-4b69-80b8-b3a256d42934 admin admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 17:07:23.341430 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 1322/2650] 10.222.0.20 () {60 vars in 1153 bytes} [Mon Aug 1 17:07:23 2022] POST /identity/v3/projects => generated 373 bytes in 139 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 17:07:23.345510 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fa51ad6-6a14-4b69-80b8-b3a256d42934 admin admin] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 17:07:23.360231 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.notifications [None req-df82741e-d25b-4a28-bb65-8606806dc5c9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1b8fdb1c491f480eb0d9d2f39e0a3e12'} {{(pid=87058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 17:07:23.361192 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.notifications [None req-df82741e-d25b-4a28-bb65-8606806dc5c9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1b8fdb1c491f480eb0d9d2f39e0a3e12'} {{(pid=87058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 17:07:23.361886 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.notifications [None req-df82741e-d25b-4a28-bb65-8606806dc5c9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1b8fdb1c491f480eb0d9d2f39e0a3e12'} {{(pid=87058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 17:07:23.374518 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.notifications [None req-df82741e-d25b-4a28-bb65-8606806dc5c9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1b8fdb1c491f480eb0d9d2f39e0a3e12'} {{(pid=87058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 17:07:23.382544 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.notifications [None req-df82741e-d25b-4a28-bb65-8606806dc5c9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1b8fdb1c491f480eb0d9d2f39e0a3e12'} {{(pid=87058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 17:07:23.411120 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fa51ad6-6a14-4b69-80b8-b3a256d42934 admin admin] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 17:07:23.414128 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd68478-1c7f-4051-a7e2-d2825e2e2f17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 17:07:23.414744 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd68478-1c7f-4051-a7e2-d2825e2e2f17 admin admin] REQUEST_METHOD: `POST` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 17:07:23.415024 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd68478-1c7f-4051-a7e2-d2825e2e2f17 admin admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 17:07:23.415297 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd68478-1c7f-4051-a7e2-d2825e2e2f17 admin admin] PATH_INFO: `/v3/users` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 17:07:23.415966 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cd68478-1c7f-4051-a7e2-d2825e2e2f17 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 17:07:23.417925 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cd68478-1c7f-4051-a7e2-d2825e2e2f17 admin admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 17:07:23.472955 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 1329/2651] 10.222.0.20 () {58 vars in 1224 bytes} [Mon Aug 1 17:07:23 2022] DELETE /identity/v3/users/1b8fdb1c491f480eb0d9d2f39e0a3e12 => generated 0 bytes in 390 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 17:07:23.478337 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079d29aa-5fa7-42fa-8110-000dfba933de None None] REQUEST_METHOD: `POST` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 17:07:23.478337 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079d29aa-5fa7-42fa-8110-000dfba933de None None] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 17:07:23.478337 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079d29aa-5fa7-42fa-8110-000dfba933de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 17:07:23.496452 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.auth.core [None req-079d29aa-5fa7-42fa-8110-000dfba933de None None] MFA Rules not processed for user `1007f8e379b74e169b98fb7c3c0ba5b4`. Rule list: `[]` (Enabled: `True`). {{(pid=87058) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 17:07:23.497659 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.fernet_utils [None req-079d29aa-5fa7-42fa-8110-000dfba933de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 17:07:23.532550 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 1323/2652] 10.222.0.20 () {60 vars in 1144 bytes} [Mon Aug 1 17:07:23 2022] POST /identity/v3/users => generated 287 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 17:07:23.533624 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd68478-1c7f-4051-a7e2-d2825e2e2f17 admin admin] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 17:07:23.535392 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.fernet_utils [None req-5cd68478-1c7f-4051-a7e2-d2825e2e2f17 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87057) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 17:07:23.548397 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd68478-1c7f-4051-a7e2-d2825e2e2f17 admin admin] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 17:07:23.557496 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 1330/2653] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 1 17:07:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 17:07:23.567012 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-079d29aa-5fa7-42fa-8110-000dfba933de None None] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 17:07:23.568367 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.fernet_utils [None req-079d29aa-5fa7-42fa-8110-000dfba933de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 17:07:23.606838 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c16ca1e0-c8a7-4d72-a00a-b00d1482996f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 17:07:23.607493 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16ca1e0-c8a7-4d72-a00a-b00d1482996f admin admin] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 17:07:23.607775 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16ca1e0-c8a7-4d72-a00a-b00d1482996f admin admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 17:07:23.608068 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16ca1e0-c8a7-4d72-a00a-b00d1482996f admin admin] PATH_INFO: `/v3/roles` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 17:07:23.608767 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c16ca1e0-c8a7-4d72-a00a-b00d1482996f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 17:07:23.610411 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c16ca1e0-c8a7-4d72-a00a-b00d1482996f admin admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 17:07:23.620573 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 1324/2654] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Aug 1 17:07:23 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 17:07:23.624497 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c16ca1e0-c8a7-4d72-a00a-b00d1482996f admin admin] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 17:07:23.628870 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-079d29aa-5fa7-42fa-8110-000dfba933de None None] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 17:07:23.633347 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77631ed5-9611-4528-be48-b84ab717e5a0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d9bd8d649154c979be3744bfd016f95', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b992d6a65df4bf9b27c13192d57c5f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 17:07:23.633966 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77631ed5-9611-4528-be48-b84ab717e5a0 service neutron] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 17:07:23.634287 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77631ed5-9611-4528-be48-b84ab717e5a0 service neutron] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 17:07:23.634564 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77631ed5-9611-4528-be48-b84ab717e5a0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 17:07:23.635377 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c16ca1e0-c8a7-4d72-a00a-b00d1482996f admin admin] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 17:07:23.639900 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba71b25-598d-4cd8-a7e6-e1eda26fffb0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 17:07:23.640489 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba71b25-598d-4cd8-a7e6-e1eda26fffb0 admin admin] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 17:07:23.640763 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba71b25-598d-4cd8-a7e6-e1eda26fffb0 admin admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 17:07:23.641034 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba71b25-598d-4cd8-a7e6-e1eda26fffb0 admin admin] PATH_INFO: `/v3/roles` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 17:07:23.641719 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ba71b25-598d-4cd8-a7e6-e1eda26fffb0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 17:07:23.643322 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ba71b25-598d-4cd8-a7e6-e1eda26fffb0 admin admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 17:07:23.643817 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77631ed5-9611-4528-be48-b84ab717e5a0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 17:07:23.645254 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77631ed5-9611-4528-be48-b84ab717e5a0 service neutron] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 17:07:23.654198 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 1325/2655] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Aug 1 17:07:23 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 01 17:07:23.655763 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 1331/2656] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 1 17:07:23 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 01 17:07:23.658195 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba71b25-598d-4cd8-a7e6-e1eda26fffb0 admin admin] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 17:07:23.668607 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba71b25-598d-4cd8-a7e6-e1eda26fffb0 admin admin] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 17:07:23.673193 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-309990d1-3755-49c8-868b-b6903bcd320e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 17:07:23.673842 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309990d1-3755-49c8-868b-b6903bcd320e admin admin] REQUEST_METHOD: `PUT` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 17:07:23.674152 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309990d1-3755-49c8-868b-b6903bcd320e admin admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 17:07:23.674425 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309990d1-3755-49c8-868b-b6903bcd320e admin admin] PATH_INFO: `/v3/projects/72831c952b8a4ede9f305c94c597a4fc/users/d283bc9875454ebbad389e12fa19a3c1/roles/0999d3da63a545f18ff3252afdc90518` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 17:07:23.690340 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-309990d1-3755-49c8-868b-b6903bcd320e admin admin] RBAC: Authorizing `identity:create_grant(project_id=72831c952b8a4ede9f305c94c597a4fc, user_id=d283bc9875454ebbad389e12fa19a3c1, role_id=0999d3da63a545f18ff3252afdc90518)` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 17:07:23.693017 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-309990d1-3755-49c8-868b-b6903bcd320e admin admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 17:07:23.785672 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 1326/2657] 10.222.0.20 () {60 vars in 1483 bytes} [Mon Aug 1 17:07:23 2022] PUT /identity/v3/projects/72831c952b8a4ede9f305c94c597a4fc/users/d283bc9875454ebbad389e12fa19a3c1/roles/0999d3da63a545f18ff3252afdc90518 => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 17:07:23.796667 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf8035d-eb62-4a44-ad4f-0518104ee870 None None] REQUEST_METHOD: `POST` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 17:07:23.796956 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf8035d-eb62-4a44-ad4f-0518104ee870 None None] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 17:07:23.797233 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf8035d-eb62-4a44-ad4f-0518104ee870 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 17:07:23.829415 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.auth.core [None req-bcf8035d-eb62-4a44-ad4f-0518104ee870 None None] MFA Rules not processed for user `d283bc9875454ebbad389e12fa19a3c1`. Rule list: `[]` (Enabled: `True`). {{(pid=87058) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 17:07:23.830937 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.fernet_utils [None req-bcf8035d-eb62-4a44-ad4f-0518104ee870 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 17:07:23.852941 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG dbcounter [-] [87058] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=87058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 17:07:23.902076 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 1332/2658] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 1 17:07:23 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 17:07:23.913330 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-309990d1-3755-49c8-868b-b6903bcd320e admin admin] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 17:07:23.915269 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.fernet_utils [None req-309990d1-3755-49c8-868b-b6903bcd320e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87057) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 17:07:23.930265 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG dbcounter [-] [87057] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=87057) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 17:07:23.992971 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-309990d1-3755-49c8-868b-b6903bcd320e admin admin] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 17:07:23.998758 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73fc4dcb-5ad9-4aa2-86d1-e442a8d5cfed service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6533106d03eb4ca185a8839336d2aff9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b992d6a65df4bf9b27c13192d57c5f0', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 17:07:23.999385 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73fc4dcb-5ad9-4aa2-86d1-e442a8d5cfed service nova] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 17:07:23.999667 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73fc4dcb-5ad9-4aa2-86d1-e442a8d5cfed service nova] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 17:07:23.999942 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73fc4dcb-5ad9-4aa2-86d1-e442a8d5cfed service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 17:07:24.008382 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73fc4dcb-5ad9-4aa2-86d1-e442a8d5cfed service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 17:07:24.010451 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73fc4dcb-5ad9-4aa2-86d1-e442a8d5cfed service nova] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 17:07:24.018841 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 1327/2659] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 1 17:07:23 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 17:07:24.097612 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf8035d-eb62-4a44-ad4f-0518104ee870 None None] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 17:07:24.098963 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.fernet_utils [None req-bcf8035d-eb62-4a44-ad4f-0518104ee870 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 17:07:24.183552 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf8035d-eb62-4a44-ad4f-0518104ee870 None None] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 17:07:24.188434 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-423e01cf-9aff-49a8-890e-f3bfd1b7e9d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 17:07:24.189106 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423e01cf-9aff-49a8-890e-f3bfd1b7e9d6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 17:07:24.189594 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423e01cf-9aff-49a8-890e-f3bfd1b7e9d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 17:07:24.189873 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423e01cf-9aff-49a8-890e-f3bfd1b7e9d6 admin admin] PATH_INFO: `/v3/projects/536501fe636446debe5097d1ea57b4dd` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 17:07:24.191103 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-423e01cf-9aff-49a8-890e-f3bfd1b7e9d6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=536501fe636446debe5097d1ea57b4dd)` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 17:07:24.193278 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-423e01cf-9aff-49a8-890e-f3bfd1b7e9d6 admin admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 17:07:24.251906 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73fc4dcb-5ad9-4aa2-86d1-e442a8d5cfed service nova] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 17:07:24.262996 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73fc4dcb-5ad9-4aa2-86d1-e442a8d5cfed service nova] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 17:07:24.267729 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a86ea25c-b7fe-4414-9693-77400e154cd2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 17:07:24.268485 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86ea25c-b7fe-4414-9693-77400e154cd2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 17:07:24.268871 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86ea25c-b7fe-4414-9693-77400e154cd2 admin admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 17:07:24.269155 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86ea25c-b7fe-4414-9693-77400e154cd2 admin admin] PATH_INFO: `/v3/users/d283bc9875454ebbad389e12fa19a3c1` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 17:07:24.270454 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a86ea25c-b7fe-4414-9693-77400e154cd2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d283bc9875454ebbad389e12fa19a3c1)` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 17:07:24.274421 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a86ea25c-b7fe-4414-9693-77400e154cd2 admin admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 17:07:24.311545 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.notifications [None req-423e01cf-9aff-49a8-890e-f3bfd1b7e9d6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '536501fe636446debe5097d1ea57b4dd', 'request_id': 'req-423e01cf-9aff-49a8-890e-f3bfd1b7e9d6'} {{(pid=87058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 17:07:24.317631 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.notifications [None req-423e01cf-9aff-49a8-890e-f3bfd1b7e9d6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '536501fe636446debe5097d1ea57b4dd', 'request_id': 'req-423e01cf-9aff-49a8-890e-f3bfd1b7e9d6'} {{(pid=87058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 17:07:24.323807 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.notifications [None req-423e01cf-9aff-49a8-890e-f3bfd1b7e9d6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '536501fe636446debe5097d1ea57b4dd', 'request_id': 'req-423e01cf-9aff-49a8-890e-f3bfd1b7e9d6'} {{(pid=87058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 17:07:24.407100 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.notifications [None req-423e01cf-9aff-49a8-890e-f3bfd1b7e9d6 admin admin] The token cache is being invalidate because project 536501fe636446debe5097d1ea57b4dd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87058) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 17:07:24.407383 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.notifications [None req-423e01cf-9aff-49a8-890e-f3bfd1b7e9d6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 17:07:24.408032 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.notifications [None req-423e01cf-9aff-49a8-890e-f3bfd1b7e9d6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 17:07:24.409535 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 1333/2660] 10.222.0.20 () {58 vars in 1233 bytes} [Mon Aug 1 17:07:24 2022] DELETE /identity/v3/projects/536501fe636446debe5097d1ea57b4dd => generated 0 bytes in 312 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 17:07:24.447361 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.notifications [None req-a86ea25c-b7fe-4414-9693-77400e154cd2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd283bc9875454ebbad389e12fa19a3c1'} {{(pid=87057) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 17:07:24.447984 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.notifications [None req-a86ea25c-b7fe-4414-9693-77400e154cd2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd283bc9875454ebbad389e12fa19a3c1'} {{(pid=87057) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 17:07:24.494241 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.notifications [None req-a86ea25c-b7fe-4414-9693-77400e154cd2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd283bc9875454ebbad389e12fa19a3c1'} {{(pid=87057) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 17:07:24.494910 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.notifications [None req-a86ea25c-b7fe-4414-9693-77400e154cd2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd283bc9875454ebbad389e12fa19a3c1'} {{(pid=87057) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 17:07:24.503369 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.notifications [None req-a86ea25c-b7fe-4414-9693-77400e154cd2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd283bc9875454ebbad389e12fa19a3c1'} {{(pid=87057) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 17:07:24.523304 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 1328/2661] 10.222.0.20 () {58 vars in 1224 bytes} [Mon Aug 1 17:07:24 2022] DELETE /identity/v3/users/d283bc9875454ebbad389e12fa19a3c1 => generated 0 bytes in 272 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 17:07:24.527931 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f7773c7-35c1-41a5-886a-55c2c577e5ab None None] REQUEST_METHOD: `POST` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 17:07:24.528219 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f7773c7-35c1-41a5-886a-55c2c577e5ab None None] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 17:07:24.528501 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f7773c7-35c1-41a5-886a-55c2c577e5ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 17:07:24.548487 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.auth.core [None req-9f7773c7-35c1-41a5-886a-55c2c577e5ab None None] MFA Rules not processed for user `1007f8e379b74e169b98fb7c3c0ba5b4`. Rule list: `[]` (Enabled: `True`). {{(pid=87058) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 17:07:24.549823 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.fernet_utils [None req-9f7773c7-35c1-41a5-886a-55c2c577e5ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 17:07:24.601231 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG dbcounter [-] [87058] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=87058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 17:07:24.614129 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 1334/2662] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 1 17:07:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 17:07:24.624614 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a86ea25c-b7fe-4414-9693-77400e154cd2 admin admin] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 17:07:24.626118 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.fernet_utils [None req-a86ea25c-b7fe-4414-9693-77400e154cd2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87057) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 17:07:24.689160 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a86ea25c-b7fe-4414-9693-77400e154cd2 admin admin] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 17:07:24.692880 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-652a8cb8-0afd-4b30-aa8b-ccf95e17af67 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d9bd8d649154c979be3744bfd016f95', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b992d6a65df4bf9b27c13192d57c5f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 17:07:24.693498 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652a8cb8-0afd-4b30-aa8b-ccf95e17af67 service neutron] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 17:07:24.693776 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652a8cb8-0afd-4b30-aa8b-ccf95e17af67 service neutron] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 17:07:24.694118 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652a8cb8-0afd-4b30-aa8b-ccf95e17af67 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 17:07:24.703651 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-652a8cb8-0afd-4b30-aa8b-ccf95e17af67 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 17:07:24.705013 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-652a8cb8-0afd-4b30-aa8b-ccf95e17af67 service neutron] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 17:07:24.715326 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 1329/2663] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 1 17:07:24 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 17:07:25.129845 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f7773c7-35c1-41a5-886a-55c2c577e5ab None None] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 17:07:25.131368 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.fernet_utils [None req-9f7773c7-35c1-41a5-886a-55c2c577e5ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 17:07:25.145013 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f7773c7-35c1-41a5-886a-55c2c577e5ab None None] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 17:07:25.149974 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d35ef0-3d9a-46bd-88a7-ada91a5f222b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 17:07:25.150662 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d35ef0-3d9a-46bd-88a7-ada91a5f222b admin admin] REQUEST_METHOD: `DELETE` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 17:07:25.150940 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d35ef0-3d9a-46bd-88a7-ada91a5f222b admin admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 17:07:25.151218 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d35ef0-3d9a-46bd-88a7-ada91a5f222b admin admin] PATH_INFO: `/v3/projects/72831c952b8a4ede9f305c94c597a4fc` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 17:07:25.152389 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94d35ef0-3d9a-46bd-88a7-ada91a5f222b admin admin] RBAC: Authorizing `identity:delete_project(project_id=72831c952b8a4ede9f305c94c597a4fc)` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 17:07:25.154415 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94d35ef0-3d9a-46bd-88a7-ada91a5f222b admin admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 17:07:25.265899 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.notifications [None req-94d35ef0-3d9a-46bd-88a7-ada91a5f222b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '72831c952b8a4ede9f305c94c597a4fc', 'request_id': 'req-94d35ef0-3d9a-46bd-88a7-ada91a5f222b'} {{(pid=87058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 17:07:25.272375 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.notifications [None req-94d35ef0-3d9a-46bd-88a7-ada91a5f222b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '72831c952b8a4ede9f305c94c597a4fc', 'request_id': 'req-94d35ef0-3d9a-46bd-88a7-ada91a5f222b'} {{(pid=87058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 17:07:25.278233 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.notifications [None req-94d35ef0-3d9a-46bd-88a7-ada91a5f222b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '72831c952b8a4ede9f305c94c597a4fc', 'request_id': 'req-94d35ef0-3d9a-46bd-88a7-ada91a5f222b'} {{(pid=87058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 17:07:25.349786 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.notifications [None req-94d35ef0-3d9a-46bd-88a7-ada91a5f222b admin admin] The token cache is being invalidate because project 72831c952b8a4ede9f305c94c597a4fc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87058) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 17:07:25.350100 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.notifications [None req-94d35ef0-3d9a-46bd-88a7-ada91a5f222b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 17:07:25.350718 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.notifications [None req-94d35ef0-3d9a-46bd-88a7-ada91a5f222b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 17:07:25.352356 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 1335/2664] 10.222.0.20 () {58 vars in 1233 bytes} [Mon Aug 1 17:07:25 2022] DELETE /identity/v3/projects/72831c952b8a4ede9f305c94c597a4fc => generated 0 bytes in 223 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 17:07:25.357211 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b7d437-6139-49b0-b09b-232c594f5232 None None] REQUEST_METHOD: `POST` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 17:07:25.357721 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b7d437-6139-49b0-b09b-232c594f5232 None None] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 17:07:25.357721 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b7d437-6139-49b0-b09b-232c594f5232 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 17:07:25.377640 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.auth.core [None req-50b7d437-6139-49b0-b09b-232c594f5232 None None] MFA Rules not processed for user `1007f8e379b74e169b98fb7c3c0ba5b4`. Rule list: `[]` (Enabled: `True`). {{(pid=87057) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 17:07:25.378935 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.fernet_utils [None req-50b7d437-6139-49b0-b09b-232c594f5232 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87057) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 17:07:25.384940 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG dbcounter [-] [87057] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87057) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 17:07:25.438047 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 1330/2665] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 1 17:07:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 17:07:25.447982 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ddabdd-6698-4f24-8653-aef354756781 None None] REQUEST_METHOD: `POST` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 17:07:25.448271 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ddabdd-6698-4f24-8653-aef354756781 None None] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 17:07:25.448549 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ddabdd-6698-4f24-8653-aef354756781 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 17:07:25.468762 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.auth.core [None req-66ddabdd-6698-4f24-8653-aef354756781 None None] MFA Rules not processed for user `1007f8e379b74e169b98fb7c3c0ba5b4`. Rule list: `[]` (Enabled: `True`). {{(pid=87058) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 17:07:25.469966 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.fernet_utils [None req-66ddabdd-6698-4f24-8653-aef354756781 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 17:07:25.478888 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 1336/2666] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 1 17:07:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 17:07:25.483163 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b7d437-6139-49b0-b09b-232c594f5232 None None] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 17:07:25.493734 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b7d437-6139-49b0-b09b-232c594f5232 None None] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 17:07:25.498365 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdfabbae-17be-4686-86c8-0966d7cbd3f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 17:07:25.498971 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdfabbae-17be-4686-86c8-0966d7cbd3f0 admin admin] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 17:07:25.499252 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdfabbae-17be-4686-86c8-0966d7cbd3f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 17:07:25.499526 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdfabbae-17be-4686-86c8-0966d7cbd3f0 admin admin] PATH_INFO: `/v3/domains` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 17:07:25.500218 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdfabbae-17be-4686-86c8-0966d7cbd3f0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87057) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 17:07:25.500527 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdfabbae-17be-4686-86c8-0966d7cbd3f0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 17:07:25.502153 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdfabbae-17be-4686-86c8-0966d7cbd3f0 admin admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 17:07:25.513409 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 1331/2667] 10.222.0.20 () {58 vars in 1153 bytes} [Mon Aug 1 17:07:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 17:07:25.517571 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ddabdd-6698-4f24-8653-aef354756781 None None] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 17:07:25.527309 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ddabdd-6698-4f24-8653-aef354756781 None None] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 17:07:25.531940 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87146a6f-385b-4b9d-a5f5-ff73838e4999 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 17:07:25.532533 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87146a6f-385b-4b9d-a5f5-ff73838e4999 admin admin] REQUEST_METHOD: `POST` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 17:07:25.532804 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87146a6f-385b-4b9d-a5f5-ff73838e4999 admin admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 17:07:25.533076 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87146a6f-385b-4b9d-a5f5-ff73838e4999 admin admin] PATH_INFO: `/v3/projects` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 17:07:25.533766 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87146a6f-385b-4b9d-a5f5-ff73838e4999 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 17:07:25.535687 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87146a6f-385b-4b9d-a5f5-ff73838e4999 admin admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 17:07:25.614692 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 1337/2668] 10.222.0.20 () {60 vars in 1153 bytes} [Mon Aug 1 17:07:25 2022] POST /identity/v3/projects => generated 363 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 17:07:25.618954 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdfabbae-17be-4686-86c8-0966d7cbd3f0 admin admin] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 17:07:25.628887 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdfabbae-17be-4686-86c8-0966d7cbd3f0 admin admin] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 17:07:25.685731 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6086616-9a2c-4da5-a3f0-8e491467ad2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 17:07:25.686369 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6086616-9a2c-4da5-a3f0-8e491467ad2f admin admin] REQUEST_METHOD: `POST` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 17:07:25.686648 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6086616-9a2c-4da5-a3f0-8e491467ad2f admin admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 17:07:25.686920 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6086616-9a2c-4da5-a3f0-8e491467ad2f admin admin] PATH_INFO: `/v3/users` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 17:07:25.687612 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6086616-9a2c-4da5-a3f0-8e491467ad2f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 17:07:25.689565 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6086616-9a2c-4da5-a3f0-8e491467ad2f admin admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 17:07:25.813585 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 1332/2669] 10.222.0.20 () {60 vars in 1144 bytes} [Mon Aug 1 17:07:25 2022] POST /identity/v3/users => generated 282 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 17:07:25.817742 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87146a6f-385b-4b9d-a5f5-ff73838e4999 admin admin] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 17:07:25.829079 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87146a6f-385b-4b9d-a5f5-ff73838e4999 admin admin] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 17:07:25.833797 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8fa93a0-4c6f-4143-9de6-82d5f4870bfc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 17:07:25.834493 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8fa93a0-4c6f-4143-9de6-82d5f4870bfc admin admin] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 17:07:25.834798 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8fa93a0-4c6f-4143-9de6-82d5f4870bfc admin admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 17:07:25.835075 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8fa93a0-4c6f-4143-9de6-82d5f4870bfc admin admin] PATH_INFO: `/v3/roles` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 17:07:25.835817 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8fa93a0-4c6f-4143-9de6-82d5f4870bfc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 17:07:25.837949 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8fa93a0-4c6f-4143-9de6-82d5f4870bfc admin admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 17:07:25.847821 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 1338/2670] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Aug 1 17:07:25 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 17:07:25.851692 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6086616-9a2c-4da5-a3f0-8e491467ad2f admin admin] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 17:07:25.861606 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6086616-9a2c-4da5-a3f0-8e491467ad2f admin admin] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 17:07:25.866584 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54ec9a9b-7d89-4db7-9a42-5bd86b8425a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 17:07:25.867309 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ec9a9b-7d89-4db7-9a42-5bd86b8425a1 admin admin] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 17:07:25.867591 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ec9a9b-7d89-4db7-9a42-5bd86b8425a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 17:07:25.867870 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ec9a9b-7d89-4db7-9a42-5bd86b8425a1 admin admin] PATH_INFO: `/v3/roles` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 17:07:25.868565 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54ec9a9b-7d89-4db7-9a42-5bd86b8425a1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 17:07:25.870236 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54ec9a9b-7d89-4db7-9a42-5bd86b8425a1 admin admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 17:07:25.880809 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 1333/2671] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Aug 1 17:07:25 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 01 17:07:25.885431 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8fa93a0-4c6f-4143-9de6-82d5f4870bfc admin admin] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 17:07:25.895553 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8fa93a0-4c6f-4143-9de6-82d5f4870bfc admin admin] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 17:07:25.900837 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5ef7711-f84c-4af2-a581-74142dc34458 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 17:07:25.901692 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ef7711-f84c-4af2-a581-74142dc34458 admin admin] REQUEST_METHOD: `PUT` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 17:07:25.901972 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ef7711-f84c-4af2-a581-74142dc34458 admin admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 17:07:25.902277 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ef7711-f84c-4af2-a581-74142dc34458 admin admin] PATH_INFO: `/v3/projects/7687142082f24d4ab96ba1ff4391e2cf/users/9a2fa8a4a51a4b538e5241206d8cf452/roles/0999d3da63a545f18ff3252afdc90518` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 17:07:25.917931 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5ef7711-f84c-4af2-a581-74142dc34458 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7687142082f24d4ab96ba1ff4391e2cf, user_id=9a2fa8a4a51a4b538e5241206d8cf452, role_id=0999d3da63a545f18ff3252afdc90518)` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 17:07:25.920608 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5ef7711-f84c-4af2-a581-74142dc34458 admin admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 17:07:25.991049 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 1339/2672] 10.222.0.20 () {60 vars in 1483 bytes} [Mon Aug 1 17:07:25 2022] PUT /identity/v3/projects/7687142082f24d4ab96ba1ff4391e2cf/users/9a2fa8a4a51a4b538e5241206d8cf452/roles/0999d3da63a545f18ff3252afdc90518 => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 17:07:26.003277 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267ab69c-072f-42b6-a3f1-64ea12090e75 None None] REQUEST_METHOD: `POST` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 17:07:26.003567 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267ab69c-072f-42b6-a3f1-64ea12090e75 None None] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 17:07:26.003850 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267ab69c-072f-42b6-a3f1-64ea12090e75 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 17:07:26.031733 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.auth.core [None req-267ab69c-072f-42b6-a3f1-64ea12090e75 None None] MFA Rules not processed for user `9a2fa8a4a51a4b538e5241206d8cf452`. Rule list: `[]` (Enabled: `True`). {{(pid=87057) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 17:07:26.032951 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.fernet_utils [None req-267ab69c-072f-42b6-a3f1-64ea12090e75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87057) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 17:07:26.096429 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 1334/2673] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 1 17:07:26 2022] POST /identity/v3/auth/tokens => generated 3230 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 17:07:26.101086 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5ef7711-f84c-4af2-a581-74142dc34458 admin admin] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 17:07:26.162166 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5ef7711-f84c-4af2-a581-74142dc34458 admin admin] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 17:07:26.166980 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9917b75b-9fbc-4814-be12-0d4a01e59fd0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 17:07:26.167587 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9917b75b-9fbc-4814-be12-0d4a01e59fd0 admin admin] REQUEST_METHOD: `POST` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 17:07:26.167870 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9917b75b-9fbc-4814-be12-0d4a01e59fd0 admin admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 17:07:26.168146 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9917b75b-9fbc-4814-be12-0d4a01e59fd0 admin admin] PATH_INFO: `/v3/projects` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 17:07:26.168810 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9917b75b-9fbc-4814-be12-0d4a01e59fd0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 17:07:26.170740 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9917b75b-9fbc-4814-be12-0d4a01e59fd0 admin admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 17:07:26.243034 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG dbcounter [-] [87058] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=87058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 17:07:26.250060 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 1340/2674] 10.222.0.20 () {60 vars in 1153 bytes} [Mon Aug 1 17:07:26 2022] POST /identity/v3/projects => generated 365 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 17:07:26.254219 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-267ab69c-072f-42b6-a3f1-64ea12090e75 None None] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 17:07:26.265537 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-267ab69c-072f-42b6-a3f1-64ea12090e75 None None] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 17:07:26.269170 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG dbcounter [-] [87057] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87057) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 17:07:26.322346 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c2db80-f9b6-42a4-8e6a-84e8d069de63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 17:07:26.322964 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c2db80-f9b6-42a4-8e6a-84e8d069de63 admin admin] REQUEST_METHOD: `POST` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 17:07:26.323246 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c2db80-f9b6-42a4-8e6a-84e8d069de63 admin admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 17:07:26.323523 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c2db80-f9b6-42a4-8e6a-84e8d069de63 admin admin] PATH_INFO: `/v3/users` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 17:07:26.324201 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2c2db80-f9b6-42a4-8e6a-84e8d069de63 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 17:07:26.326166 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2c2db80-f9b6-42a4-8e6a-84e8d069de63 admin admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 17:07:26.378868 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 1335/2675] 10.222.0.20 () {60 vars in 1144 bytes} [Mon Aug 1 17:07:26 2022] POST /identity/v3/users => generated 289 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 17:07:26.383895 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9917b75b-9fbc-4814-be12-0d4a01e59fd0 admin admin] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 17:07:26.395184 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9917b75b-9fbc-4814-be12-0d4a01e59fd0 admin admin] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 17:07:26.399988 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dafaf69-f29e-488d-a17a-4f2277af89ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 17:07:26.400589 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dafaf69-f29e-488d-a17a-4f2277af89ee admin admin] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 17:07:26.401458 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dafaf69-f29e-488d-a17a-4f2277af89ee admin admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 17:07:26.401458 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dafaf69-f29e-488d-a17a-4f2277af89ee admin admin] PATH_INFO: `/v3/roles` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 17:07:26.401924 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dafaf69-f29e-488d-a17a-4f2277af89ee admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 17:07:26.403537 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dafaf69-f29e-488d-a17a-4f2277af89ee admin admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 17:07:26.415008 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 1341/2676] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Aug 1 17:07:26 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 17:07:26.419114 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c2db80-f9b6-42a4-8e6a-84e8d069de63 admin admin] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 17:07:26.429182 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c2db80-f9b6-42a4-8e6a-84e8d069de63 admin admin] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 17:07:26.434671 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f922182-9525-4dd7-b095-268911daea0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 17:07:26.435332 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f922182-9525-4dd7-b095-268911daea0f admin admin] REQUEST_METHOD: `PUT` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 17:07:26.435615 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f922182-9525-4dd7-b095-268911daea0f admin admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 17:07:26.435892 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f922182-9525-4dd7-b095-268911daea0f admin admin] PATH_INFO: `/v3/projects/76628644371a43668f32996f67dddfc0/users/bb2cf338c6bc4b1a83647d2326d57189/roles/440bb943dd5b46fea038b903b6eaaf81` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 17:07:26.450832 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f922182-9525-4dd7-b095-268911daea0f admin admin] RBAC: Authorizing `identity:create_grant(project_id=76628644371a43668f32996f67dddfc0, user_id=bb2cf338c6bc4b1a83647d2326d57189, role_id=440bb943dd5b46fea038b903b6eaaf81)` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 17:07:26.453576 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f922182-9525-4dd7-b095-268911daea0f admin admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 17:07:26.528614 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 1336/2677] 10.222.0.20 () {60 vars in 1483 bytes} [Mon Aug 1 17:07:26 2022] PUT /identity/v3/projects/76628644371a43668f32996f67dddfc0/users/bb2cf338c6bc4b1a83647d2326d57189/roles/440bb943dd5b46fea038b903b6eaaf81 => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 17:07:26.532561 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dafaf69-f29e-488d-a17a-4f2277af89ee admin admin] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 17:07:26.543883 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dafaf69-f29e-488d-a17a-4f2277af89ee admin admin] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 17:07:26.600428 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b60aa564-8603-4cc2-8dc8-adffe77a22c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 17:07:26.601039 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60aa564-8603-4cc2-8dc8-adffe77a22c9 admin admin] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 17:07:26.601915 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60aa564-8603-4cc2-8dc8-adffe77a22c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 17:07:26.602226 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60aa564-8603-4cc2-8dc8-adffe77a22c9 admin admin] PATH_INFO: `/v3/roles` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 17:07:26.602916 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b60aa564-8603-4cc2-8dc8-adffe77a22c9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 17:07:26.604507 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b60aa564-8603-4cc2-8dc8-adffe77a22c9 admin admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 17:07:26.614432 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 1342/2678] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Aug 1 17:07:26 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 17:07:26.619418 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f922182-9525-4dd7-b095-268911daea0f admin admin] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 17:07:26.631740 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f922182-9525-4dd7-b095-268911daea0f admin admin] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 17:07:26.635082 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d692ab-e8be-45a2-be79-48f0198682c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 17:07:26.635741 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d692ab-e8be-45a2-be79-48f0198682c5 admin admin] REQUEST_METHOD: `PUT` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 17:07:26.636023 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d692ab-e8be-45a2-be79-48f0198682c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 17:07:26.636295 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d692ab-e8be-45a2-be79-48f0198682c5 admin admin] PATH_INFO: `/v3/projects/76628644371a43668f32996f67dddfc0/users/bb2cf338c6bc4b1a83647d2326d57189/roles/440bb943dd5b46fea038b903b6eaaf81` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 17:07:26.638319 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06d692ab-e8be-45a2-be79-48f0198682c5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=76628644371a43668f32996f67dddfc0, user_id=bb2cf338c6bc4b1a83647d2326d57189, role_id=440bb943dd5b46fea038b903b6eaaf81)` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 17:07:26.641025 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06d692ab-e8be-45a2-be79-48f0198682c5 admin admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 17:07:26.651569 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 1337/2679] 10.222.0.20 () {60 vars in 1483 bytes} [Mon Aug 1 17:07:26 2022] PUT /identity/v3/projects/76628644371a43668f32996f67dddfc0/users/bb2cf338c6bc4b1a83647d2326d57189/roles/440bb943dd5b46fea038b903b6eaaf81 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 17:07:26.661752 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10738961-6193-4035-8aae-90ad63121970 None None] REQUEST_METHOD: `POST` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 17:07:26.662097 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10738961-6193-4035-8aae-90ad63121970 None None] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 17:07:26.662383 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10738961-6193-4035-8aae-90ad63121970 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 17:07:26.691068 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.auth.core [None req-10738961-6193-4035-8aae-90ad63121970 None None] MFA Rules not processed for user `bb2cf338c6bc4b1a83647d2326d57189`. Rule list: `[]` (Enabled: `True`). {{(pid=87058) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 17:07:26.692376 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.fernet_utils [None req-10738961-6193-4035-8aae-90ad63121970 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 17:07:26.762927 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 1343/2680] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 1 17:07:26 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 17:07:26.774728 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d692ab-e8be-45a2-be79-48f0198682c5 admin admin] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 17:07:26.776407 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.fernet_utils [None req-06d692ab-e8be-45a2-be79-48f0198682c5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87057) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 17:07:26.832284 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d692ab-e8be-45a2-be79-48f0198682c5 admin admin] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 17:07:26.836401 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed7d1e2f-c121-4946-a737-efd3afd3df03 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25970b52ca15448fa1abb1f974d64038', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b992d6a65df4bf9b27c13192d57c5f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 17:07:26.836999 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7d1e2f-c121-4946-a737-efd3afd3df03 service cinder] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 17:07:26.837274 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7d1e2f-c121-4946-a737-efd3afd3df03 service cinder] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 17:07:26.837562 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7d1e2f-c121-4946-a737-efd3afd3df03 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 17:07:26.846279 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed7d1e2f-c121-4946-a737-efd3afd3df03 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 17:07:26.847703 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed7d1e2f-c121-4946-a737-efd3afd3df03 service cinder] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 17:07:26.857529 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 1338/2681] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 1 17:07:26 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 17:07:27.062020 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10738961-6193-4035-8aae-90ad63121970 None None] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 17:07:27.073002 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10738961-6193-4035-8aae-90ad63121970 None None] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 17:07:27.076689 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b61d31-f020-4d02-b5e6-dddc85b04529 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25970b52ca15448fa1abb1f974d64038', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b992d6a65df4bf9b27c13192d57c5f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 17:07:27.077288 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b61d31-f020-4d02-b5e6-dddc85b04529 service cinder] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 17:07:27.077562 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b61d31-f020-4d02-b5e6-dddc85b04529 service cinder] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 17:07:27.077829 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b61d31-f020-4d02-b5e6-dddc85b04529 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 17:07:27.136403 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95b61d31-f020-4d02-b5e6-dddc85b04529 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 17:07:27.137917 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95b61d31-f020-4d02-b5e6-dddc85b04529 service cinder] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 17:07:27.147078 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 1344/2682] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 1 17:07:27 2022] GET /identity/v3/auth/tokens => generated 3230 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 17:07:30.451332 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed7d1e2f-c121-4946-a737-efd3afd3df03 service cinder] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 17:07:30.515294 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed7d1e2f-c121-4946-a737-efd3afd3df03 service cinder] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 17:07:30.519732 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b1d7102-4790-4362-afe3-f9b091ed1526 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 17:07:30.520364 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b1d7102-4790-4362-afe3-f9b091ed1526 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 17:07:30.520637 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b1d7102-4790-4362-afe3-f9b091ed1526 admin admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 17:07:30.520909 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b1d7102-4790-4362-afe3-f9b091ed1526 admin admin] PATH_INFO: `/v3/users/9a2fa8a4a51a4b538e5241206d8cf452` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 17:07:30.522071 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b1d7102-4790-4362-afe3-f9b091ed1526 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9a2fa8a4a51a4b538e5241206d8cf452)` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 17:07:30.524181 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b1d7102-4790-4362-afe3-f9b091ed1526 admin admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 17:07:30.526447 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG dbcounter [-] [87057] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=87057) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 17:07:30.673252 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.notifications [None req-4b1d7102-4790-4362-afe3-f9b091ed1526 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9a2fa8a4a51a4b538e5241206d8cf452'} {{(pid=87057) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 17:07:30.673904 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.notifications [None req-4b1d7102-4790-4362-afe3-f9b091ed1526 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9a2fa8a4a51a4b538e5241206d8cf452'} {{(pid=87057) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 17:07:30.744606 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.notifications [None req-4b1d7102-4790-4362-afe3-f9b091ed1526 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9a2fa8a4a51a4b538e5241206d8cf452'} {{(pid=87057) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 17:07:30.745144 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.notifications [None req-4b1d7102-4790-4362-afe3-f9b091ed1526 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9a2fa8a4a51a4b538e5241206d8cf452'} {{(pid=87057) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 17:07:30.753355 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.notifications [None req-4b1d7102-4790-4362-afe3-f9b091ed1526 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9a2fa8a4a51a4b538e5241206d8cf452'} {{(pid=87057) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 17:07:30.770829 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 1339/2683] 10.222.0.20 () {58 vars in 1224 bytes} [Mon Aug 1 17:07:30 2022] DELETE /identity/v3/users/9a2fa8a4a51a4b538e5241206d8cf452 => generated 0 bytes in 320 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 17:07:30.775986 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a545aa2-4804-456a-87c1-f6e0094a956b None None] REQUEST_METHOD: `POST` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 17:07:30.776453 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a545aa2-4804-456a-87c1-f6e0094a956b None None] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 17:07:30.776553 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a545aa2-4804-456a-87c1-f6e0094a956b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 17:07:30.790294 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG dbcounter [-] [87058] Writing DB stats keystone:SELECT=100 {{(pid=87058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 17:07:30.796495 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.auth.core [None req-6a545aa2-4804-456a-87c1-f6e0094a956b None None] MFA Rules not processed for user `1007f8e379b74e169b98fb7c3c0ba5b4`. Rule list: `[]` (Enabled: `True`). {{(pid=87058) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 17:07:30.807995 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.fernet_utils [None req-6a545aa2-4804-456a-87c1-f6e0094a956b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 17:07:30.856195 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 1345/2684] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 1 17:07:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 17:07:30.866249 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b1d7102-4790-4362-afe3-f9b091ed1526 admin admin] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 17:07:30.868336 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.fernet_utils [None req-4b1d7102-4790-4362-afe3-f9b091ed1526 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87057) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 17:07:30.926949 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b1d7102-4790-4362-afe3-f9b091ed1526 admin admin] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 17:07:30.931237 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-460697e1-389f-4115-99d5-c0f27a730d02 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d9bd8d649154c979be3744bfd016f95', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b992d6a65df4bf9b27c13192d57c5f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 17:07:30.931881 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460697e1-389f-4115-99d5-c0f27a730d02 service neutron] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 17:07:30.932167 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460697e1-389f-4115-99d5-c0f27a730d02 service neutron] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 17:07:30.932443 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460697e1-389f-4115-99d5-c0f27a730d02 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 17:07:30.941704 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-460697e1-389f-4115-99d5-c0f27a730d02 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 17:07:30.943208 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-460697e1-389f-4115-99d5-c0f27a730d02 service neutron] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 17:07:30.953504 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 1340/2685] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 1 17:07:30 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 17:07:31.415867 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a545aa2-4804-456a-87c1-f6e0094a956b None None] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 17:07:31.417458 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.fernet_utils [None req-6a545aa2-4804-456a-87c1-f6e0094a956b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 17:07:31.431116 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a545aa2-4804-456a-87c1-f6e0094a956b None None] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 17:07:31.436054 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de25f88-9e3d-418a-a60e-ac52aabbf914 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 17:07:31.436705 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de25f88-9e3d-418a-a60e-ac52aabbf914 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 17:07:31.443279 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de25f88-9e3d-418a-a60e-ac52aabbf914 admin admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 17:07:31.443279 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de25f88-9e3d-418a-a60e-ac52aabbf914 admin admin] PATH_INFO: `/v3/projects/7687142082f24d4ab96ba1ff4391e2cf` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 17:07:31.443279 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7de25f88-9e3d-418a-a60e-ac52aabbf914 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7687142082f24d4ab96ba1ff4391e2cf)` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 17:07:31.443279 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7de25f88-9e3d-418a-a60e-ac52aabbf914 admin admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 17:07:31.534336 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.notifications [None req-7de25f88-9e3d-418a-a60e-ac52aabbf914 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7687142082f24d4ab96ba1ff4391e2cf', 'request_id': 'req-7de25f88-9e3d-418a-a60e-ac52aabbf914'} {{(pid=87058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 17:07:31.541681 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.notifications [None req-7de25f88-9e3d-418a-a60e-ac52aabbf914 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7687142082f24d4ab96ba1ff4391e2cf', 'request_id': 'req-7de25f88-9e3d-418a-a60e-ac52aabbf914'} {{(pid=87058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 17:07:31.547302 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.notifications [None req-7de25f88-9e3d-418a-a60e-ac52aabbf914 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7687142082f24d4ab96ba1ff4391e2cf', 'request_id': 'req-7de25f88-9e3d-418a-a60e-ac52aabbf914'} {{(pid=87058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 17:07:31.612312 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.notifications [None req-7de25f88-9e3d-418a-a60e-ac52aabbf914 admin admin] The token cache is being invalidate because project 7687142082f24d4ab96ba1ff4391e2cf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87058) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 17:07:31.612588 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.notifications [None req-7de25f88-9e3d-418a-a60e-ac52aabbf914 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 17:07:31.613457 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.notifications [None req-7de25f88-9e3d-418a-a60e-ac52aabbf914 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 17:07:31.614985 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 1346/2686] 10.222.0.20 () {58 vars in 1233 bytes} [Mon Aug 1 17:07:31 2022] DELETE /identity/v3/projects/7687142082f24d4ab96ba1ff4391e2cf => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 17:07:31.620563 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-460697e1-389f-4115-99d5-c0f27a730d02 service neutron] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 17:07:31.629053 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-460697e1-389f-4115-99d5-c0f27a730d02 service neutron] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 17:07:31.685514 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27931d10-2425-47b7-ba01-43401fee1ec1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 17:07:31.686111 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27931d10-2425-47b7-ba01-43401fee1ec1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 17:07:31.686397 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27931d10-2425-47b7-ba01-43401fee1ec1 admin admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 17:07:31.686671 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27931d10-2425-47b7-ba01-43401fee1ec1 admin admin] PATH_INFO: `/v3/users/bb2cf338c6bc4b1a83647d2326d57189` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 17:07:31.687854 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27931d10-2425-47b7-ba01-43401fee1ec1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bb2cf338c6bc4b1a83647d2326d57189)` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 17:07:31.689784 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27931d10-2425-47b7-ba01-43401fee1ec1 admin admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 17:07:31.797394 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG dbcounter [-] [87057] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=87057) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 17:07:31.925313 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.notifications [None req-27931d10-2425-47b7-ba01-43401fee1ec1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bb2cf338c6bc4b1a83647d2326d57189'} {{(pid=87057) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 17:07:31.925975 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.notifications [None req-27931d10-2425-47b7-ba01-43401fee1ec1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bb2cf338c6bc4b1a83647d2326d57189'} {{(pid=87057) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 17:07:31.995844 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.notifications [None req-27931d10-2425-47b7-ba01-43401fee1ec1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bb2cf338c6bc4b1a83647d2326d57189'} {{(pid=87057) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 17:07:31.996459 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.notifications [None req-27931d10-2425-47b7-ba01-43401fee1ec1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bb2cf338c6bc4b1a83647d2326d57189'} {{(pid=87057) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 17:07:32.003854 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.notifications [None req-27931d10-2425-47b7-ba01-43401fee1ec1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bb2cf338c6bc4b1a83647d2326d57189'} {{(pid=87057) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 17:07:32.022334 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 1341/2687] 10.222.0.20 () {58 vars in 1224 bytes} [Mon Aug 1 17:07:31 2022] DELETE /identity/v3/users/bb2cf338c6bc4b1a83647d2326d57189 => generated 0 bytes in 404 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 17:07:32.403826 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de25f88-9e3d-418a-a60e-ac52aabbf914 admin admin] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 17:07:32.405311 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.fernet_utils [None req-7de25f88-9e3d-418a-a60e-ac52aabbf914 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 17:07:32.419015 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de25f88-9e3d-418a-a60e-ac52aabbf914 admin admin] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 17:07:32.478812 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b8e1a9-754d-447a-bd66-a80870b10aed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 17:07:32.479389 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b8e1a9-754d-447a-bd66-a80870b10aed admin admin] REQUEST_METHOD: `DELETE` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 17:07:32.479670 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b8e1a9-754d-447a-bd66-a80870b10aed admin admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 17:07:32.479981 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b8e1a9-754d-447a-bd66-a80870b10aed admin admin] PATH_INFO: `/v3/projects/76628644371a43668f32996f67dddfc0` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 17:07:32.481165 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23b8e1a9-754d-447a-bd66-a80870b10aed admin admin] RBAC: Authorizing `identity:delete_project(project_id=76628644371a43668f32996f67dddfc0)` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 17:07:32.483102 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23b8e1a9-754d-447a-bd66-a80870b10aed admin admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 17:07:32.567511 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG dbcounter [-] [87058] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=87058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 17:07:32.579165 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.notifications [None req-23b8e1a9-754d-447a-bd66-a80870b10aed admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '76628644371a43668f32996f67dddfc0', 'request_id': 'req-23b8e1a9-754d-447a-bd66-a80870b10aed'} {{(pid=87058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 17:07:32.585232 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.notifications [None req-23b8e1a9-754d-447a-bd66-a80870b10aed admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '76628644371a43668f32996f67dddfc0', 'request_id': 'req-23b8e1a9-754d-447a-bd66-a80870b10aed'} {{(pid=87058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 17:07:32.590874 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.notifications [None req-23b8e1a9-754d-447a-bd66-a80870b10aed admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '76628644371a43668f32996f67dddfc0', 'request_id': 'req-23b8e1a9-754d-447a-bd66-a80870b10aed'} {{(pid=87058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 17:07:32.664199 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.notifications [None req-23b8e1a9-754d-447a-bd66-a80870b10aed admin admin] The token cache is being invalidate because project 76628644371a43668f32996f67dddfc0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87058) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 17:07:32.664700 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.notifications [None req-23b8e1a9-754d-447a-bd66-a80870b10aed admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 17:07:32.665429 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.notifications [None req-23b8e1a9-754d-447a-bd66-a80870b10aed admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 17:07:32.666982 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 1347/2688] 10.222.0.20 () {58 vars in 1233 bytes} [Mon Aug 1 17:07:32 2022] DELETE /identity/v3/projects/76628644371a43668f32996f67dddfc0 => generated 0 bytes in 264 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 17:07:32.672027 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62e0dc9-7d3b-4bce-917f-3582f5a1c891 None None] REQUEST_METHOD: `POST` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 17:07:32.672510 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62e0dc9-7d3b-4bce-917f-3582f5a1c891 None None] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 17:07:32.672608 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62e0dc9-7d3b-4bce-917f-3582f5a1c891 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 17:07:32.692805 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.auth.core [None req-a62e0dc9-7d3b-4bce-917f-3582f5a1c891 None None] MFA Rules not processed for user `1007f8e379b74e169b98fb7c3c0ba5b4`. Rule list: `[]` (Enabled: `True`). {{(pid=87057) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 17:07:32.694103 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.fernet_utils [None req-a62e0dc9-7d3b-4bce-917f-3582f5a1c891 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87057) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 17:07:32.754237 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 1342/2689] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 1 17:07:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 17:07:32.763555 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2bda1a-cbf5-430e-aba1-52e2dfa754be None None] REQUEST_METHOD: `POST` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 17:07:32.763846 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2bda1a-cbf5-430e-aba1-52e2dfa754be None None] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 17:07:32.764127 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2bda1a-cbf5-430e-aba1-52e2dfa754be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 17:07:32.784544 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.auth.core [None req-ce2bda1a-cbf5-430e-aba1-52e2dfa754be None None] MFA Rules not processed for user `1007f8e379b74e169b98fb7c3c0ba5b4`. Rule list: `[]` (Enabled: `True`). {{(pid=87058) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 17:07:32.785767 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.fernet_utils [None req-ce2bda1a-cbf5-430e-aba1-52e2dfa754be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 17:07:32.794394 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 1348/2690] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 1 17:07:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 17:07:32.798489 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a62e0dc9-7d3b-4bce-917f-3582f5a1c891 None None] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 17:07:32.808048 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a62e0dc9-7d3b-4bce-917f-3582f5a1c891 None None] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 17:07:32.812553 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a1523f-05eb-4b8b-ad6a-1cf041dbd1ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 17:07:32.813157 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a1523f-05eb-4b8b-ad6a-1cf041dbd1ba admin admin] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 17:07:32.813434 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a1523f-05eb-4b8b-ad6a-1cf041dbd1ba admin admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 17:07:32.813703 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a1523f-05eb-4b8b-ad6a-1cf041dbd1ba admin admin] PATH_INFO: `/v3/domains` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 17:07:32.814424 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13a1523f-05eb-4b8b-ad6a-1cf041dbd1ba admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87057) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 17:07:32.814724 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13a1523f-05eb-4b8b-ad6a-1cf041dbd1ba admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 17:07:32.816316 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13a1523f-05eb-4b8b-ad6a-1cf041dbd1ba admin admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 17:07:32.827647 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 1343/2691] 10.222.0.20 () {58 vars in 1153 bytes} [Mon Aug 1 17:07:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 17:07:32.831590 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce2bda1a-cbf5-430e-aba1-52e2dfa754be None None] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 17:07:32.841482 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce2bda1a-cbf5-430e-aba1-52e2dfa754be None None] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 17:07:32.846007 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf49733-2d5a-4ef3-b17e-37fc605fd3a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 17:07:32.846623 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf49733-2d5a-4ef3-b17e-37fc605fd3a0 admin admin] REQUEST_METHOD: `POST` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 17:07:32.846895 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf49733-2d5a-4ef3-b17e-37fc605fd3a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 17:07:32.847164 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf49733-2d5a-4ef3-b17e-37fc605fd3a0 admin admin] PATH_INFO: `/v3/projects` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 17:07:32.847828 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caf49733-2d5a-4ef3-b17e-37fc605fd3a0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 17:07:32.849699 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caf49733-2d5a-4ef3-b17e-37fc605fd3a0 admin admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 17:07:32.929002 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 1349/2692] 10.222.0.20 () {60 vars in 1153 bytes} [Mon Aug 1 17:07:32 2022] POST /identity/v3/projects => generated 377 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 17:07:32.934132 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a1523f-05eb-4b8b-ad6a-1cf041dbd1ba admin admin] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 17:07:32.944697 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a1523f-05eb-4b8b-ad6a-1cf041dbd1ba admin admin] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 17:07:33.001553 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43021fb5-6add-457c-8c47-6a7ced22546a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 17:07:33.002203 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43021fb5-6add-457c-8c47-6a7ced22546a admin admin] REQUEST_METHOD: `POST` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 17:07:33.002491 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43021fb5-6add-457c-8c47-6a7ced22546a admin admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 17:07:33.002771 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43021fb5-6add-457c-8c47-6a7ced22546a admin admin] PATH_INFO: `/v3/users` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 17:07:33.003432 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43021fb5-6add-457c-8c47-6a7ced22546a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 17:07:33.005471 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43021fb5-6add-457c-8c47-6a7ced22546a admin admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 17:07:33.076570 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 1344/2693] 10.222.0.20 () {60 vars in 1144 bytes} [Mon Aug 1 17:07:32 2022] POST /identity/v3/users => generated 289 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 17:07:33.080872 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf49733-2d5a-4ef3-b17e-37fc605fd3a0 admin admin] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 17:07:33.091501 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf49733-2d5a-4ef3-b17e-37fc605fd3a0 admin admin] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 17:07:33.096197 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1e224dc-4d77-49cf-8ae4-50fd9e9f2daa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 17:07:33.096794 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e224dc-4d77-49cf-8ae4-50fd9e9f2daa admin admin] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 17:07:33.097080 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e224dc-4d77-49cf-8ae4-50fd9e9f2daa admin admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 17:07:33.097354 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e224dc-4d77-49cf-8ae4-50fd9e9f2daa admin admin] PATH_INFO: `/v3/roles` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 17:07:33.098060 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1e224dc-4d77-49cf-8ae4-50fd9e9f2daa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 17:07:33.099670 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1e224dc-4d77-49cf-8ae4-50fd9e9f2daa admin admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 17:07:33.109737 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 1350/2694] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Aug 1 17:07:33 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 17:07:33.114200 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43021fb5-6add-457c-8c47-6a7ced22546a admin admin] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 17:07:33.124825 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43021fb5-6add-457c-8c47-6a7ced22546a admin admin] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 17:07:33.129682 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d004b9-c2df-4d7f-b670-680fb895ddb1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 17:07:33.130798 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d004b9-c2df-4d7f-b670-680fb895ddb1 admin admin] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 17:07:33.130798 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d004b9-c2df-4d7f-b670-680fb895ddb1 admin admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 17:07:33.131044 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d004b9-c2df-4d7f-b670-680fb895ddb1 admin admin] PATH_INFO: `/v3/roles` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 17:07:33.131652 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10d004b9-c2df-4d7f-b670-680fb895ddb1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 17:07:33.134132 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10d004b9-c2df-4d7f-b670-680fb895ddb1 admin admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 17:07:33.145076 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 1345/2695] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Aug 1 17:07:33 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 01 17:07:33.149513 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1e224dc-4d77-49cf-8ae4-50fd9e9f2daa admin admin] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 17:07:33.159570 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1e224dc-4d77-49cf-8ae4-50fd9e9f2daa admin admin] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 17:07:33.164933 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95bb96d1-ea9f-4e59-b42b-6848a41da70f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 17:07:33.165622 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95bb96d1-ea9f-4e59-b42b-6848a41da70f admin admin] REQUEST_METHOD: `PUT` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 17:07:33.165907 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95bb96d1-ea9f-4e59-b42b-6848a41da70f admin admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 17:07:33.166223 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95bb96d1-ea9f-4e59-b42b-6848a41da70f admin admin] PATH_INFO: `/v3/projects/cdaf92328c904668b6844cf3e8536a87/users/c87d99b35866483b97ce11c0d818aa79/roles/0999d3da63a545f18ff3252afdc90518` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 17:07:33.181421 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95bb96d1-ea9f-4e59-b42b-6848a41da70f admin admin] RBAC: Authorizing `identity:create_grant(project_id=cdaf92328c904668b6844cf3e8536a87, user_id=c87d99b35866483b97ce11c0d818aa79, role_id=0999d3da63a545f18ff3252afdc90518)` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 17:07:33.184119 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95bb96d1-ea9f-4e59-b42b-6848a41da70f admin admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 17:07:33.230617 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 1351/2696] 10.222.0.20 () {60 vars in 1483 bytes} [Mon Aug 1 17:07:33 2022] PUT /identity/v3/projects/cdaf92328c904668b6844cf3e8536a87/users/c87d99b35866483b97ce11c0d818aa79/roles/0999d3da63a545f18ff3252afdc90518 => generated 0 bytes in 82 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 17:07:33.241388 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025285c1-789b-4768-bf65-ab18f83e724f None None] REQUEST_METHOD: `POST` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 17:07:33.241784 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025285c1-789b-4768-bf65-ab18f83e724f None None] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 17:07:33.242092 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025285c1-789b-4768-bf65-ab18f83e724f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 17:07:33.262234 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG dbcounter [-] [87057] Writing DB stats keystone:DELETE=6,keystone:SELECT=90,keystone:INSERT=4 {{(pid=87057) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 17:07:33.270671 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.auth.core [None req-025285c1-789b-4768-bf65-ab18f83e724f None None] MFA Rules not processed for user `c87d99b35866483b97ce11c0d818aa79`. Rule list: `[]` (Enabled: `True`). {{(pid=87057) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 17:07:33.271879 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.fernet_utils [None req-025285c1-789b-4768-bf65-ab18f83e724f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87057) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 17:07:33.337474 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 1346/2697] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 1 17:07:33 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 17:07:33.348724 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95bb96d1-ea9f-4e59-b42b-6848a41da70f admin admin] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 17:07:33.350439 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.fernet_utils [None req-95bb96d1-ea9f-4e59-b42b-6848a41da70f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 17:07:33.428400 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95bb96d1-ea9f-4e59-b42b-6848a41da70f admin admin] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 17:07:33.432332 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b147662b-5752-4ece-b78f-1a203fda5f1b service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd80694e439e2401d9491712666409016', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b992d6a65df4bf9b27c13192d57c5f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 17:07:33.432950 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b147662b-5752-4ece-b78f-1a203fda5f1b service glance] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 17:07:33.433231 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b147662b-5752-4ece-b78f-1a203fda5f1b service glance] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 17:07:33.433504 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b147662b-5752-4ece-b78f-1a203fda5f1b service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 17:07:33.441734 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b147662b-5752-4ece-b78f-1a203fda5f1b service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 17:07:33.443184 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b147662b-5752-4ece-b78f-1a203fda5f1b service glance] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 17:07:33.452645 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 1352/2698] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 1 17:07:33 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 17:07:43.339079 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG dbcounter [-] [87057] Writing DB stats keystone:SELECT=28 {{(pid=87057) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 17:07:43.448639 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG dbcounter [-] [87058] Writing DB stats keystone:SELECT=74,keystone:DELETE=5,keystone:INSERT=3 {{(pid=87058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 17:07:59.009681 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-025285c1-789b-4768-bf65-ab18f83e724f None None] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 17:07:59.070568 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-025285c1-789b-4768-bf65-ab18f83e724f None None] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 17:07:59.080996 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f5c5fd7-b899-4d47-9a84-cce589e2b2fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 17:07:59.080996 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5c5fd7-b899-4d47-9a84-cce589e2b2fe admin admin] REQUEST_METHOD: `DELETE` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 17:07:59.080996 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5c5fd7-b899-4d47-9a84-cce589e2b2fe admin admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 17:07:59.080996 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5c5fd7-b899-4d47-9a84-cce589e2b2fe admin admin] PATH_INFO: `/v3/users/c87d99b35866483b97ce11c0d818aa79` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 17:07:59.081419 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f5c5fd7-b899-4d47-9a84-cce589e2b2fe admin admin] RBAC: Authorizing `identity:delete_user(user_id=c87d99b35866483b97ce11c0d818aa79)` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 17:07:59.081419 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f5c5fd7-b899-4d47-9a84-cce589e2b2fe admin admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 17:07:59.203123 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.notifications [None req-3f5c5fd7-b899-4d47-9a84-cce589e2b2fe admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c87d99b35866483b97ce11c0d818aa79'} {{(pid=87057) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 17:07:59.204287 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.notifications [None req-3f5c5fd7-b899-4d47-9a84-cce589e2b2fe admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c87d99b35866483b97ce11c0d818aa79'} {{(pid=87057) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 17:07:59.258442 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.notifications [None req-3f5c5fd7-b899-4d47-9a84-cce589e2b2fe admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c87d99b35866483b97ce11c0d818aa79'} {{(pid=87057) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 17:07:59.259060 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.notifications [None req-3f5c5fd7-b899-4d47-9a84-cce589e2b2fe admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c87d99b35866483b97ce11c0d818aa79'} {{(pid=87057) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 17:07:59.266661 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.notifications [None req-3f5c5fd7-b899-4d47-9a84-cce589e2b2fe admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c87d99b35866483b97ce11c0d818aa79'} {{(pid=87057) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 17:07:59.284235 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 1347/2699] 10.222.0.20 () {58 vars in 1224 bytes} [Mon Aug 1 17:07:59 2022] DELETE /identity/v3/users/c87d99b35866483b97ce11c0d818aa79 => generated 0 bytes in 275 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 17:07:59.288732 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc4d05e-1b56-4375-878a-7c255197a37c None None] REQUEST_METHOD: `POST` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 17:07:59.289251 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc4d05e-1b56-4375-878a-7c255197a37c None None] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 17:07:59.289379 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc4d05e-1b56-4375-878a-7c255197a37c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 17:07:59.308242 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.auth.core [None req-0dc4d05e-1b56-4375-878a-7c255197a37c None None] MFA Rules not processed for user `1007f8e379b74e169b98fb7c3c0ba5b4`. Rule list: `[]` (Enabled: `True`). {{(pid=87058) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 17:07:59.309452 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.fernet_utils [None req-0dc4d05e-1b56-4375-878a-7c255197a37c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 17:07:59.368181 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 1353/2700] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 1 17:07:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 17:07:59.377872 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f5c5fd7-b899-4d47-9a84-cce589e2b2fe admin admin] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 17:07:59.379338 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.fernet_utils [None req-3f5c5fd7-b899-4d47-9a84-cce589e2b2fe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87057) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 17:07:59.435548 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f5c5fd7-b899-4d47-9a84-cce589e2b2fe admin admin] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 17:07:59.439198 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b3041b4-64a7-49d4-ad3a-a75e9e1e1402 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d9bd8d649154c979be3744bfd016f95', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b992d6a65df4bf9b27c13192d57c5f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 17:07:59.439812 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3041b4-64a7-49d4-ad3a-a75e9e1e1402 service neutron] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 17:07:59.440092 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3041b4-64a7-49d4-ad3a-a75e9e1e1402 service neutron] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 17:07:59.440365 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3041b4-64a7-49d4-ad3a-a75e9e1e1402 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 17:07:59.449169 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b3041b4-64a7-49d4-ad3a-a75e9e1e1402 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 17:07:59.450629 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b3041b4-64a7-49d4-ad3a-a75e9e1e1402 service neutron] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 17:07:59.460802 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 1348/2701] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 1 17:07:59 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 17:07:59.837853 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dc4d05e-1b56-4375-878a-7c255197a37c None None] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 17:07:59.839318 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.fernet_utils [None req-0dc4d05e-1b56-4375-878a-7c255197a37c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 17:07:59.851433 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dc4d05e-1b56-4375-878a-7c255197a37c None None] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 17:07:59.858238 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a15fe6a-c460-409b-8518-09bd7ef0357b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 17:07:59.858238 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a15fe6a-c460-409b-8518-09bd7ef0357b admin admin] REQUEST_METHOD: `DELETE` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 17:07:59.858238 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a15fe6a-c460-409b-8518-09bd7ef0357b admin admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 17:07:59.858238 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a15fe6a-c460-409b-8518-09bd7ef0357b admin admin] PATH_INFO: `/v3/projects/cdaf92328c904668b6844cf3e8536a87` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 17:07:59.859231 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a15fe6a-c460-409b-8518-09bd7ef0357b admin admin] RBAC: Authorizing `identity:delete_project(project_id=cdaf92328c904668b6844cf3e8536a87)` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 17:07:59.861151 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a15fe6a-c460-409b-8518-09bd7ef0357b admin admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 17:07:59.985192 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.notifications [None req-6a15fe6a-c460-409b-8518-09bd7ef0357b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cdaf92328c904668b6844cf3e8536a87', 'request_id': 'req-6a15fe6a-c460-409b-8518-09bd7ef0357b'} {{(pid=87058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 17:07:59.991811 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.notifications [None req-6a15fe6a-c460-409b-8518-09bd7ef0357b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cdaf92328c904668b6844cf3e8536a87', 'request_id': 'req-6a15fe6a-c460-409b-8518-09bd7ef0357b'} {{(pid=87058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 17:07:59.997269 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.notifications [None req-6a15fe6a-c460-409b-8518-09bd7ef0357b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cdaf92328c904668b6844cf3e8536a87', 'request_id': 'req-6a15fe6a-c460-409b-8518-09bd7ef0357b'} {{(pid=87058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 17:08:00.063984 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.notifications [None req-6a15fe6a-c460-409b-8518-09bd7ef0357b admin admin] The token cache is being invalidate because project cdaf92328c904668b6844cf3e8536a87 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87058) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 17:08:00.064274 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.notifications [None req-6a15fe6a-c460-409b-8518-09bd7ef0357b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 17:08:00.065566 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.notifications [None req-6a15fe6a-c460-409b-8518-09bd7ef0357b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 17:08:00.067119 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 1354/2702] 10.222.0.20 () {58 vars in 1233 bytes} [Mon Aug 1 17:07:59 2022] DELETE /identity/v3/projects/cdaf92328c904668b6844cf3e8536a87 => generated 0 bytes in 230 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 17:08:00.072148 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05ac5e8-27d6-4ff3-821e-e1f22f806574 None None] REQUEST_METHOD: `POST` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 17:08:00.072662 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05ac5e8-27d6-4ff3-821e-e1f22f806574 None None] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 17:08:00.072770 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05ac5e8-27d6-4ff3-821e-e1f22f806574 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 17:08:00.093185 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.auth.core [None req-b05ac5e8-27d6-4ff3-821e-e1f22f806574 None None] MFA Rules not processed for user `1007f8e379b74e169b98fb7c3c0ba5b4`. Rule list: `[]` (Enabled: `True`). {{(pid=87057) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 17:08:00.093705 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.fernet_utils [None req-b05ac5e8-27d6-4ff3-821e-e1f22f806574 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87057) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 17:08:00.111175 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG dbcounter [-] [87057] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87057) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 17:08:00.153436 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 1349/2703] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 1 17:08:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 17:08:00.162335 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4643c7-7ef8-49b0-8432-4995a862d8ee None None] REQUEST_METHOD: `POST` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 17:08:00.162623 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4643c7-7ef8-49b0-8432-4995a862d8ee None None] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 17:08:00.162898 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4643c7-7ef8-49b0-8432-4995a862d8ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 17:08:00.183654 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.auth.core [None req-eb4643c7-7ef8-49b0-8432-4995a862d8ee None None] MFA Rules not processed for user `1007f8e379b74e169b98fb7c3c0ba5b4`. Rule list: `[]` (Enabled: `True`). {{(pid=87058) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 17:08:00.186206 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.fernet_utils [None req-eb4643c7-7ef8-49b0-8432-4995a862d8ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 17:08:00.193551 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 1355/2704] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 1 17:08:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 17:08:00.197739 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b05ac5e8-27d6-4ff3-821e-e1f22f806574 None None] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 17:08:00.207417 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b05ac5e8-27d6-4ff3-821e-e1f22f806574 None None] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 17:08:00.212542 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f4d3474-e165-46e4-881b-157de31e0a27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 17:08:00.213156 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4d3474-e165-46e4-881b-157de31e0a27 admin admin] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 17:08:00.213440 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4d3474-e165-46e4-881b-157de31e0a27 admin admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 17:08:00.213715 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4d3474-e165-46e4-881b-157de31e0a27 admin admin] PATH_INFO: `/v3/domains` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 17:08:00.214450 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f4d3474-e165-46e4-881b-157de31e0a27 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87057) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 17:08:00.214756 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f4d3474-e165-46e4-881b-157de31e0a27 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 17:08:00.216353 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f4d3474-e165-46e4-881b-157de31e0a27 admin admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 17:08:00.228595 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 1350/2705] 10.222.0.20 () {58 vars in 1153 bytes} [Mon Aug 1 17:08:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 17:08:00.290921 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb4643c7-7ef8-49b0-8432-4995a862d8ee None None] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 17:08:00.300452 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb4643c7-7ef8-49b0-8432-4995a862d8ee None None] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 17:08:00.305103 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4bc11a4-bedc-47fd-9689-05b91007078d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 17:08:00.305711 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4bc11a4-bedc-47fd-9689-05b91007078d admin admin] REQUEST_METHOD: `POST` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 17:08:00.305988 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4bc11a4-bedc-47fd-9689-05b91007078d admin admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 17:08:00.306292 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4bc11a4-bedc-47fd-9689-05b91007078d admin admin] PATH_INFO: `/v3/projects` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 17:08:00.306956 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4bc11a4-bedc-47fd-9689-05b91007078d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 17:08:00.308848 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4bc11a4-bedc-47fd-9689-05b91007078d admin admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 17:08:00.392221 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 1356/2706] 10.222.0.20 () {60 vars in 1153 bytes} [Mon Aug 1 17:08:00 2022] POST /identity/v3/projects => generated 377 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 17:08:00.396376 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f4d3474-e165-46e4-881b-157de31e0a27 admin admin] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 17:08:00.406368 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f4d3474-e165-46e4-881b-157de31e0a27 admin admin] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 17:08:00.461862 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-238ab061-d7a4-47ad-89d0-0e324420bee1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 17:08:00.462506 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238ab061-d7a4-47ad-89d0-0e324420bee1 admin admin] REQUEST_METHOD: `POST` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 17:08:00.462788 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238ab061-d7a4-47ad-89d0-0e324420bee1 admin admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 17:08:00.463063 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238ab061-d7a4-47ad-89d0-0e324420bee1 admin admin] PATH_INFO: `/v3/users` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 17:08:00.463722 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-238ab061-d7a4-47ad-89d0-0e324420bee1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 17:08:00.465648 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-238ab061-d7a4-47ad-89d0-0e324420bee1 admin admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 17:08:00.520966 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 1351/2707] 10.222.0.20 () {60 vars in 1144 bytes} [Mon Aug 1 17:08:00 2022] POST /identity/v3/users => generated 289 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 17:08:00.525204 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4bc11a4-bedc-47fd-9689-05b91007078d admin admin] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 17:08:00.536558 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4bc11a4-bedc-47fd-9689-05b91007078d admin admin] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 17:08:00.541250 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f8d9734-6d1b-4a91-b3ff-2de163a0e544 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 17:08:00.541855 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8d9734-6d1b-4a91-b3ff-2de163a0e544 admin admin] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 17:08:00.542174 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8d9734-6d1b-4a91-b3ff-2de163a0e544 admin admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 17:08:00.542447 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8d9734-6d1b-4a91-b3ff-2de163a0e544 admin admin] PATH_INFO: `/v3/roles` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 17:08:00.543163 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f8d9734-6d1b-4a91-b3ff-2de163a0e544 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 17:08:00.544791 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f8d9734-6d1b-4a91-b3ff-2de163a0e544 admin admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 17:08:00.555169 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 1357/2708] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Aug 1 17:08:00 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 17:08:00.559029 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-238ab061-d7a4-47ad-89d0-0e324420bee1 admin admin] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 17:08:00.569487 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-238ab061-d7a4-47ad-89d0-0e324420bee1 admin admin] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 17:08:00.574188 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2fdca77-e3d0-4b59-b745-65709371dc0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 17:08:00.574787 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2fdca77-e3d0-4b59-b745-65709371dc0e admin admin] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 17:08:00.575068 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2fdca77-e3d0-4b59-b745-65709371dc0e admin admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 17:08:00.575343 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2fdca77-e3d0-4b59-b745-65709371dc0e admin admin] PATH_INFO: `/v3/roles` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 17:08:00.576030 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2fdca77-e3d0-4b59-b745-65709371dc0e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 17:08:00.577626 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2fdca77-e3d0-4b59-b745-65709371dc0e admin admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 17:08:00.589056 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 1352/2709] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Aug 1 17:08:00 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 17:08:00.593061 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f8d9734-6d1b-4a91-b3ff-2de163a0e544 admin admin] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 17:08:00.603863 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f8d9734-6d1b-4a91-b3ff-2de163a0e544 admin admin] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 17:08:00.608620 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4bacb2f-947d-48a2-b0c7-eda20173997d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 17:08:00.609264 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4bacb2f-947d-48a2-b0c7-eda20173997d admin admin] REQUEST_METHOD: `PUT` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 17:08:00.610171 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4bacb2f-947d-48a2-b0c7-eda20173997d admin admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 17:08:00.610451 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4bacb2f-947d-48a2-b0c7-eda20173997d admin admin] PATH_INFO: `/v3/projects/fd7f6a84b7df45ef8dafe2a75f4c5f25/users/f45dfa3f4eeb48e58b0a0c5ec37599d9/roles/0999d3da63a545f18ff3252afdc90518` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 17:08:00.626023 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4bacb2f-947d-48a2-b0c7-eda20173997d admin admin] RBAC: Authorizing `identity:create_grant(project_id=fd7f6a84b7df45ef8dafe2a75f4c5f25, user_id=f45dfa3f4eeb48e58b0a0c5ec37599d9, role_id=0999d3da63a545f18ff3252afdc90518)` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 17:08:00.628721 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4bacb2f-947d-48a2-b0c7-eda20173997d admin admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 17:08:00.681520 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 1358/2710] 10.222.0.20 () {60 vars in 1483 bytes} [Mon Aug 1 17:08:00 2022] PUT /identity/v3/projects/fd7f6a84b7df45ef8dafe2a75f4c5f25/users/f45dfa3f4eeb48e58b0a0c5ec37599d9/roles/0999d3da63a545f18ff3252afdc90518 => generated 0 bytes in 89 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 17:08:00.691797 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442d226d-7f01-4ec6-9386-a5a3d1347975 None None] REQUEST_METHOD: `POST` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 17:08:00.692092 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442d226d-7f01-4ec6-9386-a5a3d1347975 None None] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 17:08:00.692374 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442d226d-7f01-4ec6-9386-a5a3d1347975 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 17:08:00.720585 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.auth.core [None req-442d226d-7f01-4ec6-9386-a5a3d1347975 None None] MFA Rules not processed for user `f45dfa3f4eeb48e58b0a0c5ec37599d9`. Rule list: `[]` (Enabled: `True`). {{(pid=87057) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 17:08:00.721813 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.fernet_utils [None req-442d226d-7f01-4ec6-9386-a5a3d1347975 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87057) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 17:08:00.785643 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 1353/2711] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 1 17:08:00 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 17:08:00.796826 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4bacb2f-947d-48a2-b0c7-eda20173997d admin admin] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 17:08:00.798680 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.fernet_utils [None req-b4bacb2f-947d-48a2-b0c7-eda20173997d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 17:08:00.820397 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG dbcounter [-] [87058] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=87058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 17:08:00.855836 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4bacb2f-947d-48a2-b0c7-eda20173997d admin admin] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 17:08:00.859458 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-678c26f3-2c51-414e-8c2c-57b46c0426b9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25970b52ca15448fa1abb1f974d64038', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b992d6a65df4bf9b27c13192d57c5f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 17:08:00.860060 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678c26f3-2c51-414e-8c2c-57b46c0426b9 service cinder] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 17:08:00.860338 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678c26f3-2c51-414e-8c2c-57b46c0426b9 service cinder] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 17:08:00.860600 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678c26f3-2c51-414e-8c2c-57b46c0426b9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 17:08:00.869870 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-678c26f3-2c51-414e-8c2c-57b46c0426b9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 17:08:00.871319 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-678c26f3-2c51-414e-8c2c-57b46c0426b9 service cinder] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 17:08:00.881960 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 1359/2712] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 1 17:08:00 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 17:08:10.782737 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG dbcounter [-] [87057] Writing DB stats keystone:SELECT=89,keystone:INSERT=3 {{(pid=87057) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 17:08:10.898652 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG dbcounter [-] [87058] Writing DB stats keystone:SELECT=17 {{(pid=87058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 17:08:13.142373 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-442d226d-7f01-4ec6-9386-a5a3d1347975 None None] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 17:08:13.143764 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.fernet_utils [None req-442d226d-7f01-4ec6-9386-a5a3d1347975 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87057) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 17:08:13.213020 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-442d226d-7f01-4ec6-9386-a5a3d1347975 None None] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 17:08:13.217920 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb695cd-ba93-4288-88a7-10c7f4b86909 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8362293261614623bcdabc4120a97a9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b992d6a65df4bf9b27c13192d57c5f0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 17:08:13.219112 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb695cd-ba93-4288-88a7-10c7f4b86909 service placement] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 17:08:13.219384 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb695cd-ba93-4288-88a7-10c7f4b86909 service placement] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 17:08:13.219795 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb695cd-ba93-4288-88a7-10c7f4b86909 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 17:08:13.221552 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.fernet_utils [None req-1bb695cd-ba93-4288-88a7-10c7f4b86909 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87057) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 17:08:13.233948 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bb695cd-ba93-4288-88a7-10c7f4b86909 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 17:08:13.236333 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bb695cd-ba93-4288-88a7-10c7f4b86909 service placement] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 17:08:13.246975 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 1354/2713] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 1 17:08:13 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 17:08:23.242541 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG dbcounter [-] [87057] Writing DB stats keystone:SELECT=32 {{(pid=87057) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 17:08:27.701975 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-678c26f3-2c51-414e-8c2c-57b46c0426b9 service cinder] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 17:08:27.767141 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-678c26f3-2c51-414e-8c2c-57b46c0426b9 service cinder] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 17:08:27.771767 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab14af5-2b2e-47ad-8fb0-785c7d96e27e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 17:08:27.772410 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab14af5-2b2e-47ad-8fb0-785c7d96e27e admin admin] REQUEST_METHOD: `DELETE` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 17:08:27.772721 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab14af5-2b2e-47ad-8fb0-785c7d96e27e admin admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 17:08:27.772996 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab14af5-2b2e-47ad-8fb0-785c7d96e27e admin admin] PATH_INFO: `/v3/users/f45dfa3f4eeb48e58b0a0c5ec37599d9` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 17:08:27.774279 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dab14af5-2b2e-47ad-8fb0-785c7d96e27e admin admin] RBAC: Authorizing `identity:delete_user(user_id=f45dfa3f4eeb48e58b0a0c5ec37599d9)` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 17:08:27.776336 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dab14af5-2b2e-47ad-8fb0-785c7d96e27e admin admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 17:08:27.935338 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.notifications [None req-dab14af5-2b2e-47ad-8fb0-785c7d96e27e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f45dfa3f4eeb48e58b0a0c5ec37599d9'} {{(pid=87058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 17:08:27.935997 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.notifications [None req-dab14af5-2b2e-47ad-8fb0-785c7d96e27e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f45dfa3f4eeb48e58b0a0c5ec37599d9'} {{(pid=87058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 17:08:27.936757 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.notifications [None req-dab14af5-2b2e-47ad-8fb0-785c7d96e27e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f45dfa3f4eeb48e58b0a0c5ec37599d9'} {{(pid=87058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 17:08:27.950132 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.notifications [None req-dab14af5-2b2e-47ad-8fb0-785c7d96e27e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f45dfa3f4eeb48e58b0a0c5ec37599d9'} {{(pid=87058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 17:08:27.957688 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.notifications [None req-dab14af5-2b2e-47ad-8fb0-785c7d96e27e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f45dfa3f4eeb48e58b0a0c5ec37599d9'} {{(pid=87058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 17:08:28.029524 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 1360/2714] 10.222.0.20 () {58 vars in 1224 bytes} [Mon Aug 1 17:08:27 2022] DELETE /identity/v3/users/f45dfa3f4eeb48e58b0a0c5ec37599d9 => generated 0 bytes in 328 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 17:08:28.034808 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85eb228-e139-459c-9fd7-1e473621b4ec None None] REQUEST_METHOD: `POST` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 17:08:28.035315 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85eb228-e139-459c-9fd7-1e473621b4ec None None] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 17:08:28.035452 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85eb228-e139-459c-9fd7-1e473621b4ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 17:08:28.055193 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.auth.core [None req-e85eb228-e139-459c-9fd7-1e473621b4ec None None] MFA Rules not processed for user `1007f8e379b74e169b98fb7c3c0ba5b4`. Rule list: `[]` (Enabled: `True`). {{(pid=87057) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 17:08:28.056512 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.fernet_utils [None req-e85eb228-e139-459c-9fd7-1e473621b4ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87057) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 17:08:28.119997 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 1355/2715] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 1 17:08:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 17:08:28.129813 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab14af5-2b2e-47ad-8fb0-785c7d96e27e admin admin] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 17:08:28.131631 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.fernet_utils [None req-dab14af5-2b2e-47ad-8fb0-785c7d96e27e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 17:08:28.190148 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab14af5-2b2e-47ad-8fb0-785c7d96e27e admin admin] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 17:08:28.193973 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eaf8d8b-56f5-4f84-b903-f35ae67996c2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d9bd8d649154c979be3744bfd016f95', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b992d6a65df4bf9b27c13192d57c5f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 17:08:28.194621 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eaf8d8b-56f5-4f84-b903-f35ae67996c2 service neutron] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 17:08:28.194902 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eaf8d8b-56f5-4f84-b903-f35ae67996c2 service neutron] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 17:08:28.195176 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eaf8d8b-56f5-4f84-b903-f35ae67996c2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 17:08:28.204596 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eaf8d8b-56f5-4f84-b903-f35ae67996c2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 17:08:28.211960 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eaf8d8b-56f5-4f84-b903-f35ae67996c2 service neutron] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 17:08:28.216366 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 1361/2716] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 1 17:08:28 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 17:08:28.765972 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e85eb228-e139-459c-9fd7-1e473621b4ec None None] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 17:08:28.768231 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.fernet_utils [None req-e85eb228-e139-459c-9fd7-1e473621b4ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87057) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 17:08:28.780069 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e85eb228-e139-459c-9fd7-1e473621b4ec None None] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 17:08:28.784830 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c3e044a-cd30-46eb-ac63-7ac2c4cef917 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 17:08:28.785473 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3e044a-cd30-46eb-ac63-7ac2c4cef917 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 17:08:28.785753 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3e044a-cd30-46eb-ac63-7ac2c4cef917 admin admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 17:08:28.786051 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3e044a-cd30-46eb-ac63-7ac2c4cef917 admin admin] PATH_INFO: `/v3/projects/fd7f6a84b7df45ef8dafe2a75f4c5f25` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 17:08:28.787220 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c3e044a-cd30-46eb-ac63-7ac2c4cef917 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fd7f6a84b7df45ef8dafe2a75f4c5f25)` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 17:08:28.789300 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c3e044a-cd30-46eb-ac63-7ac2c4cef917 admin admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 17:08:28.872708 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828f4bef-2f6c-493f-a7f6-ef413bb00d37 None None] REQUEST_METHOD: `POST` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 17:08:28.873214 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828f4bef-2f6c-493f-a7f6-ef413bb00d37 None None] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 17:08:28.873339 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828f4bef-2f6c-493f-a7f6-ef413bb00d37 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 17:08:28.894801 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.auth.core [None req-828f4bef-2f6c-493f-a7f6-ef413bb00d37 None None] MFA Rules not processed for user `f3484bf0457a407d954de53bf9bc8a91`. Rule list: `[]` (Enabled: `True`). {{(pid=87058) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 17:08:28.896023 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.fernet_utils [None req-828f4bef-2f6c-493f-a7f6-ef413bb00d37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 17:08:28.908539 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.notifications [None req-4c3e044a-cd30-46eb-ac63-7ac2c4cef917 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fd7f6a84b7df45ef8dafe2a75f4c5f25', 'request_id': 'req-4c3e044a-cd30-46eb-ac63-7ac2c4cef917'} {{(pid=87057) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 17:08:28.915056 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.notifications [None req-4c3e044a-cd30-46eb-ac63-7ac2c4cef917 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fd7f6a84b7df45ef8dafe2a75f4c5f25', 'request_id': 'req-4c3e044a-cd30-46eb-ac63-7ac2c4cef917'} {{(pid=87057) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 17:08:28.918348 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG dbcounter [-] [87058] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 17:08:28.920750 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.notifications [None req-4c3e044a-cd30-46eb-ac63-7ac2c4cef917 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fd7f6a84b7df45ef8dafe2a75f4c5f25', 'request_id': 'req-4c3e044a-cd30-46eb-ac63-7ac2c4cef917'} {{(pid=87057) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 17:08:28.957885 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 1362/2717] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 1 17:08:28 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 17:08:28.971788 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.notifications [None req-4c3e044a-cd30-46eb-ac63-7ac2c4cef917 admin admin] The token cache is being invalidate because project fd7f6a84b7df45ef8dafe2a75f4c5f25 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87057) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 17:08:28.972083 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.notifications [None req-4c3e044a-cd30-46eb-ac63-7ac2c4cef917 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87057) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 17:08:28.972734 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.notifications [None req-4c3e044a-cd30-46eb-ac63-7ac2c4cef917 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87057) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 17:08:28.974308 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 1356/2718] 10.222.0.20 () {58 vars in 1233 bytes} [Mon Aug 1 17:08:28 2022] DELETE /identity/v3/projects/fd7f6a84b7df45ef8dafe2a75f4c5f25 => generated 0 bytes in 209 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 17:08:28.984683 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432820ca-5a43-4d60-b5d3-6ab90dd8d2db None None] REQUEST_METHOD: `POST` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 17:08:28.985022 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432820ca-5a43-4d60-b5d3-6ab90dd8d2db None None] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 17:08:28.985327 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432820ca-5a43-4d60-b5d3-6ab90dd8d2db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 17:08:29.007760 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.auth.core [None req-432820ca-5a43-4d60-b5d3-6ab90dd8d2db None None] MFA Rules not processed for user `1007f8e379b74e169b98fb7c3c0ba5b4`. Rule list: `[]` (Enabled: `True`). {{(pid=87058) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 17:08:29.009038 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.fernet_utils [None req-432820ca-5a43-4d60-b5d3-6ab90dd8d2db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 17:08:29.072963 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 1363/2719] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 1 17:08:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 17:08:29.081585 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a54557-3483-409b-88a6-3fb0878b0442 None None] REQUEST_METHOD: `POST` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 17:08:29.082102 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a54557-3483-409b-88a6-3fb0878b0442 None None] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 17:08:29.082230 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a54557-3483-409b-88a6-3fb0878b0442 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 17:08:29.103974 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.auth.core [None req-61a54557-3483-409b-88a6-3fb0878b0442 None None] MFA Rules not processed for user `1007f8e379b74e169b98fb7c3c0ba5b4`. Rule list: `[]` (Enabled: `True`). {{(pid=87057) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 17:08:29.105194 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.fernet_utils [None req-61a54557-3483-409b-88a6-3fb0878b0442 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87057) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 17:08:29.114402 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 1357/2720] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 1 17:08:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 17:08:29.118675 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-432820ca-5a43-4d60-b5d3-6ab90dd8d2db None None] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 17:08:29.129436 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-432820ca-5a43-4d60-b5d3-6ab90dd8d2db None None] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 17:08:29.134415 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c59bf2f6-cd1e-448d-a9d9-d36feb67dd73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 17:08:29.134891 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59bf2f6-cd1e-448d-a9d9-d36feb67dd73 admin admin] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 17:08:29.135176 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59bf2f6-cd1e-448d-a9d9-d36feb67dd73 admin admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 17:08:29.135448 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59bf2f6-cd1e-448d-a9d9-d36feb67dd73 admin admin] PATH_INFO: `/v3/domains` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 17:08:29.136219 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c59bf2f6-cd1e-448d-a9d9-d36feb67dd73 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87058) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 17:08:29.136515 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c59bf2f6-cd1e-448d-a9d9-d36feb67dd73 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 17:08:29.138133 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c59bf2f6-cd1e-448d-a9d9-d36feb67dd73 admin admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 17:08:29.150132 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 1364/2721] 10.222.0.20 () {58 vars in 1153 bytes} [Mon Aug 1 17:08:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 17:08:29.154355 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a54557-3483-409b-88a6-3fb0878b0442 None None] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 17:08:29.163950 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a54557-3483-409b-88a6-3fb0878b0442 None None] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 17:08:29.168774 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a456e16-4b31-4d41-939b-3cb128ea9348 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 17:08:29.169442 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a456e16-4b31-4d41-939b-3cb128ea9348 admin admin] REQUEST_METHOD: `POST` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 17:08:29.169729 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a456e16-4b31-4d41-939b-3cb128ea9348 admin admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 17:08:29.170042 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a456e16-4b31-4d41-939b-3cb128ea9348 admin admin] PATH_INFO: `/v3/projects` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 17:08:29.170737 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a456e16-4b31-4d41-939b-3cb128ea9348 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 17:08:29.172646 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a456e16-4b31-4d41-939b-3cb128ea9348 admin admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 17:08:29.258394 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 1358/2722] 10.222.0.20 () {60 vars in 1153 bytes} [Mon Aug 1 17:08:29 2022] POST /identity/v3/projects => generated 391 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 17:08:29.260598 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c59bf2f6-cd1e-448d-a9d9-d36feb67dd73 admin admin] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 17:08:29.271272 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c59bf2f6-cd1e-448d-a9d9-d36feb67dd73 admin admin] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 17:08:29.332108 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a23978b9-07c2-4ef1-b11b-7bb948cf1c16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 17:08:29.332761 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23978b9-07c2-4ef1-b11b-7bb948cf1c16 admin admin] REQUEST_METHOD: `POST` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 17:08:29.333045 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23978b9-07c2-4ef1-b11b-7bb948cf1c16 admin admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 17:08:29.333327 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23978b9-07c2-4ef1-b11b-7bb948cf1c16 admin admin] PATH_INFO: `/v3/users` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 17:08:29.334011 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a23978b9-07c2-4ef1-b11b-7bb948cf1c16 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 17:08:29.335975 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a23978b9-07c2-4ef1-b11b-7bb948cf1c16 admin admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 17:08:29.395898 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 1365/2723] 10.222.0.20 () {60 vars in 1144 bytes} [Mon Aug 1 17:08:29 2022] POST /identity/v3/users => generated 296 bytes in 136 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 17:08:29.400142 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a456e16-4b31-4d41-939b-3cb128ea9348 admin admin] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 17:08:29.411088 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a456e16-4b31-4d41-939b-3cb128ea9348 admin admin] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 17:08:29.416116 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e86088f-25e0-4241-a642-2179491affcd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 17:08:29.416738 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e86088f-25e0-4241-a642-2179491affcd admin admin] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 17:08:29.417017 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e86088f-25e0-4241-a642-2179491affcd admin admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 17:08:29.417291 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e86088f-25e0-4241-a642-2179491affcd admin admin] PATH_INFO: `/v3/roles` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 17:08:29.419744 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e86088f-25e0-4241-a642-2179491affcd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 17:08:29.419744 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e86088f-25e0-4241-a642-2179491affcd admin admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 17:08:29.433992 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 1359/2724] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Aug 1 17:08:29 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 17:08:29.435281 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a23978b9-07c2-4ef1-b11b-7bb948cf1c16 admin admin] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 17:08:29.445720 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a23978b9-07c2-4ef1-b11b-7bb948cf1c16 admin admin] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 17:08:29.450352 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b6cb056-931f-4c0c-a74c-f895bf341b68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 17:08:29.450966 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6cb056-931f-4c0c-a74c-f895bf341b68 admin admin] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 17:08:29.451243 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6cb056-931f-4c0c-a74c-f895bf341b68 admin admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 17:08:29.451519 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6cb056-931f-4c0c-a74c-f895bf341b68 admin admin] PATH_INFO: `/v3/roles` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 17:08:29.452214 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b6cb056-931f-4c0c-a74c-f895bf341b68 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 17:08:29.453818 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b6cb056-931f-4c0c-a74c-f895bf341b68 admin admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 17:08:29.465228 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 1366/2725] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Aug 1 17:08:29 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 17:08:29.469513 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e86088f-25e0-4241-a642-2179491affcd admin admin] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 17:08:29.480923 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e86088f-25e0-4241-a642-2179491affcd admin admin] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 17:08:29.485554 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f67d04d2-002a-407e-8ada-27f65bbe1008 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 17:08:29.486664 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67d04d2-002a-407e-8ada-27f65bbe1008 admin admin] REQUEST_METHOD: `PUT` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 17:08:29.487300 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67d04d2-002a-407e-8ada-27f65bbe1008 admin admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 17:08:29.487640 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67d04d2-002a-407e-8ada-27f65bbe1008 admin admin] PATH_INFO: `/v3/projects/99ce356724184bfab96ef639eff11716/users/e2b4bb16bf5e44bb85394165d4fd4411/roles/0999d3da63a545f18ff3252afdc90518` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 17:08:29.509992 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f67d04d2-002a-407e-8ada-27f65bbe1008 admin admin] RBAC: Authorizing `identity:create_grant(project_id=99ce356724184bfab96ef639eff11716, user_id=e2b4bb16bf5e44bb85394165d4fd4411, role_id=0999d3da63a545f18ff3252afdc90518)` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 17:08:29.512819 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f67d04d2-002a-407e-8ada-27f65bbe1008 admin admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 17:08:29.589630 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 1360/2726] 10.222.0.20 () {60 vars in 1483 bytes} [Mon Aug 1 17:08:29 2022] PUT /identity/v3/projects/99ce356724184bfab96ef639eff11716/users/e2b4bb16bf5e44bb85394165d4fd4411/roles/0999d3da63a545f18ff3252afdc90518 => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 17:08:29.601653 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5e04d2-4293-4292-84ea-888070d035f0 None None] REQUEST_METHOD: `POST` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 17:08:29.601944 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5e04d2-4293-4292-84ea-888070d035f0 None None] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 17:08:29.602293 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5e04d2-4293-4292-84ea-888070d035f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 17:08:29.631541 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.auth.core [None req-3f5e04d2-4293-4292-84ea-888070d035f0 None None] MFA Rules not processed for user `e2b4bb16bf5e44bb85394165d4fd4411`. Rule list: `[]` (Enabled: `True`). {{(pid=87058) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 17:08:29.632759 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.fernet_utils [None req-3f5e04d2-4293-4292-84ea-888070d035f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 17:08:29.656267 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG dbcounter [-] [87058] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 17:08:29.717641 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 1367/2727] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 1 17:08:29 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 17:08:29.728659 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f67d04d2-002a-407e-8ada-27f65bbe1008 admin admin] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 17:08:29.730491 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.fernet_utils [None req-f67d04d2-002a-407e-8ada-27f65bbe1008 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87057) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 17:08:29.754410 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG dbcounter [-] [87057] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=87057) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 17:08:29.803926 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f67d04d2-002a-407e-8ada-27f65bbe1008 admin admin] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 17:08:29.803926 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbdb47ca-d6f5-4ed4-a64b-1afe1f823472 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25970b52ca15448fa1abb1f974d64038', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b992d6a65df4bf9b27c13192d57c5f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 17:08:29.803926 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbdb47ca-d6f5-4ed4-a64b-1afe1f823472 service cinder] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 17:08:29.803926 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbdb47ca-d6f5-4ed4-a64b-1afe1f823472 service cinder] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 17:08:29.804341 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbdb47ca-d6f5-4ed4-a64b-1afe1f823472 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 17:08:29.807528 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbdb47ca-d6f5-4ed4-a64b-1afe1f823472 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 17:08:29.808980 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbdb47ca-d6f5-4ed4-a64b-1afe1f823472 service cinder] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 17:08:29.818607 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 1361/2728] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 1 17:08:29 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 17:08:39.720785 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG dbcounter [-] [87058] Writing DB stats keystone:SELECT=17 {{(pid=87058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 17:08:39.886404 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG dbcounter [-] [87057] Writing DB stats keystone:SELECT=17 {{(pid=87057) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 17:08:53.504749 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f5e04d2-4293-4292-84ea-888070d035f0 None None] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 17:08:53.567169 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f5e04d2-4293-4292-84ea-888070d035f0 None None] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 17:08:53.571629 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f4283e6-9fbf-4541-b1a2-ff6045120ff6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 17:08:53.572257 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4283e6-9fbf-4541-b1a2-ff6045120ff6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 17:08:53.572533 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4283e6-9fbf-4541-b1a2-ff6045120ff6 admin admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 17:08:53.572802 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4283e6-9fbf-4541-b1a2-ff6045120ff6 admin admin] PATH_INFO: `/v3/users/e2b4bb16bf5e44bb85394165d4fd4411` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 17:08:53.573943 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f4283e6-9fbf-4541-b1a2-ff6045120ff6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e2b4bb16bf5e44bb85394165d4fd4411)` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 17:08:53.575893 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f4283e6-9fbf-4541-b1a2-ff6045120ff6 admin admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 17:08:53.723072 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.notifications [None req-7f4283e6-9fbf-4541-b1a2-ff6045120ff6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e2b4bb16bf5e44bb85394165d4fd4411'} {{(pid=87058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 17:08:53.723815 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.notifications [None req-7f4283e6-9fbf-4541-b1a2-ff6045120ff6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e2b4bb16bf5e44bb85394165d4fd4411'} {{(pid=87058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 17:08:53.724421 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.notifications [None req-7f4283e6-9fbf-4541-b1a2-ff6045120ff6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e2b4bb16bf5e44bb85394165d4fd4411'} {{(pid=87058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 17:08:53.743470 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.notifications [None req-7f4283e6-9fbf-4541-b1a2-ff6045120ff6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e2b4bb16bf5e44bb85394165d4fd4411'} {{(pid=87058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 17:08:53.743990 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.notifications [None req-7f4283e6-9fbf-4541-b1a2-ff6045120ff6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e2b4bb16bf5e44bb85394165d4fd4411'} {{(pid=87058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 17:08:53.812721 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 1368/2729] 10.222.0.20 () {58 vars in 1224 bytes} [Mon Aug 1 17:08:53 2022] DELETE /identity/v3/users/e2b4bb16bf5e44bb85394165d4fd4411 => generated 0 bytes in 308 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 17:08:53.817731 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad420917-b615-4890-a6c2-ff260f3e20b9 None None] REQUEST_METHOD: `POST` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 17:08:53.818384 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad420917-b615-4890-a6c2-ff260f3e20b9 None None] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 17:08:53.818384 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad420917-b615-4890-a6c2-ff260f3e20b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 17:08:53.840505 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.auth.core [None req-ad420917-b615-4890-a6c2-ff260f3e20b9 None None] MFA Rules not processed for user `1007f8e379b74e169b98fb7c3c0ba5b4`. Rule list: `[]` (Enabled: `True`). {{(pid=87057) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 17:08:53.841769 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.fernet_utils [None req-ad420917-b615-4890-a6c2-ff260f3e20b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87057) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 17:08:53.906302 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 1362/2730] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 1 17:08:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 17:08:53.915964 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f4283e6-9fbf-4541-b1a2-ff6045120ff6 admin admin] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 17:08:53.917600 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.fernet_utils [None req-7f4283e6-9fbf-4541-b1a2-ff6045120ff6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 17:08:53.977643 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f4283e6-9fbf-4541-b1a2-ff6045120ff6 admin admin] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 17:08:53.981299 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc68c333-5ec3-42ee-aeea-abb293bf16f7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d9bd8d649154c979be3744bfd016f95', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b992d6a65df4bf9b27c13192d57c5f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 17:08:53.981910 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc68c333-5ec3-42ee-aeea-abb293bf16f7 service neutron] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 17:08:53.982222 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc68c333-5ec3-42ee-aeea-abb293bf16f7 service neutron] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 17:08:53.982500 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc68c333-5ec3-42ee-aeea-abb293bf16f7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 17:08:53.991836 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc68c333-5ec3-42ee-aeea-abb293bf16f7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 17:08:53.993262 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc68c333-5ec3-42ee-aeea-abb293bf16f7 service neutron] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 17:08:54.002951 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 1369/2731] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 1 17:08:53 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 17:08:54.396414 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad420917-b615-4890-a6c2-ff260f3e20b9 None None] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 17:08:54.397287 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.fernet_utils [None req-ad420917-b615-4890-a6c2-ff260f3e20b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87057) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 17:08:54.410424 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad420917-b615-4890-a6c2-ff260f3e20b9 None None] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 17:08:54.415654 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f7b413-b6ed-4b8f-841a-cfce4a58eb1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 17:08:54.416304 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f7b413-b6ed-4b8f-841a-cfce4a58eb1c admin admin] REQUEST_METHOD: `DELETE` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 17:08:54.419403 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f7b413-b6ed-4b8f-841a-cfce4a58eb1c admin admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 17:08:54.419403 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f7b413-b6ed-4b8f-841a-cfce4a58eb1c admin admin] PATH_INFO: `/v3/projects/99ce356724184bfab96ef639eff11716` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 17:08:54.419403 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00f7b413-b6ed-4b8f-841a-cfce4a58eb1c admin admin] RBAC: Authorizing `identity:delete_project(project_id=99ce356724184bfab96ef639eff11716)` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 17:08:54.420156 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00f7b413-b6ed-4b8f-841a-cfce4a58eb1c admin admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 17:08:54.531527 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.notifications [None req-00f7b413-b6ed-4b8f-841a-cfce4a58eb1c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '99ce356724184bfab96ef639eff11716', 'request_id': 'req-00f7b413-b6ed-4b8f-841a-cfce4a58eb1c'} {{(pid=87057) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 17:08:54.537838 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.notifications [None req-00f7b413-b6ed-4b8f-841a-cfce4a58eb1c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '99ce356724184bfab96ef639eff11716', 'request_id': 'req-00f7b413-b6ed-4b8f-841a-cfce4a58eb1c'} {{(pid=87057) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 17:08:54.543663 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.notifications [None req-00f7b413-b6ed-4b8f-841a-cfce4a58eb1c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '99ce356724184bfab96ef639eff11716', 'request_id': 'req-00f7b413-b6ed-4b8f-841a-cfce4a58eb1c'} {{(pid=87057) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 17:08:54.610720 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.notifications [None req-00f7b413-b6ed-4b8f-841a-cfce4a58eb1c admin admin] The token cache is being invalidate because project 99ce356724184bfab96ef639eff11716 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87057) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 17:08:54.611003 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.notifications [None req-00f7b413-b6ed-4b8f-841a-cfce4a58eb1c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87057) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 17:08:54.611612 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.notifications [None req-00f7b413-b6ed-4b8f-841a-cfce4a58eb1c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87057) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 17:08:54.614268 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 1363/2732] 10.222.0.20 () {58 vars in 1233 bytes} [Mon Aug 1 17:08:54 2022] DELETE /identity/v3/projects/99ce356724184bfab96ef639eff11716 => generated 0 bytes in 219 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 17:08:54.619193 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28f2017-0204-4f8e-8a91-c0107e1b560b None None] REQUEST_METHOD: `POST` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 17:08:54.622482 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28f2017-0204-4f8e-8a91-c0107e1b560b None None] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 17:08:54.622482 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28f2017-0204-4f8e-8a91-c0107e1b560b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 17:08:54.639601 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.auth.core [None req-a28f2017-0204-4f8e-8a91-c0107e1b560b None None] MFA Rules not processed for user `1007f8e379b74e169b98fb7c3c0ba5b4`. Rule list: `[]` (Enabled: `True`). {{(pid=87058) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 17:08:54.640805 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.fernet_utils [None req-a28f2017-0204-4f8e-8a91-c0107e1b560b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 17:08:54.657016 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG dbcounter [-] [87058] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 17:08:54.699009 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 1370/2733] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 1 17:08:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 17:08:54.706815 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24d9a2f-f31b-4d34-a7d1-5725f84ab52c None None] REQUEST_METHOD: `POST` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 17:08:54.707097 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24d9a2f-f31b-4d34-a7d1-5725f84ab52c None None] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 17:08:54.707374 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24d9a2f-f31b-4d34-a7d1-5725f84ab52c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 17:08:54.727378 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.auth.core [None req-d24d9a2f-f31b-4d34-a7d1-5725f84ab52c None None] MFA Rules not processed for user `1007f8e379b74e169b98fb7c3c0ba5b4`. Rule list: `[]` (Enabled: `True`). {{(pid=87057) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 17:08:54.728576 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.fernet_utils [None req-d24d9a2f-f31b-4d34-a7d1-5725f84ab52c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87057) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 17:08:54.741341 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 1364/2734] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 1 17:08:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 17:08:54.745767 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a28f2017-0204-4f8e-8a91-c0107e1b560b None None] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 17:08:54.758751 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a28f2017-0204-4f8e-8a91-c0107e1b560b None None] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 17:08:54.763861 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe8dbdc-94ae-418b-8bc7-6d593f357242 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 17:08:54.764472 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe8dbdc-94ae-418b-8bc7-6d593f357242 admin admin] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 17:08:54.764911 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe8dbdc-94ae-418b-8bc7-6d593f357242 admin admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 17:08:54.765251 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe8dbdc-94ae-418b-8bc7-6d593f357242 admin admin] PATH_INFO: `/v3/domains` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 17:08:54.766085 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebe8dbdc-94ae-418b-8bc7-6d593f357242 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87058) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 17:08:54.766430 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebe8dbdc-94ae-418b-8bc7-6d593f357242 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 17:08:54.768574 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebe8dbdc-94ae-418b-8bc7-6d593f357242 admin admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 17:08:54.805961 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 1371/2735] 10.222.0.20 () {58 vars in 1153 bytes} [Mon Aug 1 17:08:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 17:08:54.806370 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d24d9a2f-f31b-4d34-a7d1-5725f84ab52c None None] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 17:08:54.806370 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d24d9a2f-f31b-4d34-a7d1-5725f84ab52c None None] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 17:08:54.806370 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a91268c-389d-4997-a5e3-797a8ebab5ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 17:08:54.806370 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a91268c-389d-4997-a5e3-797a8ebab5ea admin admin] REQUEST_METHOD: `POST` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 17:08:54.806814 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a91268c-389d-4997-a5e3-797a8ebab5ea admin admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 17:08:54.806814 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a91268c-389d-4997-a5e3-797a8ebab5ea admin admin] PATH_INFO: `/v3/projects` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 17:08:54.806814 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a91268c-389d-4997-a5e3-797a8ebab5ea admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 17:08:54.808986 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a91268c-389d-4997-a5e3-797a8ebab5ea admin admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 17:08:54.912023 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 1365/2736] 10.222.0.20 () {60 vars in 1153 bytes} [Mon Aug 1 17:08:54 2022] POST /identity/v3/projects => generated 379 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 17:08:54.916226 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe8dbdc-94ae-418b-8bc7-6d593f357242 admin admin] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 17:08:54.928565 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe8dbdc-94ae-418b-8bc7-6d593f357242 admin admin] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 17:08:54.989219 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d5d0bf3-efb3-4b2c-b733-bc2e62bf80c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 17:08:54.989849 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5d0bf3-efb3-4b2c-b733-bc2e62bf80c2 admin admin] REQUEST_METHOD: `POST` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 17:08:54.990155 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5d0bf3-efb3-4b2c-b733-bc2e62bf80c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 17:08:54.990431 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5d0bf3-efb3-4b2c-b733-bc2e62bf80c2 admin admin] PATH_INFO: `/v3/users` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 17:08:54.991110 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d5d0bf3-efb3-4b2c-b733-bc2e62bf80c2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 17:08:54.993023 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d5d0bf3-efb3-4b2c-b733-bc2e62bf80c2 admin admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 17:08:55.069601 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 1372/2737] 10.222.0.20 () {60 vars in 1144 bytes} [Mon Aug 1 17:08:54 2022] POST /identity/v3/users => generated 290 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 17:08:55.070370 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a91268c-389d-4997-a5e3-797a8ebab5ea admin admin] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 17:08:55.080504 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a91268c-389d-4997-a5e3-797a8ebab5ea admin admin] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 17:08:55.085812 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-679c02a4-88f9-495b-9a98-f96a47eb419c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 17:08:55.086471 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679c02a4-88f9-495b-9a98-f96a47eb419c admin admin] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 17:08:55.086755 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679c02a4-88f9-495b-9a98-f96a47eb419c admin admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 17:08:55.087069 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679c02a4-88f9-495b-9a98-f96a47eb419c admin admin] PATH_INFO: `/v3/roles` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 17:08:55.087739 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-679c02a4-88f9-495b-9a98-f96a47eb419c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 17:08:55.089391 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-679c02a4-88f9-495b-9a98-f96a47eb419c admin admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 17:08:55.101209 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 1366/2738] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Aug 1 17:08:55 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 17:08:55.112946 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d5d0bf3-efb3-4b2c-b733-bc2e62bf80c2 admin admin] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 17:08:55.123552 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d5d0bf3-efb3-4b2c-b733-bc2e62bf80c2 admin admin] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 17:08:55.128257 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7d346f9-3ba7-4cc5-be8b-649d6355eea2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 17:08:55.132665 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d346f9-3ba7-4cc5-be8b-649d6355eea2 admin admin] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 17:08:55.132665 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d346f9-3ba7-4cc5-be8b-649d6355eea2 admin admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 17:08:55.132665 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d346f9-3ba7-4cc5-be8b-649d6355eea2 admin admin] PATH_INFO: `/v3/roles` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 17:08:55.132665 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7d346f9-3ba7-4cc5-be8b-649d6355eea2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 17:08:55.133588 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7d346f9-3ba7-4cc5-be8b-649d6355eea2 admin admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 17:08:55.144308 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 1373/2739] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Aug 1 17:08:55 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 17:08:55.148820 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-679c02a4-88f9-495b-9a98-f96a47eb419c admin admin] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 17:08:55.160483 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-679c02a4-88f9-495b-9a98-f96a47eb419c admin admin] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 17:08:55.165567 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-becc326c-fa40-44f4-9d6d-784afdce7c2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 17:08:55.166265 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-becc326c-fa40-44f4-9d6d-784afdce7c2b admin admin] REQUEST_METHOD: `PUT` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 17:08:55.166549 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-becc326c-fa40-44f4-9d6d-784afdce7c2b admin admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 17:08:55.166825 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-becc326c-fa40-44f4-9d6d-784afdce7c2b admin admin] PATH_INFO: `/v3/projects/c2cff009abd049a7aff67696680687ca/users/8ca353ec7cb948c9a803e6422ea3476e/roles/0999d3da63a545f18ff3252afdc90518` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 17:08:55.182147 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-becc326c-fa40-44f4-9d6d-784afdce7c2b admin admin] RBAC: Authorizing `identity:create_grant(project_id=c2cff009abd049a7aff67696680687ca, user_id=8ca353ec7cb948c9a803e6422ea3476e, role_id=0999d3da63a545f18ff3252afdc90518)` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 17:08:55.184841 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-becc326c-fa40-44f4-9d6d-784afdce7c2b admin admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 17:08:55.246817 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 1367/2740] 10.222.0.20 () {60 vars in 1483 bytes} [Mon Aug 1 17:08:55 2022] PUT /identity/v3/projects/c2cff009abd049a7aff67696680687ca/users/8ca353ec7cb948c9a803e6422ea3476e/roles/0999d3da63a545f18ff3252afdc90518 => generated 0 bytes in 98 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 17:08:55.252643 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9709ab-78ac-4e5b-bcf4-445af6126332 None None] REQUEST_METHOD: `POST` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 17:08:55.252949 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9709ab-78ac-4e5b-bcf4-445af6126332 None None] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 17:08:55.253284 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9709ab-78ac-4e5b-bcf4-445af6126332 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 17:08:55.274117 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.auth.core [None req-cc9709ab-78ac-4e5b-bcf4-445af6126332 None None] MFA Rules not processed for user `1007f8e379b74e169b98fb7c3c0ba5b4`. Rule list: `[]` (Enabled: `True`). {{(pid=87058) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 17:08:55.275357 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.fernet_utils [None req-cc9709ab-78ac-4e5b-bcf4-445af6126332 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 17:08:55.335794 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 1374/2741] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 1 17:08:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 17:08:55.346214 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-becc326c-fa40-44f4-9d6d-784afdce7c2b admin admin] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 17:08:55.348172 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.fernet_utils [None req-becc326c-fa40-44f4-9d6d-784afdce7c2b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87057) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 17:08:55.371043 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG dbcounter [-] [87057] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=87057) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 17:08:55.407573 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-becc326c-fa40-44f4-9d6d-784afdce7c2b admin admin] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 17:08:55.411246 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-182353c5-9795-4d17-8975-94f0a211494a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d9bd8d649154c979be3744bfd016f95', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b992d6a65df4bf9b27c13192d57c5f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 17:08:55.411965 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182353c5-9795-4d17-8975-94f0a211494a service neutron] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 17:08:55.412247 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182353c5-9795-4d17-8975-94f0a211494a service neutron] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 17:08:55.412523 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182353c5-9795-4d17-8975-94f0a211494a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 17:08:55.421841 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-182353c5-9795-4d17-8975-94f0a211494a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 17:08:55.423463 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-182353c5-9795-4d17-8975-94f0a211494a service neutron] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 17:08:55.433377 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 1368/2742] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 1 17:08:55 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 17:09:00.590862 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccce3755-2c43-44f0-afdc-c80d9db33bc8 None None] REQUEST_METHOD: `POST` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 17:09:00.591352 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccce3755-2c43-44f0-afdc-c80d9db33bc8 None None] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 17:09:00.591463 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccce3755-2c43-44f0-afdc-c80d9db33bc8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 17:09:00.626735 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.auth.core [None req-ccce3755-2c43-44f0-afdc-c80d9db33bc8 None None] MFA Rules not processed for user `8ca353ec7cb948c9a803e6422ea3476e`. Rule list: `[]` (Enabled: `True`). {{(pid=87058) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 17:09:00.628670 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.fernet_utils [None req-ccce3755-2c43-44f0-afdc-c80d9db33bc8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 17:09:00.649930 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG dbcounter [-] [87058] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 17:09:00.701091 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 1375/2743] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 1 17:09:00 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 17:09:00.711531 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-182353c5-9795-4d17-8975-94f0a211494a service neutron] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 17:09:00.728404 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-182353c5-9795-4d17-8975-94f0a211494a service neutron] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 17:09:00.734637 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93a22758-70b2-41c5-8b82-d8e4aee345cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 17:09:00.735262 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a22758-70b2-41c5-8b82-d8e4aee345cb admin admin] REQUEST_METHOD: `POST` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 17:09:00.735658 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a22758-70b2-41c5-8b82-d8e4aee345cb admin admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 17:09:00.735847 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a22758-70b2-41c5-8b82-d8e4aee345cb admin admin] PATH_INFO: `/v3/projects` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 17:09:00.736858 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93a22758-70b2-41c5-8b82-d8e4aee345cb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 17:09:00.738909 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93a22758-70b2-41c5-8b82-d8e4aee345cb admin admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 17:09:00.813484 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 1369/2744] 10.222.0.20 () {60 vars in 1153 bytes} [Mon Aug 1 17:09:00 2022] POST /identity/v3/projects => generated 377 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 17:09:00.818314 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccce3755-2c43-44f0-afdc-c80d9db33bc8 None None] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 17:09:00.829649 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccce3755-2c43-44f0-afdc-c80d9db33bc8 None None] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 17:09:00.890899 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-131d6d80-d330-471b-b027-4f8db061c6b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 17:09:00.891573 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131d6d80-d330-471b-b027-4f8db061c6b0 admin admin] REQUEST_METHOD: `POST` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 17:09:00.891855 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131d6d80-d330-471b-b027-4f8db061c6b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 17:09:00.892145 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131d6d80-d330-471b-b027-4f8db061c6b0 admin admin] PATH_INFO: `/v3/users` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 17:09:00.892826 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-131d6d80-d330-471b-b027-4f8db061c6b0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 17:09:00.895449 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-131d6d80-d330-471b-b027-4f8db061c6b0 admin admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 17:09:01.017762 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 1376/2745] 10.222.0.20 () {60 vars in 1144 bytes} [Mon Aug 1 17:09:00 2022] POST /identity/v3/users => generated 295 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 17:09:01.021813 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93a22758-70b2-41c5-8b82-d8e4aee345cb admin admin] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 17:09:01.032498 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93a22758-70b2-41c5-8b82-d8e4aee345cb admin admin] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 17:09:01.037697 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3abf892-025b-41cf-b59e-157121f01e03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 17:09:01.038378 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3abf892-025b-41cf-b59e-157121f01e03 admin admin] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 17:09:01.038666 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3abf892-025b-41cf-b59e-157121f01e03 admin admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 17:09:01.038945 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3abf892-025b-41cf-b59e-157121f01e03 admin admin] PATH_INFO: `/v3/roles` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 17:09:01.039656 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3abf892-025b-41cf-b59e-157121f01e03 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 17:09:01.041276 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3abf892-025b-41cf-b59e-157121f01e03 admin admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 17:09:01.053189 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 1370/2746] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Aug 1 17:09:01 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 17:09:01.057649 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-131d6d80-d330-471b-b027-4f8db061c6b0 admin admin] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 17:09:01.069965 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-131d6d80-d330-471b-b027-4f8db061c6b0 admin admin] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 17:09:01.075009 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb0177f-24b5-41b7-b6cb-1cd80fd8c19e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 17:09:01.081198 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb0177f-24b5-41b7-b6cb-1cd80fd8c19e admin admin] REQUEST_METHOD: `PUT` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 17:09:01.081198 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb0177f-24b5-41b7-b6cb-1cd80fd8c19e admin admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 17:09:01.081198 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb0177f-24b5-41b7-b6cb-1cd80fd8c19e admin admin] PATH_INFO: `/v3/projects/e82a966c39d74959b691be5aac26f0dd/users/916b40e076f34ef090e6c68882f25657/roles/440bb943dd5b46fea038b903b6eaaf81` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 17:09:01.093671 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afb0177f-24b5-41b7-b6cb-1cd80fd8c19e admin admin] RBAC: Authorizing `identity:create_grant(project_id=e82a966c39d74959b691be5aac26f0dd, user_id=916b40e076f34ef090e6c68882f25657, role_id=440bb943dd5b46fea038b903b6eaaf81)` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 17:09:01.096526 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afb0177f-24b5-41b7-b6cb-1cd80fd8c19e admin admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 17:09:01.201086 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 1377/2747] 10.222.0.20 () {60 vars in 1483 bytes} [Mon Aug 1 17:09:01 2022] PUT /identity/v3/projects/e82a966c39d74959b691be5aac26f0dd/users/916b40e076f34ef090e6c68882f25657/roles/440bb943dd5b46fea038b903b6eaaf81 => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 17:09:01.205113 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3abf892-025b-41cf-b59e-157121f01e03 admin admin] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 17:09:01.215818 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3abf892-025b-41cf-b59e-157121f01e03 admin admin] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 17:09:01.285475 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e8bd3c-2c64-48bc-b632-c4037403eebb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 17:09:01.286136 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e8bd3c-2c64-48bc-b632-c4037403eebb admin admin] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 17:09:01.286421 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e8bd3c-2c64-48bc-b632-c4037403eebb admin admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 17:09:01.286703 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e8bd3c-2c64-48bc-b632-c4037403eebb admin admin] PATH_INFO: `/v3/roles` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 17:09:01.287582 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74e8bd3c-2c64-48bc-b632-c4037403eebb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 17:09:01.289231 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74e8bd3c-2c64-48bc-b632-c4037403eebb admin admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 17:09:01.300605 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 1371/2748] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Aug 1 17:09:01 2022] GET /identity/v3/roles => generated 1420 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 17:09:01.304736 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb0177f-24b5-41b7-b6cb-1cd80fd8c19e admin admin] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 17:09:01.315278 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb0177f-24b5-41b7-b6cb-1cd80fd8c19e admin admin] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 17:09:01.320933 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f7ce41b-83be-4de7-9743-bdbbd0a03fad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 17:09:01.321611 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f7ce41b-83be-4de7-9743-bdbbd0a03fad admin admin] REQUEST_METHOD: `PUT` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 17:09:01.321955 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f7ce41b-83be-4de7-9743-bdbbd0a03fad admin admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 17:09:01.322385 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f7ce41b-83be-4de7-9743-bdbbd0a03fad admin admin] PATH_INFO: `/v3/projects/e82a966c39d74959b691be5aac26f0dd/users/916b40e076f34ef090e6c68882f25657/roles/440bb943dd5b46fea038b903b6eaaf81` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 17:09:01.324468 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f7ce41b-83be-4de7-9743-bdbbd0a03fad admin admin] RBAC: Authorizing `identity:create_grant(project_id=e82a966c39d74959b691be5aac26f0dd, user_id=916b40e076f34ef090e6c68882f25657, role_id=440bb943dd5b46fea038b903b6eaaf81)` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 17:09:01.327161 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f7ce41b-83be-4de7-9743-bdbbd0a03fad admin admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 17:09:01.338247 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 1378/2749] 10.222.0.20 () {60 vars in 1483 bytes} [Mon Aug 1 17:09:01 2022] PUT /identity/v3/projects/e82a966c39d74959b691be5aac26f0dd/users/916b40e076f34ef090e6c68882f25657/roles/440bb943dd5b46fea038b903b6eaaf81 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 17:09:06.512309 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ceac12-78b0-4507-aac2-9e1e86440de4 None None] REQUEST_METHOD: `POST` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 17:09:06.512825 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ceac12-78b0-4507-aac2-9e1e86440de4 None None] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 17:09:06.512928 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ceac12-78b0-4507-aac2-9e1e86440de4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 17:09:06.549045 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.auth.core [None req-45ceac12-78b0-4507-aac2-9e1e86440de4 None None] MFA Rules not processed for user `916b40e076f34ef090e6c68882f25657`. Rule list: `[]` (Enabled: `True`). {{(pid=87057) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 17:09:06.549045 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.fernet_utils [None req-45ceac12-78b0-4507-aac2-9e1e86440de4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87057) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 17:09:06.618530 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 1372/2750] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 1 17:09:06 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 17:09:06.635215 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f7ce41b-83be-4de7-9743-bdbbd0a03fad admin admin] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 17:09:06.637132 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.fernet_utils [None req-9f7ce41b-83be-4de7-9743-bdbbd0a03fad admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 17:09:06.718757 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f7ce41b-83be-4de7-9743-bdbbd0a03fad admin admin] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 17:09:06.725438 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c5cfbda-62c1-4095-986d-bf44ef433b83 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6533106d03eb4ca185a8839336d2aff9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b992d6a65df4bf9b27c13192d57c5f0', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 17:09:06.726740 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5cfbda-62c1-4095-986d-bf44ef433b83 service nova] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 17:09:06.727019 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5cfbda-62c1-4095-986d-bf44ef433b83 service nova] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 17:09:06.727286 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5cfbda-62c1-4095-986d-bf44ef433b83 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 17:09:06.733121 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG dbcounter [-] [87058] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=87058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 17:09:06.808379 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c5cfbda-62c1-4095-986d-bf44ef433b83 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 17:09:06.811586 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c5cfbda-62c1-4095-986d-bf44ef433b83 service nova] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 17:09:06.822462 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 1379/2751] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 1 17:09:06 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 188 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 17:09:16.615462 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG dbcounter [-] [87057] Writing DB stats keystone:SELECT=94,keystone:INSERT=1 {{(pid=87057) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 17:09:16.817832 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG dbcounter [-] [87058] Writing DB stats keystone:SELECT=24 {{(pid=87058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 17:09:33.167484 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73f28e1-bbf0-4979-9b6c-9677bfa84418 None None] REQUEST_METHOD: `POST` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 17:09:33.168022 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73f28e1-bbf0-4979-9b6c-9677bfa84418 None None] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 17:09:33.168022 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73f28e1-bbf0-4979-9b6c-9677bfa84418 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 17:09:33.187364 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.auth.core [None req-a73f28e1-bbf0-4979-9b6c-9677bfa84418 None None] MFA Rules not processed for user `8ca353ec7cb948c9a803e6422ea3476e`. Rule list: `[]` (Enabled: `True`). {{(pid=87057) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 17:09:33.188573 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.fernet_utils [None req-a73f28e1-bbf0-4979-9b6c-9677bfa84418 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87057) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 17:09:33.195930 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 1373/2752] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 1 17:09:33 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 17:09:33.508568 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c5cfbda-62c1-4095-986d-bf44ef433b83 service nova] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 17:09:33.568248 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c5cfbda-62c1-4095-986d-bf44ef433b83 service nova] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 17:09:33.572052 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b059a791-b4d5-4400-9171-914d0f97e266 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d9bd8d649154c979be3744bfd016f95', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b992d6a65df4bf9b27c13192d57c5f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 17:09:33.572668 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b059a791-b4d5-4400-9171-914d0f97e266 service neutron] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 17:09:33.572946 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b059a791-b4d5-4400-9171-914d0f97e266 service neutron] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 17:09:33.573219 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b059a791-b4d5-4400-9171-914d0f97e266 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 17:09:33.574946 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.fernet_utils [None req-b059a791-b4d5-4400-9171-914d0f97e266 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 17:09:33.584346 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b059a791-b4d5-4400-9171-914d0f97e266 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 17:09:33.585757 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b059a791-b4d5-4400-9171-914d0f97e266 service neutron] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 17:09:33.594381 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 1380/2753] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 1 17:09:33 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 17:09:43.182802 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG dbcounter [-] [87057] Writing DB stats keystone:SELECT=3 {{(pid=87057) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 17:09:43.590634 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG dbcounter [-] [87058] Writing DB stats keystone:SELECT=26 {{(pid=87058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 17:09:50.905041 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a73f28e1-bbf0-4979-9b6c-9677bfa84418 None None] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 17:09:50.914886 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a73f28e1-bbf0-4979-9b6c-9677bfa84418 None None] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 17:09:50.919041 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6249c674-4d4c-4dc5-9139-675e7c2bb8db service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d9bd8d649154c979be3744bfd016f95', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b992d6a65df4bf9b27c13192d57c5f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 17:09:50.919651 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6249c674-4d4c-4dc5-9139-675e7c2bb8db service neutron] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 17:09:50.919929 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6249c674-4d4c-4dc5-9139-675e7c2bb8db service neutron] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 17:09:50.920202 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6249c674-4d4c-4dc5-9139-675e7c2bb8db service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 17:09:50.929155 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6249c674-4d4c-4dc5-9139-675e7c2bb8db service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 17:09:50.930629 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6249c674-4d4c-4dc5-9139-675e7c2bb8db service neutron] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 17:09:50.940842 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 1374/2754] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 1 17:09:50 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 17:10:00.936362 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG dbcounter [-] [87057] Writing DB stats keystone:SELECT=6 {{(pid=87057) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 17:10:14.849559 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b059a791-b4d5-4400-9171-914d0f97e266 service neutron] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 17:10:14.850988 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.fernet_utils [None req-b059a791-b4d5-4400-9171-914d0f97e266 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 17:10:14.867850 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b059a791-b4d5-4400-9171-914d0f97e266 service neutron] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 17:10:14.873525 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbcbb344-97be-48d2-81db-6192f04a4e65 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6533106d03eb4ca185a8839336d2aff9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b992d6a65df4bf9b27c13192d57c5f0', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 17:10:14.874163 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbcbb344-97be-48d2-81db-6192f04a4e65 service nova] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 17:10:14.874447 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbcbb344-97be-48d2-81db-6192f04a4e65 service nova] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 17:10:14.874720 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbcbb344-97be-48d2-81db-6192f04a4e65 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 17:10:14.876527 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.fernet_utils [None req-bbcbb344-97be-48d2-81db-6192f04a4e65 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 17:10:14.937256 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbcbb344-97be-48d2-81db-6192f04a4e65 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 17:10:14.938863 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbcbb344-97be-48d2-81db-6192f04a4e65 service nova] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 17:10:14.947798 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 1381/2755] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 1 17:10:14 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 17:10:16.606021 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6249c674-4d4c-4dc5-9139-675e7c2bb8db service neutron] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 17:10:16.608067 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.fernet_utils [None req-6249c674-4d4c-4dc5-9139-675e7c2bb8db service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87057) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 17:10:16.667772 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6249c674-4d4c-4dc5-9139-675e7c2bb8db service neutron] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 17:10:16.671969 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5523d8f-7f2e-4703-acbd-cb9654707c31 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25970b52ca15448fa1abb1f974d64038', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b992d6a65df4bf9b27c13192d57c5f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 17:10:16.672581 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5523d8f-7f2e-4703-acbd-cb9654707c31 service cinder] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 17:10:16.672858 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5523d8f-7f2e-4703-acbd-cb9654707c31 service cinder] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 17:10:16.673130 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5523d8f-7f2e-4703-acbd-cb9654707c31 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 17:10:16.674880 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.fernet_utils [None req-d5523d8f-7f2e-4703-acbd-cb9654707c31 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87057) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 17:10:16.689125 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5523d8f-7f2e-4703-acbd-cb9654707c31 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 17:10:16.690591 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5523d8f-7f2e-4703-acbd-cb9654707c31 service cinder] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 17:10:16.701649 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 1375/2756] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 1 17:10:16 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 17:10:24.871242 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbcbb344-97be-48d2-81db-6192f04a4e65 service nova] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 17:10:24.881762 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbcbb344-97be-48d2-81db-6192f04a4e65 service nova] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 17:10:24.885816 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab6be8f-de52-4ccf-a4a3-711d2fd0c7e9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25970b52ca15448fa1abb1f974d64038', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b992d6a65df4bf9b27c13192d57c5f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 17:10:24.886466 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab6be8f-de52-4ccf-a4a3-711d2fd0c7e9 service cinder] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 17:10:24.886748 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab6be8f-de52-4ccf-a4a3-711d2fd0c7e9 service cinder] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 17:10:24.887024 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab6be8f-de52-4ccf-a4a3-711d2fd0c7e9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 17:10:24.888828 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.fernet_utils [None req-fab6be8f-de52-4ccf-a4a3-711d2fd0c7e9 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 17:10:24.903436 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fab6be8f-de52-4ccf-a4a3-711d2fd0c7e9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 17:10:24.904913 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fab6be8f-de52-4ccf-a4a3-711d2fd0c7e9 service cinder] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 17:10:24.916505 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 1382/2757] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 1 17:10:24 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 17:10:26.696547 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG dbcounter [-] [87057] Writing DB stats keystone:SELECT=26 {{(pid=87057) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 17:10:34.605568 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087430dc-3023-49e1-ad2f-e5ff4adf7242 None None] REQUEST_METHOD: `POST` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 17:10:34.606220 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087430dc-3023-49e1-ad2f-e5ff4adf7242 None None] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 17:10:34.606220 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087430dc-3023-49e1-ad2f-e5ff4adf7242 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 17:10:34.626908 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.auth.core [None req-087430dc-3023-49e1-ad2f-e5ff4adf7242 None None] MFA Rules not processed for user `f3484bf0457a407d954de53bf9bc8a91`. Rule list: `[]` (Enabled: `True`). {{(pid=87057) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 17:10:34.628112 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.fernet_utils [None req-087430dc-3023-49e1-ad2f-e5ff4adf7242 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87057) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 17:10:34.635183 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 1376/2758] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 1 17:10:34 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 17:10:34.911491 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG dbcounter [-] [87058] Writing DB stats keystone:SELECT=34 {{(pid=87058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 17:10:37.944751 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab6be8f-de52-4ccf-a4a3-711d2fd0c7e9 service cinder] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 17:10:37.946358 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.fernet_utils [None req-fab6be8f-de52-4ccf-a4a3-711d2fd0c7e9 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 17:10:37.957982 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab6be8f-de52-4ccf-a4a3-711d2fd0c7e9 service cinder] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 17:10:37.961545 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e19d59f6-0f24-42b9-bd20-13a2a1f5cd8d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d9bd8d649154c979be3744bfd016f95', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b992d6a65df4bf9b27c13192d57c5f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 17:10:37.962183 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19d59f6-0f24-42b9-bd20-13a2a1f5cd8d service neutron] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 17:10:37.962510 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19d59f6-0f24-42b9-bd20-13a2a1f5cd8d service neutron] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 17:10:37.962783 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19d59f6-0f24-42b9-bd20-13a2a1f5cd8d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 17:10:37.964466 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.fernet_utils [None req-e19d59f6-0f24-42b9-bd20-13a2a1f5cd8d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 17:10:37.974553 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e19d59f6-0f24-42b9-bd20-13a2a1f5cd8d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 17:10:37.975965 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e19d59f6-0f24-42b9-bd20-13a2a1f5cd8d service neutron] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 17:10:37.983873 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 1383/2759] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 1 17:10:37 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 17:10:44.620719 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG dbcounter [-] [87057] Writing DB stats keystone:SELECT=3 {{(pid=87057) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 17:10:47.980305 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG dbcounter [-] [87058] Writing DB stats keystone:SELECT=6 {{(pid=87058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 17:10:48.128643 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-087430dc-3023-49e1-ad2f-e5ff4adf7242 None None] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 17:10:48.142620 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-087430dc-3023-49e1-ad2f-e5ff4adf7242 None None] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 17:10:48.148282 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56c238c8-a54d-45a6-9582-a03f38c4fc7f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6533106d03eb4ca185a8839336d2aff9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b992d6a65df4bf9b27c13192d57c5f0', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 17:10:48.149019 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c238c8-a54d-45a6-9582-a03f38c4fc7f service nova] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 17:10:48.149303 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c238c8-a54d-45a6-9582-a03f38c4fc7f service nova] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 17:10:48.149575 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c238c8-a54d-45a6-9582-a03f38c4fc7f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 17:10:48.151382 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.fernet_utils [None req-56c238c8-a54d-45a6-9582-a03f38c4fc7f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87057) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 17:10:48.164728 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56c238c8-a54d-45a6-9582-a03f38c4fc7f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 17:10:48.166258 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56c238c8-a54d-45a6-9582-a03f38c4fc7f service nova] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 17:10:48.177192 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 1377/2760] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 1 17:10:48 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 17:10:58.172731 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG dbcounter [-] [87057] Writing DB stats keystone:SELECT=6 {{(pid=87057) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 17:12:14.446009 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e19d59f6-0f24-42b9-bd20-13a2a1f5cd8d service neutron] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 17:12:14.459255 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e19d59f6-0f24-42b9-bd20-13a2a1f5cd8d service neutron] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 17:12:14.465626 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92592e90-e690-485f-8a2c-452bb36254b9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6533106d03eb4ca185a8839336d2aff9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b992d6a65df4bf9b27c13192d57c5f0', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 17:12:14.466271 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92592e90-e690-485f-8a2c-452bb36254b9 service nova] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 17:12:14.466557 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92592e90-e690-485f-8a2c-452bb36254b9 service nova] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 17:12:14.471725 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92592e90-e690-485f-8a2c-452bb36254b9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 17:12:14.471725 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.fernet_utils [None req-92592e90-e690-485f-8a2c-452bb36254b9 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 17:12:14.482686 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92592e90-e690-485f-8a2c-452bb36254b9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 17:12:14.484563 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92592e90-e690-485f-8a2c-452bb36254b9 service nova] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 17:12:14.496763 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 1384/2761] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 1 17:12:14 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 17:12:16.809944 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4852843a-b41b-4bc3-a356-32a6c62f9af9 None None] REQUEST_METHOD: `POST` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 17:12:16.811191 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4852843a-b41b-4bc3-a356-32a6c62f9af9 None None] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 17:12:16.811564 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4852843a-b41b-4bc3-a356-32a6c62f9af9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 17:12:16.830923 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.auth.core [None req-4852843a-b41b-4bc3-a356-32a6c62f9af9 None None] MFA Rules not processed for user `8ca353ec7cb948c9a803e6422ea3476e`. Rule list: `[]` (Enabled: `True`). {{(pid=87057) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 17:12:16.832635 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.fernet_utils [None req-4852843a-b41b-4bc3-a356-32a6c62f9af9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87057) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 17:12:16.839720 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 1378/2762] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 1 17:12:16 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 17:12:17.884349 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92592e90-e690-485f-8a2c-452bb36254b9 service nova] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 17:12:17.885793 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.fernet_utils [None req-92592e90-e690-485f-8a2c-452bb36254b9 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 17:12:17.957840 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92592e90-e690-485f-8a2c-452bb36254b9 service nova] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 17:12:17.963508 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9facbcd-d3f0-4181-9ab2-db169d797869 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8362293261614623bcdabc4120a97a9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b992d6a65df4bf9b27c13192d57c5f0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 17:12:17.964154 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9facbcd-d3f0-4181-9ab2-db169d797869 service placement] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 17:12:17.964438 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9facbcd-d3f0-4181-9ab2-db169d797869 service placement] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 17:12:17.964715 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9facbcd-d3f0-4181-9ab2-db169d797869 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 17:12:17.966495 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.fernet_utils [None req-c9facbcd-d3f0-4181-9ab2-db169d797869 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 17:12:17.978398 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9facbcd-d3f0-4181-9ab2-db169d797869 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 17:12:17.979966 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9facbcd-d3f0-4181-9ab2-db169d797869 service placement] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 17:12:17.990037 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 1385/2763] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 1 17:12:17 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 17:12:26.312396 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4852843a-b41b-4bc3-a356-32a6c62f9af9 None None] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 17:12:26.322087 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4852843a-b41b-4bc3-a356-32a6c62f9af9 None None] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 17:12:26.325527 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20995304-75b7-4553-b8bb-c6ff1072033b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d9bd8d649154c979be3744bfd016f95', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b992d6a65df4bf9b27c13192d57c5f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 17:12:26.326165 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20995304-75b7-4553-b8bb-c6ff1072033b service neutron] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 17:12:26.326449 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20995304-75b7-4553-b8bb-c6ff1072033b service neutron] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 17:12:26.326721 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20995304-75b7-4553-b8bb-c6ff1072033b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 17:12:26.328426 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.fernet_utils [None req-20995304-75b7-4553-b8bb-c6ff1072033b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87057) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 17:12:26.390364 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20995304-75b7-4553-b8bb-c6ff1072033b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 17:12:26.391791 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20995304-75b7-4553-b8bb-c6ff1072033b service neutron] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 17:12:26.402102 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 1379/2764] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 1 17:12:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 17:12:27.985993 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG dbcounter [-] [87058] Writing DB stats keystone:SELECT=38 {{(pid=87058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 17:12:29.963158 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9facbcd-d3f0-4181-9ab2-db169d797869 service placement] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 17:12:29.964797 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.fernet_utils [None req-c9facbcd-d3f0-4181-9ab2-db169d797869 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 17:12:29.978106 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9facbcd-d3f0-4181-9ab2-db169d797869 service placement] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 17:12:29.983736 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39720266-6e72-4906-b4b0-13aa8fd44655 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8362293261614623bcdabc4120a97a9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b992d6a65df4bf9b27c13192d57c5f0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 17:12:29.984358 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39720266-6e72-4906-b4b0-13aa8fd44655 service placement] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 17:12:29.984639 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39720266-6e72-4906-b4b0-13aa8fd44655 service placement] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 17:12:29.985034 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39720266-6e72-4906-b4b0-13aa8fd44655 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 17:12:29.986930 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.fernet_utils [None req-39720266-6e72-4906-b4b0-13aa8fd44655 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 17:12:30.000383 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39720266-6e72-4906-b4b0-13aa8fd44655 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 17:12:30.001969 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39720266-6e72-4906-b4b0-13aa8fd44655 service placement] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 17:12:30.012829 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 1386/2765] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 1 17:12:29 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 17:12:30.092789 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20995304-75b7-4553-b8bb-c6ff1072033b service neutron] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 17:12:30.094342 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.fernet_utils [None req-20995304-75b7-4553-b8bb-c6ff1072033b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87057) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 17:12:30.106722 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20995304-75b7-4553-b8bb-c6ff1072033b service neutron] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 17:12:30.111691 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f97a441-a0e1-4da6-9c51-9c224486eeef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 17:12:30.112330 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f97a441-a0e1-4da6-9c51-9c224486eeef admin admin] REQUEST_METHOD: `DELETE` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 17:12:30.112611 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f97a441-a0e1-4da6-9c51-9c224486eeef admin admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 17:12:30.112881 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f97a441-a0e1-4da6-9c51-9c224486eeef admin admin] PATH_INFO: `/v3/users/f3484bf0457a407d954de53bf9bc8a91` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 17:12:30.114072 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f97a441-a0e1-4da6-9c51-9c224486eeef admin admin] RBAC: Authorizing `identity:delete_user(user_id=f3484bf0457a407d954de53bf9bc8a91)` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 17:12:30.115991 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f97a441-a0e1-4da6-9c51-9c224486eeef admin admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 17:12:30.277805 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.notifications [None req-5f97a441-a0e1-4da6-9c51-9c224486eeef admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f3484bf0457a407d954de53bf9bc8a91'} {{(pid=87057) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 17:12:30.278500 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.notifications [None req-5f97a441-a0e1-4da6-9c51-9c224486eeef admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f3484bf0457a407d954de53bf9bc8a91'} {{(pid=87057) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 17:12:30.343727 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.notifications [None req-5f97a441-a0e1-4da6-9c51-9c224486eeef admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f3484bf0457a407d954de53bf9bc8a91'} {{(pid=87057) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 17:12:30.344263 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.notifications [None req-5f97a441-a0e1-4da6-9c51-9c224486eeef admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f3484bf0457a407d954de53bf9bc8a91'} {{(pid=87057) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 17:12:30.352928 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.notifications [None req-5f97a441-a0e1-4da6-9c51-9c224486eeef admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f3484bf0457a407d954de53bf9bc8a91'} {{(pid=87057) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 17:12:30.373092 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 1380/2766] 10.222.0.20 () {58 vars in 1224 bytes} [Mon Aug 1 17:12:30 2022] DELETE /identity/v3/users/f3484bf0457a407d954de53bf9bc8a91 => generated 0 bytes in 281 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 17:12:30.609006 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39720266-6e72-4906-b4b0-13aa8fd44655 service placement] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 17:12:30.620551 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39720266-6e72-4906-b4b0-13aa8fd44655 service placement] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 17:12:30.685963 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d748ee7-8ea1-4fb2-968b-904a4b12c922 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 17:12:30.686643 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d748ee7-8ea1-4fb2-968b-904a4b12c922 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 17:12:30.686925 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d748ee7-8ea1-4fb2-968b-904a4b12c922 admin admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 17:12:30.687209 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d748ee7-8ea1-4fb2-968b-904a4b12c922 admin admin] PATH_INFO: `/v3/projects/29aa58f150be4722ad251dad9a7f1a7e` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 17:12:30.688378 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d748ee7-8ea1-4fb2-968b-904a4b12c922 admin admin] RBAC: Authorizing `identity:delete_project(project_id=29aa58f150be4722ad251dad9a7f1a7e)` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 17:12:30.690355 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d748ee7-8ea1-4fb2-968b-904a4b12c922 admin admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 17:12:30.813311 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.notifications [None req-3d748ee7-8ea1-4fb2-968b-904a4b12c922 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '29aa58f150be4722ad251dad9a7f1a7e', 'request_id': 'req-3d748ee7-8ea1-4fb2-968b-904a4b12c922'} {{(pid=87058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 17:12:30.820091 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.notifications [None req-3d748ee7-8ea1-4fb2-968b-904a4b12c922 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '29aa58f150be4722ad251dad9a7f1a7e', 'request_id': 'req-3d748ee7-8ea1-4fb2-968b-904a4b12c922'} {{(pid=87058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 17:12:30.825789 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.notifications [None req-3d748ee7-8ea1-4fb2-968b-904a4b12c922 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '29aa58f150be4722ad251dad9a7f1a7e', 'request_id': 'req-3d748ee7-8ea1-4fb2-968b-904a4b12c922'} {{(pid=87058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 17:12:30.889774 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.notifications [None req-3d748ee7-8ea1-4fb2-968b-904a4b12c922 admin admin] The token cache is being invalidate because project 29aa58f150be4722ad251dad9a7f1a7e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87058) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 17:12:30.890062 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.notifications [None req-3d748ee7-8ea1-4fb2-968b-904a4b12c922 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 17:12:30.891393 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.notifications [None req-3d748ee7-8ea1-4fb2-968b-904a4b12c922 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 17:12:30.893207 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 1387/2767] 10.222.0.20 () {58 vars in 1233 bytes} [Mon Aug 1 17:12:30 2022] DELETE /identity/v3/projects/29aa58f150be4722ad251dad9a7f1a7e => generated 0 bytes in 285 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 17:12:30.906506 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7460149c-d006-46e1-84c7-665bde1e1d01 None None] REQUEST_METHOD: `POST` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 17:12:30.906788 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7460149c-d006-46e1-84c7-665bde1e1d01 None None] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 17:12:30.907262 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7460149c-d006-46e1-84c7-665bde1e1d01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 17:12:30.932754 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.auth.core [None req-7460149c-d006-46e1-84c7-665bde1e1d01 None None] MFA Rules not processed for user `1007f8e379b74e169b98fb7c3c0ba5b4`. Rule list: `[]` (Enabled: `True`). {{(pid=87057) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 17:12:30.933959 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.fernet_utils [None req-7460149c-d006-46e1-84c7-665bde1e1d01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87057) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 17:12:30.983710 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG dbcounter [-] [87057] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87057) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 17:12:30.991988 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 1381/2768] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 1 17:12:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 17:12:31.001702 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c4f9ac-333a-4688-8e4f-a1576e2ed8df None None] REQUEST_METHOD: `POST` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 17:12:31.002209 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c4f9ac-333a-4688-8e4f-a1576e2ed8df None None] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 17:12:31.002318 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c4f9ac-333a-4688-8e4f-a1576e2ed8df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 17:12:31.021367 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.auth.core [None req-81c4f9ac-333a-4688-8e4f-a1576e2ed8df None None] MFA Rules not processed for user `1007f8e379b74e169b98fb7c3c0ba5b4`. Rule list: `[]` (Enabled: `True`). {{(pid=87058) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 17:12:31.022625 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.fernet_utils [None req-81c4f9ac-333a-4688-8e4f-a1576e2ed8df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 17:12:31.030741 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 1388/2769] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 1 17:12:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 17:12:31.034829 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7460149c-d006-46e1-84c7-665bde1e1d01 None None] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 17:12:31.045354 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7460149c-d006-46e1-84c7-665bde1e1d01 None None] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 17:12:31.050331 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b7021f7-64e9-4340-bd0d-012a0e8d2f14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 17:12:31.050947 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7021f7-64e9-4340-bd0d-012a0e8d2f14 admin admin] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 17:12:31.051229 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7021f7-64e9-4340-bd0d-012a0e8d2f14 admin admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 17:12:31.051510 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7021f7-64e9-4340-bd0d-012a0e8d2f14 admin admin] PATH_INFO: `/v3/domains` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 17:12:31.052208 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b7021f7-64e9-4340-bd0d-012a0e8d2f14 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87057) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 17:12:31.052516 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b7021f7-64e9-4340-bd0d-012a0e8d2f14 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 17:12:31.054155 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b7021f7-64e9-4340-bd0d-012a0e8d2f14 admin admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 17:12:31.066393 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 1382/2770] 10.222.0.20 () {58 vars in 1153 bytes} [Mon Aug 1 17:12:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 17:12:31.070803 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c4f9ac-333a-4688-8e4f-a1576e2ed8df None None] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 17:12:31.080046 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c4f9ac-333a-4688-8e4f-a1576e2ed8df None None] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 17:12:31.084730 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e61311d-6846-4cab-9cfa-663fedbac7cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 17:12:31.085333 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e61311d-6846-4cab-9cfa-663fedbac7cd admin admin] REQUEST_METHOD: `POST` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 17:12:31.085615 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e61311d-6846-4cab-9cfa-663fedbac7cd admin admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 17:12:31.085903 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e61311d-6846-4cab-9cfa-663fedbac7cd admin admin] PATH_INFO: `/v3/projects` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 17:12:31.086610 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e61311d-6846-4cab-9cfa-663fedbac7cd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 17:12:31.088499 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e61311d-6846-4cab-9cfa-663fedbac7cd admin admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 17:12:31.156809 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 1389/2771] 10.222.0.20 () {60 vars in 1153 bytes} [Mon Aug 1 17:12:31 2022] POST /identity/v3/projects => generated 373 bytes in 86 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 17:12:31.160869 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b7021f7-64e9-4340-bd0d-012a0e8d2f14 admin admin] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 17:12:31.172679 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b7021f7-64e9-4340-bd0d-012a0e8d2f14 admin admin] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 17:12:31.232919 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64c72cfa-897c-4bad-8429-43f057ad0a3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 17:12:31.233873 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c72cfa-897c-4bad-8429-43f057ad0a3f admin admin] REQUEST_METHOD: `POST` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 17:12:31.234188 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c72cfa-897c-4bad-8429-43f057ad0a3f admin admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 17:12:31.234465 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c72cfa-897c-4bad-8429-43f057ad0a3f admin admin] PATH_INFO: `/v3/users` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 17:12:31.235176 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64c72cfa-897c-4bad-8429-43f057ad0a3f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 17:12:31.237110 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64c72cfa-897c-4bad-8429-43f057ad0a3f admin admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 17:12:31.293456 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 1383/2772] 10.222.0.20 () {60 vars in 1144 bytes} [Mon Aug 1 17:12:31 2022] POST /identity/v3/users => generated 287 bytes in 133 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 17:12:31.297596 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e61311d-6846-4cab-9cfa-663fedbac7cd admin admin] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 17:12:31.308531 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e61311d-6846-4cab-9cfa-663fedbac7cd admin admin] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 17:12:31.313231 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad2aaf3-44e8-4c7d-b3db-ac63e9db8912 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 17:12:31.313844 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad2aaf3-44e8-4c7d-b3db-ac63e9db8912 admin admin] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 17:12:31.314157 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad2aaf3-44e8-4c7d-b3db-ac63e9db8912 admin admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 17:12:31.314448 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad2aaf3-44e8-4c7d-b3db-ac63e9db8912 admin admin] PATH_INFO: `/v3/roles` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 17:12:31.315295 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fad2aaf3-44e8-4c7d-b3db-ac63e9db8912 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 17:12:31.316891 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fad2aaf3-44e8-4c7d-b3db-ac63e9db8912 admin admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 17:12:31.329984 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 1390/2773] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Aug 1 17:12:31 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 17:12:31.331250 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64c72cfa-897c-4bad-8429-43f057ad0a3f admin admin] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 17:12:31.342112 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64c72cfa-897c-4bad-8429-43f057ad0a3f admin admin] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 17:12:31.347163 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d524030d-df04-42cf-954d-6e0e1b89a3a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 17:12:31.348059 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d524030d-df04-42cf-954d-6e0e1b89a3a1 admin admin] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 17:12:31.348415 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d524030d-df04-42cf-954d-6e0e1b89a3a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 17:12:31.348750 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d524030d-df04-42cf-954d-6e0e1b89a3a1 admin admin] PATH_INFO: `/v3/roles` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 17:12:31.349544 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d524030d-df04-42cf-954d-6e0e1b89a3a1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 17:12:31.351267 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d524030d-df04-42cf-954d-6e0e1b89a3a1 admin admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 17:12:31.363279 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 1384/2774] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Aug 1 17:12:31 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 01 17:12:31.367513 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad2aaf3-44e8-4c7d-b3db-ac63e9db8912 admin admin] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 17:12:31.377860 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad2aaf3-44e8-4c7d-b3db-ac63e9db8912 admin admin] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 17:12:31.382693 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48430156-32d4-42a1-8502-d5dc7660169a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 17:12:31.383364 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48430156-32d4-42a1-8502-d5dc7660169a admin admin] REQUEST_METHOD: `PUT` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 17:12:31.383647 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48430156-32d4-42a1-8502-d5dc7660169a admin admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 17:12:31.383928 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48430156-32d4-42a1-8502-d5dc7660169a admin admin] PATH_INFO: `/v3/projects/5cf76caff097453487a7eed0dd7801e9/users/b88ebdd7b20743e285c2b98397a359be/roles/0999d3da63a545f18ff3252afdc90518` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 17:12:31.399316 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48430156-32d4-42a1-8502-d5dc7660169a admin admin] RBAC: Authorizing `identity:create_grant(project_id=5cf76caff097453487a7eed0dd7801e9, user_id=b88ebdd7b20743e285c2b98397a359be, role_id=0999d3da63a545f18ff3252afdc90518)` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 17:12:31.402623 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48430156-32d4-42a1-8502-d5dc7660169a admin admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 17:12:31.481683 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 1391/2775] 10.222.0.20 () {60 vars in 1483 bytes} [Mon Aug 1 17:12:31 2022] PUT /identity/v3/projects/5cf76caff097453487a7eed0dd7801e9/users/b88ebdd7b20743e285c2b98397a359be/roles/0999d3da63a545f18ff3252afdc90518 => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 17:12:31.492173 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96505dc-cd46-48b4-9fb9-79d70c21fae4 None None] REQUEST_METHOD: `POST` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 17:12:31.492463 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96505dc-cd46-48b4-9fb9-79d70c21fae4 None None] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 17:12:31.492741 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96505dc-cd46-48b4-9fb9-79d70c21fae4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 17:12:31.520176 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.auth.core [None req-b96505dc-cd46-48b4-9fb9-79d70c21fae4 None None] MFA Rules not processed for user `b88ebdd7b20743e285c2b98397a359be`. Rule list: `[]` (Enabled: `True`). {{(pid=87057) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 17:12:31.521386 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.fernet_utils [None req-b96505dc-cd46-48b4-9fb9-79d70c21fae4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87057) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 17:12:31.584846 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 1385/2776] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 1 17:12:31 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 17:12:31.596661 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48430156-32d4-42a1-8502-d5dc7660169a admin admin] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 17:12:31.598383 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.fernet_utils [None req-48430156-32d4-42a1-8502-d5dc7660169a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 17:12:31.613448 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG dbcounter [-] [87058] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=87058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 17:12:31.672808 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48430156-32d4-42a1-8502-d5dc7660169a admin admin] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 17:12:31.679103 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6352834c-e4c1-4a5c-b714-03e11e202b0c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6533106d03eb4ca185a8839336d2aff9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b992d6a65df4bf9b27c13192d57c5f0', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 17:12:31.679851 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6352834c-e4c1-4a5c-b714-03e11e202b0c service nova] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 17:12:31.680129 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6352834c-e4c1-4a5c-b714-03e11e202b0c service nova] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 17:12:31.680397 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6352834c-e4c1-4a5c-b714-03e11e202b0c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 17:12:31.689283 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6352834c-e4c1-4a5c-b714-03e11e202b0c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 17:12:31.690969 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6352834c-e4c1-4a5c-b714-03e11e202b0c service nova] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 17:12:31.700547 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 1392/2777] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 1 17:12:31 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 17:12:32.799186 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b96505dc-cd46-48b4-9fb9-79d70c21fae4 None None] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 17:12:32.812196 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b96505dc-cd46-48b4-9fb9-79d70c21fae4 None None] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 17:12:32.817623 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-012fcd58-403b-413d-9fc6-74dfc94a71c5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6533106d03eb4ca185a8839336d2aff9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b992d6a65df4bf9b27c13192d57c5f0', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 17:12:32.818264 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012fcd58-403b-413d-9fc6-74dfc94a71c5 service nova] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 17:12:32.818547 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012fcd58-403b-413d-9fc6-74dfc94a71c5 service nova] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 17:12:32.818836 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012fcd58-403b-413d-9fc6-74dfc94a71c5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 17:12:32.820628 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.fernet_utils [None req-012fcd58-403b-413d-9fc6-74dfc94a71c5 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87057) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 17:12:32.832773 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-012fcd58-403b-413d-9fc6-74dfc94a71c5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 17:12:32.834440 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-012fcd58-403b-413d-9fc6-74dfc94a71c5 service nova] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 17:12:32.845808 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 1386/2778] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 1 17:12:32 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 17:12:34.862002 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6352834c-e4c1-4a5c-b714-03e11e202b0c service nova] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 17:12:34.924865 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6352834c-e4c1-4a5c-b714-03e11e202b0c service nova] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 17:12:34.929304 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eaa3282-b88a-4e4d-87b5-162f45604d58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 17:12:34.929935 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eaa3282-b88a-4e4d-87b5-162f45604d58 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 17:12:34.930250 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eaa3282-b88a-4e4d-87b5-162f45604d58 admin admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 17:12:34.930521 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eaa3282-b88a-4e4d-87b5-162f45604d58 admin admin] PATH_INFO: `/v3/users/b88ebdd7b20743e285c2b98397a359be` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 17:12:34.931670 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eaa3282-b88a-4e4d-87b5-162f45604d58 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b88ebdd7b20743e285c2b98397a359be)` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 17:12:34.934140 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eaa3282-b88a-4e4d-87b5-162f45604d58 admin admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 17:12:35.077556 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.notifications [None req-0eaa3282-b88a-4e4d-87b5-162f45604d58 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b88ebdd7b20743e285c2b98397a359be'} {{(pid=87058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 17:12:35.078348 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.notifications [None req-0eaa3282-b88a-4e4d-87b5-162f45604d58 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b88ebdd7b20743e285c2b98397a359be'} {{(pid=87058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 17:12:35.078997 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.notifications [None req-0eaa3282-b88a-4e4d-87b5-162f45604d58 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b88ebdd7b20743e285c2b98397a359be'} {{(pid=87058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 17:12:35.091765 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.notifications [None req-0eaa3282-b88a-4e4d-87b5-162f45604d58 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b88ebdd7b20743e285c2b98397a359be'} {{(pid=87058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 17:12:35.098904 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.notifications [None req-0eaa3282-b88a-4e4d-87b5-162f45604d58 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b88ebdd7b20743e285c2b98397a359be'} {{(pid=87058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 17:12:35.172626 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 1393/2779] 10.222.0.20 () {58 vars in 1224 bytes} [Mon Aug 1 17:12:34 2022] DELETE /identity/v3/users/b88ebdd7b20743e285c2b98397a359be => generated 0 bytes in 314 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 17:12:35.177052 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6055b470-a966-4bd3-8e14-fee31833088a None None] REQUEST_METHOD: `POST` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 17:12:35.177712 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6055b470-a966-4bd3-8e14-fee31833088a None None] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 17:12:35.177712 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6055b470-a966-4bd3-8e14-fee31833088a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 17:12:35.196677 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.auth.core [None req-6055b470-a966-4bd3-8e14-fee31833088a None None] MFA Rules not processed for user `1007f8e379b74e169b98fb7c3c0ba5b4`. Rule list: `[]` (Enabled: `True`). {{(pid=87057) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 17:12:35.200284 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.fernet_utils [None req-6055b470-a966-4bd3-8e14-fee31833088a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87057) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 17:12:35.236617 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG dbcounter [-] [87057] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87057) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 17:12:35.257716 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 1387/2780] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 1 17:12:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 17:12:35.268174 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eaa3282-b88a-4e4d-87b5-162f45604d58 admin admin] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 17:12:35.269665 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.fernet_utils [None req-0eaa3282-b88a-4e4d-87b5-162f45604d58 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 17:12:35.289847 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG dbcounter [-] [87058] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 17:12:35.326761 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eaa3282-b88a-4e4d-87b5-162f45604d58 admin admin] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 17:12:35.330439 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6814f3d-29fa-442d-8c1e-d5a0997089a6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d9bd8d649154c979be3744bfd016f95', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b992d6a65df4bf9b27c13192d57c5f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 17:12:35.331010 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6814f3d-29fa-442d-8c1e-d5a0997089a6 service neutron] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 17:12:35.331267 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6814f3d-29fa-442d-8c1e-d5a0997089a6 service neutron] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 17:12:35.331535 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6814f3d-29fa-442d-8c1e-d5a0997089a6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 17:12:35.341002 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6814f3d-29fa-442d-8c1e-d5a0997089a6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 17:12:35.342467 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6814f3d-29fa-442d-8c1e-d5a0997089a6 service neutron] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 17:12:35.351631 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 1394/2781] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 1 17:12:35 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 17:12:35.753259 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6055b470-a966-4bd3-8e14-fee31833088a None None] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 17:12:35.753259 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.fernet_utils [None req-6055b470-a966-4bd3-8e14-fee31833088a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87057) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 17:12:35.764913 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6055b470-a966-4bd3-8e14-fee31833088a None None] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 17:12:35.769527 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-826ed34f-e066-42df-8c9d-f82087f9a371 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 17:12:35.770205 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826ed34f-e066-42df-8c9d-f82087f9a371 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 17:12:35.770486 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826ed34f-e066-42df-8c9d-f82087f9a371 admin admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 17:12:35.770796 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826ed34f-e066-42df-8c9d-f82087f9a371 admin admin] PATH_INFO: `/v3/projects/5cf76caff097453487a7eed0dd7801e9` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 17:12:35.772064 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-826ed34f-e066-42df-8c9d-f82087f9a371 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5cf76caff097453487a7eed0dd7801e9)` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 17:12:35.773978 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-826ed34f-e066-42df-8c9d-f82087f9a371 admin admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 17:12:35.908848 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.notifications [None req-826ed34f-e066-42df-8c9d-f82087f9a371 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5cf76caff097453487a7eed0dd7801e9', 'request_id': 'req-826ed34f-e066-42df-8c9d-f82087f9a371'} {{(pid=87057) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 17:12:35.915553 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.notifications [None req-826ed34f-e066-42df-8c9d-f82087f9a371 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5cf76caff097453487a7eed0dd7801e9', 'request_id': 'req-826ed34f-e066-42df-8c9d-f82087f9a371'} {{(pid=87057) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 17:12:35.922696 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.notifications [None req-826ed34f-e066-42df-8c9d-f82087f9a371 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5cf76caff097453487a7eed0dd7801e9', 'request_id': 'req-826ed34f-e066-42df-8c9d-f82087f9a371'} {{(pid=87057) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 17:12:36.018310 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.notifications [None req-826ed34f-e066-42df-8c9d-f82087f9a371 admin admin] The token cache is being invalidate because project 5cf76caff097453487a7eed0dd7801e9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87057) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 17:12:36.018602 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.notifications [None req-826ed34f-e066-42df-8c9d-f82087f9a371 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87057) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 17:12:36.019226 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.notifications [None req-826ed34f-e066-42df-8c9d-f82087f9a371 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87057) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 17:12:36.020756 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 1388/2782] 10.222.0.20 () {58 vars in 1233 bytes} [Mon Aug 1 17:12:35 2022] DELETE /identity/v3/projects/5cf76caff097453487a7eed0dd7801e9 => generated 0 bytes in 270 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 17:12:36.025572 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3619126-6e24-4cb1-bd9d-e9c1a3d4b8a2 None None] REQUEST_METHOD: `POST` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 17:12:36.026061 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3619126-6e24-4cb1-bd9d-e9c1a3d4b8a2 None None] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 17:12:36.026171 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3619126-6e24-4cb1-bd9d-e9c1a3d4b8a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 17:12:36.046390 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.auth.core [None req-d3619126-6e24-4cb1-bd9d-e9c1a3d4b8a2 None None] MFA Rules not processed for user `1007f8e379b74e169b98fb7c3c0ba5b4`. Rule list: `[]` (Enabled: `True`). {{(pid=87058) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 17:12:36.047601 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.fernet_utils [None req-d3619126-6e24-4cb1-bd9d-e9c1a3d4b8a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 17:12:36.107522 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 1395/2783] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 1 17:12:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 17:12:36.116578 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea758d58-fb3c-483d-a805-0bebf09e78ae None None] REQUEST_METHOD: `POST` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 17:12:36.116953 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea758d58-fb3c-483d-a805-0bebf09e78ae None None] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 17:12:36.117808 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea758d58-fb3c-483d-a805-0bebf09e78ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 17:12:36.138455 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.auth.core [None req-ea758d58-fb3c-483d-a805-0bebf09e78ae None None] MFA Rules not processed for user `1007f8e379b74e169b98fb7c3c0ba5b4`. Rule list: `[]` (Enabled: `True`). {{(pid=87057) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 17:12:36.139791 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.fernet_utils [None req-ea758d58-fb3c-483d-a805-0bebf09e78ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87057) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 17:12:36.147869 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 1389/2784] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 1 17:12:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 17:12:36.156707 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3619126-6e24-4cb1-bd9d-e9c1a3d4b8a2 None None] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 17:12:36.167046 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3619126-6e24-4cb1-bd9d-e9c1a3d4b8a2 None None] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 17:12:36.172950 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c149497-fa5c-4cfc-bd27-80521835ff8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 17:12:36.173713 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c149497-fa5c-4cfc-bd27-80521835ff8a admin admin] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 17:12:36.173996 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c149497-fa5c-4cfc-bd27-80521835ff8a admin admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 17:12:36.174298 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c149497-fa5c-4cfc-bd27-80521835ff8a admin admin] PATH_INFO: `/v3/domains` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 17:12:36.175008 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c149497-fa5c-4cfc-bd27-80521835ff8a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87058) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 17:12:36.175317 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c149497-fa5c-4cfc-bd27-80521835ff8a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 17:12:36.176919 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c149497-fa5c-4cfc-bd27-80521835ff8a admin admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 17:12:36.192330 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 1396/2785] 10.222.0.20 () {58 vars in 1153 bytes} [Mon Aug 1 17:12:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 17:12:36.193506 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea758d58-fb3c-483d-a805-0bebf09e78ae None None] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 17:12:36.203770 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea758d58-fb3c-483d-a805-0bebf09e78ae None None] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 17:12:36.208788 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-687cd49c-3fd0-4da3-bbc5-eadd0da5c033 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 17:12:36.210672 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687cd49c-3fd0-4da3-bbc5-eadd0da5c033 admin admin] REQUEST_METHOD: `POST` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 17:12:36.210672 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687cd49c-3fd0-4da3-bbc5-eadd0da5c033 admin admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 17:12:36.210672 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687cd49c-3fd0-4da3-bbc5-eadd0da5c033 admin admin] PATH_INFO: `/v3/projects` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 17:12:36.211083 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-687cd49c-3fd0-4da3-bbc5-eadd0da5c033 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 17:12:36.212700 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-687cd49c-3fd0-4da3-bbc5-eadd0da5c033 admin admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 17:12:36.319919 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 1390/2786] 10.222.0.20 () {60 vars in 1153 bytes} [Mon Aug 1 17:12:36 2022] POST /identity/v3/projects => generated 371 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 17:12:36.321372 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c149497-fa5c-4cfc-bd27-80521835ff8a admin admin] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 17:12:36.332350 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c149497-fa5c-4cfc-bd27-80521835ff8a admin admin] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 17:12:36.391048 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c353bbb3-cbc5-425a-81ee-2e4aafa31b02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 17:12:36.391672 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c353bbb3-cbc5-425a-81ee-2e4aafa31b02 admin admin] REQUEST_METHOD: `POST` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 17:12:36.391954 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c353bbb3-cbc5-425a-81ee-2e4aafa31b02 admin admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 17:12:36.392233 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c353bbb3-cbc5-425a-81ee-2e4aafa31b02 admin admin] PATH_INFO: `/v3/users` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 17:12:36.392912 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c353bbb3-cbc5-425a-81ee-2e4aafa31b02 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 17:12:36.395030 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c353bbb3-cbc5-425a-81ee-2e4aafa31b02 admin admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 17:12:36.509885 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 1397/2787] 10.222.0.20 () {60 vars in 1144 bytes} [Mon Aug 1 17:12:36 2022] POST /identity/v3/users => generated 286 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 17:12:36.514878 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-687cd49c-3fd0-4da3-bbc5-eadd0da5c033 admin admin] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 17:12:36.525598 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-687cd49c-3fd0-4da3-bbc5-eadd0da5c033 admin admin] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 17:12:36.530601 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86fe32d9-c92d-49e2-a2b2-b2f36951d03e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 17:12:36.531293 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86fe32d9-c92d-49e2-a2b2-b2f36951d03e admin admin] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 17:12:36.531579 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86fe32d9-c92d-49e2-a2b2-b2f36951d03e admin admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 17:12:36.531853 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86fe32d9-c92d-49e2-a2b2-b2f36951d03e admin admin] PATH_INFO: `/v3/roles` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 17:12:36.532556 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86fe32d9-c92d-49e2-a2b2-b2f36951d03e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 17:12:36.534202 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86fe32d9-c92d-49e2-a2b2-b2f36951d03e admin admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 17:12:36.544923 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 1391/2788] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Aug 1 17:12:36 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 17:12:36.548969 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c353bbb3-cbc5-425a-81ee-2e4aafa31b02 admin admin] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 17:12:36.560111 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c353bbb3-cbc5-425a-81ee-2e4aafa31b02 admin admin] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 17:12:36.565135 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f0cbf1c-7997-4f93-b36f-a305c631d93d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 17:12:36.565752 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0cbf1c-7997-4f93-b36f-a305c631d93d admin admin] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 17:12:36.566056 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0cbf1c-7997-4f93-b36f-a305c631d93d admin admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 17:12:36.566343 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0cbf1c-7997-4f93-b36f-a305c631d93d admin admin] PATH_INFO: `/v3/roles` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 17:12:36.567043 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f0cbf1c-7997-4f93-b36f-a305c631d93d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 17:12:36.568634 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f0cbf1c-7997-4f93-b36f-a305c631d93d admin admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 17:12:36.583802 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 1398/2789] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Aug 1 17:12:36 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 17:12:36.587184 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86fe32d9-c92d-49e2-a2b2-b2f36951d03e admin admin] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 17:12:36.597259 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86fe32d9-c92d-49e2-a2b2-b2f36951d03e admin admin] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 17:12:36.602061 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8135713c-1c7d-4e27-9f8c-a8f942ddbb17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 17:12:36.602772 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8135713c-1c7d-4e27-9f8c-a8f942ddbb17 admin admin] REQUEST_METHOD: `PUT` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 17:12:36.603024 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8135713c-1c7d-4e27-9f8c-a8f942ddbb17 admin admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 17:12:36.603288 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8135713c-1c7d-4e27-9f8c-a8f942ddbb17 admin admin] PATH_INFO: `/v3/projects/1806a53793fd4b64910663eb78db53eb/users/2bd6762abdde4299a229e4b5cfb0a018/roles/0999d3da63a545f18ff3252afdc90518` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 17:12:36.620533 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8135713c-1c7d-4e27-9f8c-a8f942ddbb17 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1806a53793fd4b64910663eb78db53eb, user_id=2bd6762abdde4299a229e4b5cfb0a018, role_id=0999d3da63a545f18ff3252afdc90518)` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 17:12:36.621202 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8135713c-1c7d-4e27-9f8c-a8f942ddbb17 admin admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 17:12:36.681391 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 1392/2790] 10.222.0.20 () {60 vars in 1483 bytes} [Mon Aug 1 17:12:36 2022] PUT /identity/v3/projects/1806a53793fd4b64910663eb78db53eb/users/2bd6762abdde4299a229e4b5cfb0a018/roles/0999d3da63a545f18ff3252afdc90518 => generated 0 bytes in 95 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 17:12:36.746401 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d848efa-77f5-4da5-80b5-874f11d4bbdc None None] REQUEST_METHOD: `POST` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 17:12:36.747280 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d848efa-77f5-4da5-80b5-874f11d4bbdc None None] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 17:12:36.747641 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d848efa-77f5-4da5-80b5-874f11d4bbdc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 17:12:36.775625 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.auth.core [None req-8d848efa-77f5-4da5-80b5-874f11d4bbdc None None] MFA Rules not processed for user `2bd6762abdde4299a229e4b5cfb0a018`. Rule list: `[]` (Enabled: `True`). {{(pid=87058) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 17:12:36.776874 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.fernet_utils [None req-8d848efa-77f5-4da5-80b5-874f11d4bbdc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 17:12:36.789909 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG dbcounter [-] [87058] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 17:12:36.846507 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 1399/2791] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 1 17:12:36 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 17:12:36.851072 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8135713c-1c7d-4e27-9f8c-a8f942ddbb17 admin admin] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 17:12:36.917805 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8135713c-1c7d-4e27-9f8c-a8f942ddbb17 admin admin] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 17:12:36.923200 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5357a523-d53e-4045-9f19-e9821ca4433c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 17:12:36.923839 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5357a523-d53e-4045-9f19-e9821ca4433c admin admin] REQUEST_METHOD: `POST` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 17:12:36.924124 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5357a523-d53e-4045-9f19-e9821ca4433c admin admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 17:12:36.924402 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5357a523-d53e-4045-9f19-e9821ca4433c admin admin] PATH_INFO: `/v3/projects` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 17:12:36.925094 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5357a523-d53e-4045-9f19-e9821ca4433c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 17:12:36.927076 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5357a523-d53e-4045-9f19-e9821ca4433c admin admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 17:12:36.933094 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG dbcounter [-] [87057] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=87057) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 17:12:37.017486 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 1393/2792] 10.222.0.20 () {60 vars in 1153 bytes} [Mon Aug 1 17:12:36 2022] POST /identity/v3/projects => generated 373 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 17:12:37.018894 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d848efa-77f5-4da5-80b5-874f11d4bbdc None None] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 17:12:37.029819 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d848efa-77f5-4da5-80b5-874f11d4bbdc None None] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 17:12:37.089657 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ed3e48-4b6c-4477-aa4e-284a7947d502 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 17:12:37.090321 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ed3e48-4b6c-4477-aa4e-284a7947d502 admin admin] REQUEST_METHOD: `POST` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 17:12:37.090580 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ed3e48-4b6c-4477-aa4e-284a7947d502 admin admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 17:12:37.090854 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ed3e48-4b6c-4477-aa4e-284a7947d502 admin admin] PATH_INFO: `/v3/users` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 17:12:37.091520 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95ed3e48-4b6c-4477-aa4e-284a7947d502 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 17:12:37.093462 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95ed3e48-4b6c-4477-aa4e-284a7947d502 admin admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 17:12:37.144092 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 1400/2793] 10.222.0.20 () {60 vars in 1144 bytes} [Mon Aug 1 17:12:37 2022] POST /identity/v3/users => generated 293 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 17:12:37.148039 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5357a523-d53e-4045-9f19-e9821ca4433c admin admin] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 17:12:37.158912 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5357a523-d53e-4045-9f19-e9821ca4433c admin admin] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 17:12:37.163509 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-679beffb-d241-4419-8bd7-7ebb5c5167ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 17:12:37.164178 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679beffb-d241-4419-8bd7-7ebb5c5167ab admin admin] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 17:12:37.164459 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679beffb-d241-4419-8bd7-7ebb5c5167ab admin admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 17:12:37.164732 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679beffb-d241-4419-8bd7-7ebb5c5167ab admin admin] PATH_INFO: `/v3/roles` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 17:12:37.165508 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-679beffb-d241-4419-8bd7-7ebb5c5167ab admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 17:12:37.167134 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-679beffb-d241-4419-8bd7-7ebb5c5167ab admin admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 17:12:37.177639 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 1394/2794] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Aug 1 17:12:37 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 17:12:37.181724 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ed3e48-4b6c-4477-aa4e-284a7947d502 admin admin] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 17:12:37.192347 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ed3e48-4b6c-4477-aa4e-284a7947d502 admin admin] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 17:12:37.196900 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6077406a-7a9e-431b-8bb3-103b911c8551 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 17:12:37.197547 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6077406a-7a9e-431b-8bb3-103b911c8551 admin admin] REQUEST_METHOD: `PUT` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 17:12:37.197827 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6077406a-7a9e-431b-8bb3-103b911c8551 admin admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 17:12:37.198135 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6077406a-7a9e-431b-8bb3-103b911c8551 admin admin] PATH_INFO: `/v3/projects/03953bbab9204483b056c4c130e19151/users/204c60f58669449fa4bdde23db041fab/roles/440bb943dd5b46fea038b903b6eaaf81` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 17:12:37.213792 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6077406a-7a9e-431b-8bb3-103b911c8551 admin admin] RBAC: Authorizing `identity:create_grant(project_id=03953bbab9204483b056c4c130e19151, user_id=204c60f58669449fa4bdde23db041fab, role_id=440bb943dd5b46fea038b903b6eaaf81)` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 17:12:37.216498 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6077406a-7a9e-431b-8bb3-103b911c8551 admin admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 17:12:37.292582 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 1401/2795] 10.222.0.20 () {60 vars in 1483 bytes} [Mon Aug 1 17:12:37 2022] PUT /identity/v3/projects/03953bbab9204483b056c4c130e19151/users/204c60f58669449fa4bdde23db041fab/roles/440bb943dd5b46fea038b903b6eaaf81 => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 17:12:37.296991 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-679beffb-d241-4419-8bd7-7ebb5c5167ab admin admin] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 17:12:37.307850 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-679beffb-d241-4419-8bd7-7ebb5c5167ab admin admin] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 17:12:37.368720 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b136fb4-c5eb-414b-a12c-8757c830a5b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 17:12:37.369407 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b136fb4-c5eb-414b-a12c-8757c830a5b8 admin admin] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 17:12:37.369625 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b136fb4-c5eb-414b-a12c-8757c830a5b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 17:12:37.369927 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b136fb4-c5eb-414b-a12c-8757c830a5b8 admin admin] PATH_INFO: `/v3/roles` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 17:12:37.370603 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b136fb4-c5eb-414b-a12c-8757c830a5b8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 17:12:37.372792 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b136fb4-c5eb-414b-a12c-8757c830a5b8 admin admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 17:12:37.385775 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 1395/2796] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Aug 1 17:12:37 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 17:12:37.387221 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6077406a-7a9e-431b-8bb3-103b911c8551 admin admin] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 17:12:37.398427 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6077406a-7a9e-431b-8bb3-103b911c8551 admin admin] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 17:12:37.403108 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee935a2-5ef6-498e-a5df-304c793e60df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 17:12:37.405337 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee935a2-5ef6-498e-a5df-304c793e60df admin admin] REQUEST_METHOD: `PUT` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 17:12:37.405337 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee935a2-5ef6-498e-a5df-304c793e60df admin admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 17:12:37.405337 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee935a2-5ef6-498e-a5df-304c793e60df admin admin] PATH_INFO: `/v3/projects/03953bbab9204483b056c4c130e19151/users/204c60f58669449fa4bdde23db041fab/roles/440bb943dd5b46fea038b903b6eaaf81` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 17:12:37.406433 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fee935a2-5ef6-498e-a5df-304c793e60df admin admin] RBAC: Authorizing `identity:create_grant(project_id=03953bbab9204483b056c4c130e19151, user_id=204c60f58669449fa4bdde23db041fab, role_id=440bb943dd5b46fea038b903b6eaaf81)` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 17:12:37.409122 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fee935a2-5ef6-498e-a5df-304c793e60df admin admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 17:12:37.419348 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 1402/2797] 10.222.0.20 () {60 vars in 1483 bytes} [Mon Aug 1 17:12:37 2022] PUT /identity/v3/projects/03953bbab9204483b056c4c130e19151/users/204c60f58669449fa4bdde23db041fab/roles/440bb943dd5b46fea038b903b6eaaf81 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 17:12:37.429228 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60bc1dd-d84c-4410-aae8-a9e9c2a0b8be None None] REQUEST_METHOD: `POST` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 17:12:37.434715 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60bc1dd-d84c-4410-aae8-a9e9c2a0b8be None None] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 17:12:37.434715 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60bc1dd-d84c-4410-aae8-a9e9c2a0b8be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 17:12:37.458936 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.auth.core [None req-b60bc1dd-d84c-4410-aae8-a9e9c2a0b8be None None] MFA Rules not processed for user `204c60f58669449fa4bdde23db041fab`. Rule list: `[]` (Enabled: `True`). {{(pid=87057) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 17:12:37.463061 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.fernet_utils [None req-b60bc1dd-d84c-4410-aae8-a9e9c2a0b8be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87057) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 17:12:37.532800 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 1396/2798] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 1 17:12:37 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 17:12:37.544485 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee935a2-5ef6-498e-a5df-304c793e60df admin admin] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 17:12:37.548984 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.fernet_utils [None req-fee935a2-5ef6-498e-a5df-304c793e60df admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 17:12:37.606265 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee935a2-5ef6-498e-a5df-304c793e60df admin admin] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 17:12:37.610265 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-088b82dd-a3ac-40b9-9f51-968660b1a2fd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25970b52ca15448fa1abb1f974d64038', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b992d6a65df4bf9b27c13192d57c5f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 17:12:37.610881 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088b82dd-a3ac-40b9-9f51-968660b1a2fd service cinder] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 17:12:37.611157 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088b82dd-a3ac-40b9-9f51-968660b1a2fd service cinder] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 17:12:37.611425 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088b82dd-a3ac-40b9-9f51-968660b1a2fd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 17:12:37.620756 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-088b82dd-a3ac-40b9-9f51-968660b1a2fd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 17:12:37.622462 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-088b82dd-a3ac-40b9-9f51-968660b1a2fd service cinder] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 17:12:37.633882 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 1403/2799] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 1 17:12:37 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 17:12:37.866256 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b60bc1dd-d84c-4410-aae8-a9e9c2a0b8be None None] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 17:12:37.877230 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b60bc1dd-d84c-4410-aae8-a9e9c2a0b8be None None] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 17:12:37.881084 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2803054e-58a8-49cc-9cbd-d8c21ac481b7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25970b52ca15448fa1abb1f974d64038', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b992d6a65df4bf9b27c13192d57c5f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 17:12:37.881708 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2803054e-58a8-49cc-9cbd-d8c21ac481b7 service cinder] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 17:12:37.881987 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2803054e-58a8-49cc-9cbd-d8c21ac481b7 service cinder] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 17:12:37.882751 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2803054e-58a8-49cc-9cbd-d8c21ac481b7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 17:12:37.945077 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG dbcounter [-] [87057] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87057) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 17:12:37.948588 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2803054e-58a8-49cc-9cbd-d8c21ac481b7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 17:12:37.951302 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2803054e-58a8-49cc-9cbd-d8c21ac481b7 service cinder] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 17:12:37.961549 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 1397/2800] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 1 17:12:37 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 17:12:47.628855 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG dbcounter [-] [87058] Writing DB stats keystone:SELECT=90,keystone:INSERT=5 {{(pid=87058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 17:12:48.092314 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG dbcounter [-] [87057] Writing DB stats keystone:SELECT=2 {{(pid=87057) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 17:13:14.704104 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-088b82dd-a3ac-40b9-9f51-968660b1a2fd service cinder] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 17:13:14.705693 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.fernet_utils [None req-088b82dd-a3ac-40b9-9f51-968660b1a2fd service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 17:13:14.851888 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-088b82dd-a3ac-40b9-9f51-968660b1a2fd service cinder] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 17:13:14.857047 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe1eb04-89c6-4305-b845-8d642d72d27a service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8362293261614623bcdabc4120a97a9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b992d6a65df4bf9b27c13192d57c5f0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 17:13:14.857924 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe1eb04-89c6-4305-b845-8d642d72d27a service placement] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 17:13:14.858235 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe1eb04-89c6-4305-b845-8d642d72d27a service placement] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 17:13:14.858505 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe1eb04-89c6-4305-b845-8d642d72d27a service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 17:13:14.860922 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.fernet_utils [None req-afe1eb04-89c6-4305-b845-8d642d72d27a service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 17:13:14.872519 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afe1eb04-89c6-4305-b845-8d642d72d27a service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 17:13:14.874159 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afe1eb04-89c6-4305-b845-8d642d72d27a service placement] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 17:13:14.884295 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 1404/2801] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 1 17:13:14 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 181 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 17:13:24.879936 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG dbcounter [-] [87058] Writing DB stats keystone:SELECT=63 {{(pid=87058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 17:13:28.746231 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2803054e-58a8-49cc-9cbd-d8c21ac481b7 service cinder] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 17:13:28.842306 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2803054e-58a8-49cc-9cbd-d8c21ac481b7 service cinder] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 17:13:28.847624 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d4ceb25-f442-4fc6-96ab-d7f760862357 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 17:13:28.848265 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4ceb25-f442-4fc6-96ab-d7f760862357 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 17:13:28.848545 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4ceb25-f442-4fc6-96ab-d7f760862357 admin admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 17:13:28.848823 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4ceb25-f442-4fc6-96ab-d7f760862357 admin admin] PATH_INFO: `/v3/users/2bd6762abdde4299a229e4b5cfb0a018` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 17:13:28.849999 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d4ceb25-f442-4fc6-96ab-d7f760862357 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2bd6762abdde4299a229e4b5cfb0a018)` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 17:13:28.851965 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d4ceb25-f442-4fc6-96ab-d7f760862357 admin admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 17:13:28.979679 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.notifications [None req-1d4ceb25-f442-4fc6-96ab-d7f760862357 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2bd6762abdde4299a229e4b5cfb0a018'} {{(pid=87057) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 17:13:28.980259 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.notifications [None req-1d4ceb25-f442-4fc6-96ab-d7f760862357 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2bd6762abdde4299a229e4b5cfb0a018'} {{(pid=87057) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 17:13:29.027957 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.notifications [None req-1d4ceb25-f442-4fc6-96ab-d7f760862357 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2bd6762abdde4299a229e4b5cfb0a018'} {{(pid=87057) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 17:13:29.028576 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.notifications [None req-1d4ceb25-f442-4fc6-96ab-d7f760862357 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2bd6762abdde4299a229e4b5cfb0a018'} {{(pid=87057) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 17:13:29.036472 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.notifications [None req-1d4ceb25-f442-4fc6-96ab-d7f760862357 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2bd6762abdde4299a229e4b5cfb0a018'} {{(pid=87057) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 17:13:29.055983 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 1398/2802] 10.222.0.20 () {58 vars in 1224 bytes} [Mon Aug 1 17:13:28 2022] DELETE /identity/v3/users/2bd6762abdde4299a229e4b5cfb0a018 => generated 0 bytes in 310 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 17:13:29.060470 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289a7f99-7a37-477d-b6e2-223cafb98ff3 None None] REQUEST_METHOD: `POST` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 17:13:29.060937 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289a7f99-7a37-477d-b6e2-223cafb98ff3 None None] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 17:13:29.061102 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289a7f99-7a37-477d-b6e2-223cafb98ff3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 17:13:29.093223 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.auth.core [None req-289a7f99-7a37-477d-b6e2-223cafb98ff3 None None] MFA Rules not processed for user `1007f8e379b74e169b98fb7c3c0ba5b4`. Rule list: `[]` (Enabled: `True`). {{(pid=87058) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 17:13:29.095198 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.fernet_utils [None req-289a7f99-7a37-477d-b6e2-223cafb98ff3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 17:13:29.154803 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 1405/2803] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 1 17:13:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 17:13:29.165308 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d4ceb25-f442-4fc6-96ab-d7f760862357 admin admin] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 17:13:29.166937 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.fernet_utils [None req-1d4ceb25-f442-4fc6-96ab-d7f760862357 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87057) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 17:13:29.214731 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG dbcounter [-] [87057] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87057) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 17:13:29.245846 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d4ceb25-f442-4fc6-96ab-d7f760862357 admin admin] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 17:13:29.249556 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7251a154-cf7a-4a67-8179-0cb555928844 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d9bd8d649154c979be3744bfd016f95', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b992d6a65df4bf9b27c13192d57c5f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 17:13:29.255217 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7251a154-cf7a-4a67-8179-0cb555928844 service neutron] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 17:13:29.255217 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7251a154-cf7a-4a67-8179-0cb555928844 service neutron] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 17:13:29.255217 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7251a154-cf7a-4a67-8179-0cb555928844 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 17:13:29.261346 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7251a154-cf7a-4a67-8179-0cb555928844 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 17:13:29.262794 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7251a154-cf7a-4a67-8179-0cb555928844 service neutron] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 17:13:29.274624 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 1399/2804] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 1 17:13:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 17:13:29.635424 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-289a7f99-7a37-477d-b6e2-223cafb98ff3 None None] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 17:13:29.636864 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.fernet_utils [None req-289a7f99-7a37-477d-b6e2-223cafb98ff3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 17:13:29.649306 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-289a7f99-7a37-477d-b6e2-223cafb98ff3 None None] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 17:13:29.654067 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d6de7c-7dd3-4bdc-aa87-8a03c029ce2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 17:13:29.654682 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d6de7c-7dd3-4bdc-aa87-8a03c029ce2e admin admin] REQUEST_METHOD: `DELETE` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 17:13:29.654965 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d6de7c-7dd3-4bdc-aa87-8a03c029ce2e admin admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 17:13:29.655239 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d6de7c-7dd3-4bdc-aa87-8a03c029ce2e admin admin] PATH_INFO: `/v3/projects/1806a53793fd4b64910663eb78db53eb` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 17:13:29.656474 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1d6de7c-7dd3-4bdc-aa87-8a03c029ce2e admin admin] RBAC: Authorizing `identity:delete_project(project_id=1806a53793fd4b64910663eb78db53eb)` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 17:13:29.659209 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1d6de7c-7dd3-4bdc-aa87-8a03c029ce2e admin admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 17:13:29.753693 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.notifications [None req-e1d6de7c-7dd3-4bdc-aa87-8a03c029ce2e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1806a53793fd4b64910663eb78db53eb', 'request_id': 'req-e1d6de7c-7dd3-4bdc-aa87-8a03c029ce2e'} {{(pid=87058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 17:13:29.759931 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.notifications [None req-e1d6de7c-7dd3-4bdc-aa87-8a03c029ce2e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1806a53793fd4b64910663eb78db53eb', 'request_id': 'req-e1d6de7c-7dd3-4bdc-aa87-8a03c029ce2e'} {{(pid=87058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 17:13:29.766098 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.notifications [None req-e1d6de7c-7dd3-4bdc-aa87-8a03c029ce2e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1806a53793fd4b64910663eb78db53eb', 'request_id': 'req-e1d6de7c-7dd3-4bdc-aa87-8a03c029ce2e'} {{(pid=87058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 17:13:29.815342 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.notifications [None req-e1d6de7c-7dd3-4bdc-aa87-8a03c029ce2e admin admin] The token cache is being invalidate because project 1806a53793fd4b64910663eb78db53eb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87058) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 17:13:29.815629 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.notifications [None req-e1d6de7c-7dd3-4bdc-aa87-8a03c029ce2e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 17:13:29.816250 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.notifications [None req-e1d6de7c-7dd3-4bdc-aa87-8a03c029ce2e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 17:13:29.817782 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 1406/2805] 10.222.0.20 () {58 vars in 1233 bytes} [Mon Aug 1 17:13:29 2022] DELETE /identity/v3/projects/1806a53793fd4b64910663eb78db53eb => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 17:13:29.821774 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7251a154-cf7a-4a67-8179-0cb555928844 service neutron] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 17:13:29.832102 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7251a154-cf7a-4a67-8179-0cb555928844 service neutron] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 17:13:29.888428 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c72c057-6f77-413c-9bd5-8737c26661d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 17:13:29.889072 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c72c057-6f77-413c-9bd5-8737c26661d9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 17:13:29.889356 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c72c057-6f77-413c-9bd5-8737c26661d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 17:13:29.889636 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c72c057-6f77-413c-9bd5-8737c26661d9 admin admin] PATH_INFO: `/v3/users/204c60f58669449fa4bdde23db041fab` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 17:13:29.890846 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c72c057-6f77-413c-9bd5-8737c26661d9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=204c60f58669449fa4bdde23db041fab)` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 17:13:29.893501 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c72c057-6f77-413c-9bd5-8737c26661d9 admin admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 17:13:30.052700 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.notifications [None req-9c72c057-6f77-413c-9bd5-8737c26661d9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '204c60f58669449fa4bdde23db041fab'} {{(pid=87057) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 17:13:30.053302 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.notifications [None req-9c72c057-6f77-413c-9bd5-8737c26661d9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '204c60f58669449fa4bdde23db041fab'} {{(pid=87057) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 17:13:30.119161 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.notifications [None req-9c72c057-6f77-413c-9bd5-8737c26661d9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '204c60f58669449fa4bdde23db041fab'} {{(pid=87057) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 17:13:30.119785 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.notifications [None req-9c72c057-6f77-413c-9bd5-8737c26661d9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '204c60f58669449fa4bdde23db041fab'} {{(pid=87057) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 17:13:30.127106 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.notifications [None req-9c72c057-6f77-413c-9bd5-8737c26661d9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '204c60f58669449fa4bdde23db041fab'} {{(pid=87057) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 17:13:30.144876 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 1400/2806] 10.222.0.20 () {58 vars in 1224 bytes} [Mon Aug 1 17:13:29 2022] DELETE /identity/v3/users/204c60f58669449fa4bdde23db041fab => generated 0 bytes in 324 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 17:13:30.556120 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d6de7c-7dd3-4bdc-aa87-8a03c029ce2e admin admin] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 17:13:30.557654 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.fernet_utils [None req-e1d6de7c-7dd3-4bdc-aa87-8a03c029ce2e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 17:13:30.571299 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d6de7c-7dd3-4bdc-aa87-8a03c029ce2e admin admin] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 17:13:30.630726 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b35017-85b4-4b96-b14c-8e70cb15e15a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 17:13:30.631369 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b35017-85b4-4b96-b14c-8e70cb15e15a admin admin] REQUEST_METHOD: `DELETE` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 17:13:30.631651 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b35017-85b4-4b96-b14c-8e70cb15e15a admin admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 17:13:30.631924 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b35017-85b4-4b96-b14c-8e70cb15e15a admin admin] PATH_INFO: `/v3/projects/03953bbab9204483b056c4c130e19151` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 17:13:30.633073 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4b35017-85b4-4b96-b14c-8e70cb15e15a admin admin] RBAC: Authorizing `identity:delete_project(project_id=03953bbab9204483b056c4c130e19151)` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 17:13:30.635085 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4b35017-85b4-4b96-b14c-8e70cb15e15a admin admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 17:13:30.651183 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG dbcounter [-] [87058] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=87058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 17:13:30.748875 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.notifications [None req-f4b35017-85b4-4b96-b14c-8e70cb15e15a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '03953bbab9204483b056c4c130e19151', 'request_id': 'req-f4b35017-85b4-4b96-b14c-8e70cb15e15a'} {{(pid=87058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 17:13:30.755413 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.notifications [None req-f4b35017-85b4-4b96-b14c-8e70cb15e15a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '03953bbab9204483b056c4c130e19151', 'request_id': 'req-f4b35017-85b4-4b96-b14c-8e70cb15e15a'} {{(pid=87058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 17:13:30.760982 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.notifications [None req-f4b35017-85b4-4b96-b14c-8e70cb15e15a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '03953bbab9204483b056c4c130e19151', 'request_id': 'req-f4b35017-85b4-4b96-b14c-8e70cb15e15a'} {{(pid=87058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 17:13:30.827505 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.notifications [None req-f4b35017-85b4-4b96-b14c-8e70cb15e15a admin admin] The token cache is being invalidate because project 03953bbab9204483b056c4c130e19151 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87058) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 17:13:30.827722 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.notifications [None req-f4b35017-85b4-4b96-b14c-8e70cb15e15a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 17:13:30.828349 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.notifications [None req-f4b35017-85b4-4b96-b14c-8e70cb15e15a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 17:13:30.830637 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 1407/2807] 10.222.0.20 () {58 vars in 1233 bytes} [Mon Aug 1 17:13:30 2022] DELETE /identity/v3/projects/03953bbab9204483b056c4c130e19151 => generated 0 bytes in 275 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 17:13:30.835758 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ca2584-e51a-4dd1-85f5-c803da30dccf None None] REQUEST_METHOD: `POST` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 17:13:30.836556 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ca2584-e51a-4dd1-85f5-c803da30dccf None None] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 17:13:30.836556 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ca2584-e51a-4dd1-85f5-c803da30dccf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 17:13:30.856552 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.auth.core [None req-a6ca2584-e51a-4dd1-85f5-c803da30dccf None None] MFA Rules not processed for user `1007f8e379b74e169b98fb7c3c0ba5b4`. Rule list: `[]` (Enabled: `True`). {{(pid=87057) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 17:13:30.858125 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.fernet_utils [None req-a6ca2584-e51a-4dd1-85f5-c803da30dccf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87057) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 17:13:30.900564 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG dbcounter [-] [87057] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87057) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 17:13:30.919058 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 1401/2808] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 1 17:13:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 17:13:30.927164 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17854e88-b2f3-4877-b7f9-91802ba1e31f None None] REQUEST_METHOD: `POST` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 17:13:30.927449 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17854e88-b2f3-4877-b7f9-91802ba1e31f None None] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 17:13:30.927729 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17854e88-b2f3-4877-b7f9-91802ba1e31f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 17:13:30.947461 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.auth.core [None req-17854e88-b2f3-4877-b7f9-91802ba1e31f None None] MFA Rules not processed for user `1007f8e379b74e169b98fb7c3c0ba5b4`. Rule list: `[]` (Enabled: `True`). {{(pid=87058) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 17:13:30.948717 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.fernet_utils [None req-17854e88-b2f3-4877-b7f9-91802ba1e31f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 17:13:30.957531 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 1408/2809] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 1 17:13:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 17:13:30.961697 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ca2584-e51a-4dd1-85f5-c803da30dccf None None] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 17:13:30.972807 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ca2584-e51a-4dd1-85f5-c803da30dccf None None] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 17:13:30.977335 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a67d3f-c630-4d6c-87c2-f1c308d8184d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 17:13:30.979795 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a67d3f-c630-4d6c-87c2-f1c308d8184d admin admin] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 17:13:30.979795 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a67d3f-c630-4d6c-87c2-f1c308d8184d admin admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 17:13:30.979795 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a67d3f-c630-4d6c-87c2-f1c308d8184d admin admin] PATH_INFO: `/v3/domains` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 17:13:30.979795 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20a67d3f-c630-4d6c-87c2-f1c308d8184d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87057) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 17:13:30.979795 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20a67d3f-c630-4d6c-87c2-f1c308d8184d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 17:13:30.981013 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20a67d3f-c630-4d6c-87c2-f1c308d8184d admin admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 17:13:30.992704 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 1402/2810] 10.222.0.20 () {58 vars in 1153 bytes} [Mon Aug 1 17:13:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 17:13:30.996965 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17854e88-b2f3-4877-b7f9-91802ba1e31f None None] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 17:13:31.007644 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17854e88-b2f3-4877-b7f9-91802ba1e31f None None] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 17:13:31.012317 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69151362-db5d-46eb-9b48-838306ba85d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 17:13:31.012951 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69151362-db5d-46eb-9b48-838306ba85d2 admin admin] REQUEST_METHOD: `POST` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 17:13:31.013324 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69151362-db5d-46eb-9b48-838306ba85d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 17:13:31.013595 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69151362-db5d-46eb-9b48-838306ba85d2 admin admin] PATH_INFO: `/v3/projects` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 17:13:31.014306 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69151362-db5d-46eb-9b48-838306ba85d2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 17:13:31.017722 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69151362-db5d-46eb-9b48-838306ba85d2 admin admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 17:13:31.091896 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 1409/2811] 10.222.0.20 () {60 vars in 1153 bytes} [Mon Aug 1 17:13:30 2022] POST /identity/v3/projects => generated 387 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 17:13:31.096479 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a67d3f-c630-4d6c-87c2-f1c308d8184d admin admin] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 17:13:31.106569 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a67d3f-c630-4d6c-87c2-f1c308d8184d admin admin] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 17:13:31.163319 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e80f51d7-c668-4255-b7ff-dbc1d13e5315 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 17:13:31.163935 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80f51d7-c668-4255-b7ff-dbc1d13e5315 admin admin] REQUEST_METHOD: `POST` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 17:13:31.164227 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80f51d7-c668-4255-b7ff-dbc1d13e5315 admin admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 17:13:31.164491 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80f51d7-c668-4255-b7ff-dbc1d13e5315 admin admin] PATH_INFO: `/v3/users` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 17:13:31.165149 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e80f51d7-c668-4255-b7ff-dbc1d13e5315 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 17:13:31.167102 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e80f51d7-c668-4255-b7ff-dbc1d13e5315 admin admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 17:13:31.245354 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 1403/2812] 10.222.0.20 () {60 vars in 1144 bytes} [Mon Aug 1 17:13:31 2022] POST /identity/v3/users => generated 294 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 17:13:31.249754 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69151362-db5d-46eb-9b48-838306ba85d2 admin admin] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 17:13:31.260944 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69151362-db5d-46eb-9b48-838306ba85d2 admin admin] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 17:13:31.265587 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6953640-0e39-4f3b-8146-608a7f7ac948 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 17:13:31.266902 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6953640-0e39-4f3b-8146-608a7f7ac948 admin admin] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 17:13:31.267183 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6953640-0e39-4f3b-8146-608a7f7ac948 admin admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 17:13:31.267448 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6953640-0e39-4f3b-8146-608a7f7ac948 admin admin] PATH_INFO: `/v3/roles` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 17:13:31.268129 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6953640-0e39-4f3b-8146-608a7f7ac948 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 17:13:31.269704 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6953640-0e39-4f3b-8146-608a7f7ac948 admin admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 17:13:31.279740 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 1410/2813] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Aug 1 17:13:31 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 17:13:31.284302 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e80f51d7-c668-4255-b7ff-dbc1d13e5315 admin admin] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 17:13:31.294526 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e80f51d7-c668-4255-b7ff-dbc1d13e5315 admin admin] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 17:13:31.299768 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1777732b-d5dd-4dd3-8ba5-96786c40169d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 17:13:31.300371 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1777732b-d5dd-4dd3-8ba5-96786c40169d admin admin] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 17:13:31.300652 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1777732b-d5dd-4dd3-8ba5-96786c40169d admin admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 17:13:31.300967 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1777732b-d5dd-4dd3-8ba5-96786c40169d admin admin] PATH_INFO: `/v3/roles` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 17:13:31.301652 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1777732b-d5dd-4dd3-8ba5-96786c40169d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 17:13:31.303288 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1777732b-d5dd-4dd3-8ba5-96786c40169d admin admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 17:13:31.313901 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 1404/2814] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Aug 1 17:13:31 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 01 17:13:31.318008 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6953640-0e39-4f3b-8146-608a7f7ac948 admin admin] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 17:13:31.327951 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6953640-0e39-4f3b-8146-608a7f7ac948 admin admin] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 17:13:31.332658 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-916f27ef-5c76-4423-9558-2e311687c8d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 17:13:31.333300 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916f27ef-5c76-4423-9558-2e311687c8d9 admin admin] REQUEST_METHOD: `PUT` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 17:13:31.333574 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916f27ef-5c76-4423-9558-2e311687c8d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 17:13:31.333844 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916f27ef-5c76-4423-9558-2e311687c8d9 admin admin] PATH_INFO: `/v3/projects/37ef46e9270d47468174eff141671e95/users/a4e694fc94244463a1f1a990bca9cd36/roles/0999d3da63a545f18ff3252afdc90518` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 17:13:31.348983 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-916f27ef-5c76-4423-9558-2e311687c8d9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=37ef46e9270d47468174eff141671e95, user_id=a4e694fc94244463a1f1a990bca9cd36, role_id=0999d3da63a545f18ff3252afdc90518)` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 17:13:31.351579 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-916f27ef-5c76-4423-9558-2e311687c8d9 admin admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 17:13:31.416279 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 1411/2815] 10.222.0.20 () {60 vars in 1483 bytes} [Mon Aug 1 17:13:31 2022] PUT /identity/v3/projects/37ef46e9270d47468174eff141671e95/users/a4e694fc94244463a1f1a990bca9cd36/roles/0999d3da63a545f18ff3252afdc90518 => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 17:13:31.428054 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac786750-f37b-4c15-bf4d-37a9a78b12e4 None None] REQUEST_METHOD: `POST` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 17:13:31.428345 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac786750-f37b-4c15-bf4d-37a9a78b12e4 None None] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 17:13:31.428624 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac786750-f37b-4c15-bf4d-37a9a78b12e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 17:13:31.456467 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.auth.core [None req-ac786750-f37b-4c15-bf4d-37a9a78b12e4 None None] MFA Rules not processed for user `a4e694fc94244463a1f1a990bca9cd36`. Rule list: `[]` (Enabled: `True`). {{(pid=87057) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 17:13:31.457695 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.fernet_utils [None req-ac786750-f37b-4c15-bf4d-37a9a78b12e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87057) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 17:13:31.527300 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 1405/2816] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 1 17:13:31 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 17:13:31.533659 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-916f27ef-5c76-4423-9558-2e311687c8d9 admin admin] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 17:13:31.598805 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-916f27ef-5c76-4423-9558-2e311687c8d9 admin admin] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 17:13:31.603727 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4896c65a-a667-4778-beaa-20df24ef423b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 17:13:31.604277 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4896c65a-a667-4778-beaa-20df24ef423b admin admin] REQUEST_METHOD: `POST` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 17:13:31.604613 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4896c65a-a667-4778-beaa-20df24ef423b admin admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 17:13:31.604889 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4896c65a-a667-4778-beaa-20df24ef423b admin admin] PATH_INFO: `/v3/projects` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 17:13:31.605576 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4896c65a-a667-4778-beaa-20df24ef423b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 17:13:31.607804 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4896c65a-a667-4778-beaa-20df24ef423b admin admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 17:13:31.731725 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 1412/2817] 10.222.0.20 () {60 vars in 1153 bytes} [Mon Aug 1 17:13:31 2022] POST /identity/v3/projects => generated 385 bytes in 199 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 17:13:31.736719 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac786750-f37b-4c15-bf4d-37a9a78b12e4 None None] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 17:13:31.747478 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac786750-f37b-4c15-bf4d-37a9a78b12e4 None None] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 17:13:31.787544 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG dbcounter [-] [87057] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87057) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 17:13:31.804364 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4694baf0-454a-4f7a-a344-498e314fc490 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 17:13:31.804973 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4694baf0-454a-4f7a-a344-498e314fc490 admin admin] REQUEST_METHOD: `POST` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 17:13:31.805252 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4694baf0-454a-4f7a-a344-498e314fc490 admin admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 17:13:31.805523 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4694baf0-454a-4f7a-a344-498e314fc490 admin admin] PATH_INFO: `/v3/users` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 17:13:31.806326 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4694baf0-454a-4f7a-a344-498e314fc490 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 17:13:31.808367 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4694baf0-454a-4f7a-a344-498e314fc490 admin admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 17:13:31.884999 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 1406/2818] 10.222.0.20 () {60 vars in 1144 bytes} [Mon Aug 1 17:13:31 2022] POST /identity/v3/users => generated 293 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 17:13:31.888944 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4896c65a-a667-4778-beaa-20df24ef423b admin admin] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 17:13:31.899817 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4896c65a-a667-4778-beaa-20df24ef423b admin admin] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 17:13:31.906205 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d79eeb-d733-4fd9-ad8d-16ee0b3752a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 17:13:31.907402 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d79eeb-d733-4fd9-ad8d-16ee0b3752a9 admin admin] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 17:13:31.907697 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d79eeb-d733-4fd9-ad8d-16ee0b3752a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 17:13:31.908155 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d79eeb-d733-4fd9-ad8d-16ee0b3752a9 admin admin] PATH_INFO: `/v3/roles` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 17:13:31.908846 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46d79eeb-d733-4fd9-ad8d-16ee0b3752a9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 17:13:31.910511 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46d79eeb-d733-4fd9-ad8d-16ee0b3752a9 admin admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 17:13:31.921301 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 1413/2819] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Aug 1 17:13:31 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 17:13:31.926048 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4694baf0-454a-4f7a-a344-498e314fc490 admin admin] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 17:13:31.936137 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4694baf0-454a-4f7a-a344-498e314fc490 admin admin] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 17:13:31.940712 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ad732f-5047-46c7-af60-a9e3fc235452 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 17:13:31.941315 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ad732f-5047-46c7-af60-a9e3fc235452 admin admin] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 17:13:31.941592 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ad732f-5047-46c7-af60-a9e3fc235452 admin admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 17:13:31.941864 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ad732f-5047-46c7-af60-a9e3fc235452 admin admin] PATH_INFO: `/v3/roles` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 17:13:31.942706 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01ad732f-5047-46c7-af60-a9e3fc235452 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 17:13:31.944294 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01ad732f-5047-46c7-af60-a9e3fc235452 admin admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 17:13:31.953946 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 1407/2820] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Aug 1 17:13:31 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 17:13:31.957987 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d79eeb-d733-4fd9-ad8d-16ee0b3752a9 admin admin] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 17:13:31.967934 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d79eeb-d733-4fd9-ad8d-16ee0b3752a9 admin admin] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 17:13:31.973330 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5acd61ad-66d2-4b93-8920-6f5a5dcc550a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 17:13:31.973977 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5acd61ad-66d2-4b93-8920-6f5a5dcc550a admin admin] REQUEST_METHOD: `PUT` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 17:13:31.974288 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5acd61ad-66d2-4b93-8920-6f5a5dcc550a admin admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 17:13:31.974563 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5acd61ad-66d2-4b93-8920-6f5a5dcc550a admin admin] PATH_INFO: `/v3/projects/412df787ea954a91a2fac56fba4f96f5/users/92a3c9cb4a09486d80e54547ab45d1d6/roles/0999d3da63a545f18ff3252afdc90518` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 17:13:31.986564 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG dbcounter [-] [87058] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=87058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 17:13:31.990488 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5acd61ad-66d2-4b93-8920-6f5a5dcc550a admin admin] RBAC: Authorizing `identity:create_grant(project_id=412df787ea954a91a2fac56fba4f96f5, user_id=92a3c9cb4a09486d80e54547ab45d1d6, role_id=0999d3da63a545f18ff3252afdc90518)` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 17:13:31.993222 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5acd61ad-66d2-4b93-8920-6f5a5dcc550a admin admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 17:13:32.067864 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 1414/2821] 10.222.0.20 () {60 vars in 1483 bytes} [Mon Aug 1 17:13:31 2022] PUT /identity/v3/projects/412df787ea954a91a2fac56fba4f96f5/users/92a3c9cb4a09486d80e54547ab45d1d6/roles/0999d3da63a545f18ff3252afdc90518 => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 17:13:32.078241 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898ef8e0-9014-4bc6-8f22-35a1514b0b4a None None] REQUEST_METHOD: `POST` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 17:13:32.078551 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898ef8e0-9014-4bc6-8f22-35a1514b0b4a None None] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 17:13:32.078964 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898ef8e0-9014-4bc6-8f22-35a1514b0b4a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 17:13:32.106597 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.auth.core [None req-898ef8e0-9014-4bc6-8f22-35a1514b0b4a None None] MFA Rules not processed for user `92a3c9cb4a09486d80e54547ab45d1d6`. Rule list: `[]` (Enabled: `True`). {{(pid=87057) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 17:13:32.108055 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.fernet_utils [None req-898ef8e0-9014-4bc6-8f22-35a1514b0b4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87057) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 17:13:32.171579 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 1408/2822] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 1 17:13:32 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 17:13:32.176204 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5acd61ad-66d2-4b93-8920-6f5a5dcc550a admin admin] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 17:13:32.238882 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5acd61ad-66d2-4b93-8920-6f5a5dcc550a admin admin] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 17:13:32.243574 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99af6618-22a8-4e02-879b-1a3e11629ad6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 17:13:32.248617 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99af6618-22a8-4e02-879b-1a3e11629ad6 admin admin] REQUEST_METHOD: `POST` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 17:13:32.248617 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99af6618-22a8-4e02-879b-1a3e11629ad6 admin admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 17:13:32.248617 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99af6618-22a8-4e02-879b-1a3e11629ad6 admin admin] PATH_INFO: `/v3/projects` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 17:13:32.248617 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99af6618-22a8-4e02-879b-1a3e11629ad6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 17:13:32.248617 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99af6618-22a8-4e02-879b-1a3e11629ad6 admin admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 17:13:32.343906 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 1415/2823] 10.222.0.20 () {60 vars in 1153 bytes} [Mon Aug 1 17:13:32 2022] POST /identity/v3/projects => generated 385 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 17:13:32.348677 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-898ef8e0-9014-4bc6-8f22-35a1514b0b4a None None] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 17:13:32.359678 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-898ef8e0-9014-4bc6-8f22-35a1514b0b4a None None] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 17:13:32.420577 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c4db57b-5ce1-4989-a43b-4c154c0df95e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 17:13:32.421191 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4db57b-5ce1-4989-a43b-4c154c0df95e admin admin] REQUEST_METHOD: `POST` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 17:13:32.421471 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4db57b-5ce1-4989-a43b-4c154c0df95e admin admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 17:13:32.421800 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4db57b-5ce1-4989-a43b-4c154c0df95e admin admin] PATH_INFO: `/v3/users` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 17:13:32.422473 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c4db57b-5ce1-4989-a43b-4c154c0df95e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 17:13:32.424571 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c4db57b-5ce1-4989-a43b-4c154c0df95e admin admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 17:13:32.525042 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 1409/2824] 10.222.0.20 () {60 vars in 1144 bytes} [Mon Aug 1 17:13:32 2022] POST /identity/v3/users => generated 299 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 17:13:32.529067 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99af6618-22a8-4e02-879b-1a3e11629ad6 admin admin] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 17:13:32.541216 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99af6618-22a8-4e02-879b-1a3e11629ad6 admin admin] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 17:13:32.546095 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0365347-4868-4361-a7f1-5f97e5096242 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 17:13:32.546765 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0365347-4868-4361-a7f1-5f97e5096242 admin admin] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 17:13:32.547048 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0365347-4868-4361-a7f1-5f97e5096242 admin admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 17:13:32.547418 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0365347-4868-4361-a7f1-5f97e5096242 admin admin] PATH_INFO: `/v3/roles` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 17:13:32.548715 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0365347-4868-4361-a7f1-5f97e5096242 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 17:13:32.550333 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0365347-4868-4361-a7f1-5f97e5096242 admin admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 17:13:32.560851 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 1416/2825] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Aug 1 17:13:32 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 17:13:32.565015 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c4db57b-5ce1-4989-a43b-4c154c0df95e admin admin] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 17:13:32.575066 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c4db57b-5ce1-4989-a43b-4c154c0df95e admin admin] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 17:13:32.580236 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e803722-bcd2-4953-8721-ba2ada6305b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 17:13:32.580899 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e803722-bcd2-4953-8721-ba2ada6305b8 admin admin] REQUEST_METHOD: `PUT` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 17:13:32.581180 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e803722-bcd2-4953-8721-ba2ada6305b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 17:13:32.581456 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e803722-bcd2-4953-8721-ba2ada6305b8 admin admin] PATH_INFO: `/v3/projects/030e9df1bc4f4827bd9a2518dae9ed5e/users/c136e1be49d1481ea3ac11a16044d81d/roles/440bb943dd5b46fea038b903b6eaaf81` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 17:13:32.596667 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e803722-bcd2-4953-8721-ba2ada6305b8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=030e9df1bc4f4827bd9a2518dae9ed5e, user_id=c136e1be49d1481ea3ac11a16044d81d, role_id=440bb943dd5b46fea038b903b6eaaf81)` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 17:13:32.599404 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e803722-bcd2-4953-8721-ba2ada6305b8 admin admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 17:13:32.667578 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 1410/2826] 10.222.0.20 () {60 vars in 1483 bytes} [Mon Aug 1 17:13:32 2022] PUT /identity/v3/projects/030e9df1bc4f4827bd9a2518dae9ed5e/users/c136e1be49d1481ea3ac11a16044d81d/roles/440bb943dd5b46fea038b903b6eaaf81 => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 17:13:32.671645 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0365347-4868-4361-a7f1-5f97e5096242 admin admin] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 17:13:32.682784 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0365347-4868-4361-a7f1-5f97e5096242 admin admin] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 17:13:32.739710 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f88a9384-f6d3-4ed9-b251-2dbbd6324cf7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 17:13:32.740322 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88a9384-f6d3-4ed9-b251-2dbbd6324cf7 admin admin] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 17:13:32.740599 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88a9384-f6d3-4ed9-b251-2dbbd6324cf7 admin admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 17:13:32.740877 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88a9384-f6d3-4ed9-b251-2dbbd6324cf7 admin admin] PATH_INFO: `/v3/roles` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 17:13:32.741559 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f88a9384-f6d3-4ed9-b251-2dbbd6324cf7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 17:13:32.743173 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f88a9384-f6d3-4ed9-b251-2dbbd6324cf7 admin admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 17:13:32.752854 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 1417/2827] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Aug 1 17:13:32 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 17:13:32.756967 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e803722-bcd2-4953-8721-ba2ada6305b8 admin admin] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 17:13:32.767271 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e803722-bcd2-4953-8721-ba2ada6305b8 admin admin] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 17:13:32.771876 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f21b05af-a5db-44bd-804c-4dad39e7b9c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 17:13:32.772553 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21b05af-a5db-44bd-804c-4dad39e7b9c0 admin admin] REQUEST_METHOD: `PUT` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 17:13:32.772832 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21b05af-a5db-44bd-804c-4dad39e7b9c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 17:13:32.773104 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21b05af-a5db-44bd-804c-4dad39e7b9c0 admin admin] PATH_INFO: `/v3/projects/030e9df1bc4f4827bd9a2518dae9ed5e/users/c136e1be49d1481ea3ac11a16044d81d/roles/440bb943dd5b46fea038b903b6eaaf81` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 17:13:32.775087 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f21b05af-a5db-44bd-804c-4dad39e7b9c0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=030e9df1bc4f4827bd9a2518dae9ed5e, user_id=c136e1be49d1481ea3ac11a16044d81d, role_id=440bb943dd5b46fea038b903b6eaaf81)` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 17:13:32.777731 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f21b05af-a5db-44bd-804c-4dad39e7b9c0 admin admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 17:13:32.788267 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 1411/2828] 10.222.0.20 () {60 vars in 1483 bytes} [Mon Aug 1 17:13:32 2022] PUT /identity/v3/projects/030e9df1bc4f4827bd9a2518dae9ed5e/users/c136e1be49d1481ea3ac11a16044d81d/roles/440bb943dd5b46fea038b903b6eaaf81 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 17:13:32.798660 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ff8dfa-c36e-462e-8372-7e49bebf0dfd None None] REQUEST_METHOD: `POST` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 17:13:32.798996 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ff8dfa-c36e-462e-8372-7e49bebf0dfd None None] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 17:13:32.799210 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ff8dfa-c36e-462e-8372-7e49bebf0dfd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 17:13:32.827242 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.auth.core [None req-02ff8dfa-c36e-462e-8372-7e49bebf0dfd None None] MFA Rules not processed for user `c136e1be49d1481ea3ac11a16044d81d`. Rule list: `[]` (Enabled: `True`). {{(pid=87058) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 17:13:32.828436 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.fernet_utils [None req-02ff8dfa-c36e-462e-8372-7e49bebf0dfd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 17:13:32.873780 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG dbcounter [-] [87058] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=87058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 17:13:32.895347 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 1418/2829] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 1 17:13:32 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 17:13:32.906059 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f21b05af-a5db-44bd-804c-4dad39e7b9c0 admin admin] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 17:13:32.908581 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.fernet_utils [None req-f21b05af-a5db-44bd-804c-4dad39e7b9c0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87057) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 17:13:32.914019 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG dbcounter [-] [87057] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=87057) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 17:13:32.966580 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f21b05af-a5db-44bd-804c-4dad39e7b9c0 admin admin] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 17:13:32.971002 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc2211c8-3bb1-4f87-a488-e9ce41eb930e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25970b52ca15448fa1abb1f974d64038', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b992d6a65df4bf9b27c13192d57c5f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 17:13:32.971623 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2211c8-3bb1-4f87-a488-e9ce41eb930e service cinder] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 17:13:32.971912 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2211c8-3bb1-4f87-a488-e9ce41eb930e service cinder] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 17:13:32.972183 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2211c8-3bb1-4f87-a488-e9ce41eb930e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 17:13:32.981243 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc2211c8-3bb1-4f87-a488-e9ce41eb930e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 17:13:32.982988 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc2211c8-3bb1-4f87-a488-e9ce41eb930e service cinder] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 17:13:32.992928 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 1412/2830] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 1 17:13:32 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 17:13:33.997214 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ff8dfa-c36e-462e-8372-7e49bebf0dfd None None] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 17:13:33.998654 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.fernet_utils [None req-02ff8dfa-c36e-462e-8372-7e49bebf0dfd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 17:13:34.011125 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ff8dfa-c36e-462e-8372-7e49bebf0dfd None None] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 17:13:34.014620 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a70c551-e37a-4040-983b-73dc237e7956 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25970b52ca15448fa1abb1f974d64038', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b992d6a65df4bf9b27c13192d57c5f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 17:13:34.015228 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a70c551-e37a-4040-983b-73dc237e7956 service cinder] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 17:13:34.015508 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a70c551-e37a-4040-983b-73dc237e7956 service cinder] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 17:13:34.015784 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a70c551-e37a-4040-983b-73dc237e7956 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 17:13:34.071714 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a70c551-e37a-4040-983b-73dc237e7956 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 17:13:34.073149 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a70c551-e37a-4040-983b-73dc237e7956 service cinder] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 17:13:34.082605 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 1419/2831] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 1 17:13:33 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 17:13:38.831564 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc2211c8-3bb1-4f87-a488-e9ce41eb930e service cinder] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 17:13:38.840388 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc2211c8-3bb1-4f87-a488-e9ce41eb930e service cinder] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 17:13:38.844639 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2520b0d2-860d-4747-9344-232f964f4d6c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25970b52ca15448fa1abb1f974d64038', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b992d6a65df4bf9b27c13192d57c5f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 17:13:38.845299 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2520b0d2-860d-4747-9344-232f964f4d6c service cinder] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 17:13:38.845579 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2520b0d2-860d-4747-9344-232f964f4d6c service cinder] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 17:13:38.845846 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2520b0d2-860d-4747-9344-232f964f4d6c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 17:13:38.902854 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2520b0d2-860d-4747-9344-232f964f4d6c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 17:13:38.904302 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2520b0d2-860d-4747-9344-232f964f4d6c service cinder] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 17:13:38.913999 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 1413/2832] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 1 17:13:38 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 17:13:42.643998 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a70c551-e37a-4040-983b-73dc237e7956 service cinder] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 17:13:42.705395 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a70c551-e37a-4040-983b-73dc237e7956 service cinder] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 17:13:42.709917 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c51b17-fbe1-4d0e-b5cb-4fb6623a0491 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 17:13:42.710578 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c51b17-fbe1-4d0e-b5cb-4fb6623a0491 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 17:13:42.710859 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c51b17-fbe1-4d0e-b5cb-4fb6623a0491 admin admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 17:13:42.711136 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c51b17-fbe1-4d0e-b5cb-4fb6623a0491 admin admin] PATH_INFO: `/v3/users/a4e694fc94244463a1f1a990bca9cd36` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 17:13:42.712290 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78c51b17-fbe1-4d0e-b5cb-4fb6623a0491 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a4e694fc94244463a1f1a990bca9cd36)` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 17:13:42.714239 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78c51b17-fbe1-4d0e-b5cb-4fb6623a0491 admin admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 17:13:42.881355 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.notifications [None req-78c51b17-fbe1-4d0e-b5cb-4fb6623a0491 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a4e694fc94244463a1f1a990bca9cd36'} {{(pid=87058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 17:13:42.881940 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.notifications [None req-78c51b17-fbe1-4d0e-b5cb-4fb6623a0491 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a4e694fc94244463a1f1a990bca9cd36'} {{(pid=87058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 17:13:42.882582 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.notifications [None req-78c51b17-fbe1-4d0e-b5cb-4fb6623a0491 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a4e694fc94244463a1f1a990bca9cd36'} {{(pid=87058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 17:13:42.894491 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.notifications [None req-78c51b17-fbe1-4d0e-b5cb-4fb6623a0491 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a4e694fc94244463a1f1a990bca9cd36'} {{(pid=87058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 17:13:42.901405 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.notifications [None req-78c51b17-fbe1-4d0e-b5cb-4fb6623a0491 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a4e694fc94244463a1f1a990bca9cd36'} {{(pid=87058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 17:13:42.970704 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 1420/2833] 10.222.0.20 () {58 vars in 1224 bytes} [Mon Aug 1 17:13:42 2022] DELETE /identity/v3/users/a4e694fc94244463a1f1a990bca9cd36 => generated 0 bytes in 327 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 17:13:42.975193 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96c0ce1-5683-4eec-8523-c1dd966cbe1d None None] REQUEST_METHOD: `POST` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 17:13:42.975704 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96c0ce1-5683-4eec-8523-c1dd966cbe1d None None] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 17:13:42.975816 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96c0ce1-5683-4eec-8523-c1dd966cbe1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 17:13:42.995749 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.auth.core [None req-d96c0ce1-5683-4eec-8523-c1dd966cbe1d None None] MFA Rules not processed for user `1007f8e379b74e169b98fb7c3c0ba5b4`. Rule list: `[]` (Enabled: `True`). {{(pid=87057) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 17:13:42.996960 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.fernet_utils [None req-d96c0ce1-5683-4eec-8523-c1dd966cbe1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87057) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 17:13:43.055256 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 1414/2834] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 1 17:13:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 17:13:43.065043 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c51b17-fbe1-4d0e-b5cb-4fb6623a0491 admin admin] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 17:13:43.066478 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.fernet_utils [None req-78c51b17-fbe1-4d0e-b5cb-4fb6623a0491 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 17:13:43.071925 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG dbcounter [-] [87058] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 17:13:43.125093 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c51b17-fbe1-4d0e-b5cb-4fb6623a0491 admin admin] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 17:13:43.128986 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a0a041-c071-4200-a6dc-fb8c75663027 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d9bd8d649154c979be3744bfd016f95', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b992d6a65df4bf9b27c13192d57c5f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 17:13:43.129860 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a0a041-c071-4200-a6dc-fb8c75663027 service neutron] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 17:13:43.130351 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a0a041-c071-4200-a6dc-fb8c75663027 service neutron] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 17:13:43.130997 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a0a041-c071-4200-a6dc-fb8c75663027 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 17:13:43.140518 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73a0a041-c071-4200-a6dc-fb8c75663027 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 17:13:43.141743 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73a0a041-c071-4200-a6dc-fb8c75663027 service neutron] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 17:13:43.152520 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 1421/2835] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 1 17:13:43 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 17:13:43.543574 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d96c0ce1-5683-4eec-8523-c1dd966cbe1d None None] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 17:13:43.545044 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.fernet_utils [None req-d96c0ce1-5683-4eec-8523-c1dd966cbe1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87057) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 17:13:43.560070 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d96c0ce1-5683-4eec-8523-c1dd966cbe1d None None] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 17:13:43.565529 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5928f244-4f3c-434e-8aad-4780fc526222 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 17:13:43.566213 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5928f244-4f3c-434e-8aad-4780fc526222 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 17:13:43.566491 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5928f244-4f3c-434e-8aad-4780fc526222 admin admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 17:13:43.566788 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5928f244-4f3c-434e-8aad-4780fc526222 admin admin] PATH_INFO: `/v3/projects/37ef46e9270d47468174eff141671e95` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 17:13:43.567955 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5928f244-4f3c-434e-8aad-4780fc526222 admin admin] RBAC: Authorizing `identity:delete_project(project_id=37ef46e9270d47468174eff141671e95)` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 17:13:43.569866 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5928f244-4f3c-434e-8aad-4780fc526222 admin admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 17:13:43.655410 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG dbcounter [-] [87057] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=87057) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 17:13:43.662374 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.notifications [None req-5928f244-4f3c-434e-8aad-4780fc526222 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '37ef46e9270d47468174eff141671e95', 'request_id': 'req-5928f244-4f3c-434e-8aad-4780fc526222'} {{(pid=87057) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 17:13:43.668430 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.notifications [None req-5928f244-4f3c-434e-8aad-4780fc526222 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '37ef46e9270d47468174eff141671e95', 'request_id': 'req-5928f244-4f3c-434e-8aad-4780fc526222'} {{(pid=87057) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 17:13:43.674527 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.notifications [None req-5928f244-4f3c-434e-8aad-4780fc526222 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '37ef46e9270d47468174eff141671e95', 'request_id': 'req-5928f244-4f3c-434e-8aad-4780fc526222'} {{(pid=87057) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 17:13:43.752650 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.notifications [None req-5928f244-4f3c-434e-8aad-4780fc526222 admin admin] The token cache is being invalidate because project 37ef46e9270d47468174eff141671e95 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87057) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 17:13:43.752937 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.notifications [None req-5928f244-4f3c-434e-8aad-4780fc526222 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87057) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 17:13:43.753563 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.notifications [None req-5928f244-4f3c-434e-8aad-4780fc526222 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87057) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 17:13:43.755151 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 1415/2836] 10.222.0.20 () {58 vars in 1233 bytes} [Mon Aug 1 17:13:43 2022] DELETE /identity/v3/projects/37ef46e9270d47468174eff141671e95 => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 17:13:43.759114 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a0a041-c071-4200-a6dc-fb8c75663027 service neutron] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 17:13:43.768906 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a0a041-c071-4200-a6dc-fb8c75663027 service neutron] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 17:13:43.824955 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c45970c-e5c1-421f-8206-2893de9e8795 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 17:13:43.825594 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c45970c-e5c1-421f-8206-2893de9e8795 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 17:13:43.825877 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c45970c-e5c1-421f-8206-2893de9e8795 admin admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 17:13:43.826191 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c45970c-e5c1-421f-8206-2893de9e8795 admin admin] PATH_INFO: `/v3/users/92a3c9cb4a09486d80e54547ab45d1d6` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 17:13:43.827349 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c45970c-e5c1-421f-8206-2893de9e8795 admin admin] RBAC: Authorizing `identity:delete_user(user_id=92a3c9cb4a09486d80e54547ab45d1d6)` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 17:13:43.829433 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c45970c-e5c1-421f-8206-2893de9e8795 admin admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 17:13:43.979207 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.notifications [None req-5c45970c-e5c1-421f-8206-2893de9e8795 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '92a3c9cb4a09486d80e54547ab45d1d6'} {{(pid=87058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 17:13:43.979825 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.notifications [None req-5c45970c-e5c1-421f-8206-2893de9e8795 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '92a3c9cb4a09486d80e54547ab45d1d6'} {{(pid=87058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 17:13:43.980449 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.notifications [None req-5c45970c-e5c1-421f-8206-2893de9e8795 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '92a3c9cb4a09486d80e54547ab45d1d6'} {{(pid=87058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 17:13:43.993416 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.notifications [None req-5c45970c-e5c1-421f-8206-2893de9e8795 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '92a3c9cb4a09486d80e54547ab45d1d6'} {{(pid=87058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 17:13:44.000603 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.notifications [None req-5c45970c-e5c1-421f-8206-2893de9e8795 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '92a3c9cb4a09486d80e54547ab45d1d6'} {{(pid=87058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 17:13:44.068357 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 1422/2837] 10.222.0.20 () {58 vars in 1224 bytes} [Mon Aug 1 17:13:43 2022] DELETE /identity/v3/users/92a3c9cb4a09486d80e54547ab45d1d6 => generated 0 bytes in 310 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 17:13:44.515632 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5928f244-4f3c-434e-8aad-4780fc526222 admin admin] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 17:13:44.517535 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.fernet_utils [None req-5928f244-4f3c-434e-8aad-4780fc526222 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87057) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 17:13:44.538736 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5928f244-4f3c-434e-8aad-4780fc526222 admin admin] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 17:13:44.608806 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb3f3ec-90de-4c82-a7ae-5061a7b3125e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 17:13:44.609486 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb3f3ec-90de-4c82-a7ae-5061a7b3125e admin admin] REQUEST_METHOD: `DELETE` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 17:13:44.610413 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb3f3ec-90de-4c82-a7ae-5061a7b3125e admin admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 17:13:44.610688 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb3f3ec-90de-4c82-a7ae-5061a7b3125e admin admin] PATH_INFO: `/v3/projects/412df787ea954a91a2fac56fba4f96f5` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 17:13:44.612263 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fb3f3ec-90de-4c82-a7ae-5061a7b3125e admin admin] RBAC: Authorizing `identity:delete_project(project_id=412df787ea954a91a2fac56fba4f96f5)` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 17:13:44.614220 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fb3f3ec-90de-4c82-a7ae-5061a7b3125e admin admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 17:13:44.724586 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.notifications [None req-4fb3f3ec-90de-4c82-a7ae-5061a7b3125e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '412df787ea954a91a2fac56fba4f96f5', 'request_id': 'req-4fb3f3ec-90de-4c82-a7ae-5061a7b3125e'} {{(pid=87057) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 17:13:44.731457 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.notifications [None req-4fb3f3ec-90de-4c82-a7ae-5061a7b3125e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '412df787ea954a91a2fac56fba4f96f5', 'request_id': 'req-4fb3f3ec-90de-4c82-a7ae-5061a7b3125e'} {{(pid=87057) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 17:13:44.738281 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.notifications [None req-4fb3f3ec-90de-4c82-a7ae-5061a7b3125e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '412df787ea954a91a2fac56fba4f96f5', 'request_id': 'req-4fb3f3ec-90de-4c82-a7ae-5061a7b3125e'} {{(pid=87057) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 17:13:44.799474 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.notifications [None req-4fb3f3ec-90de-4c82-a7ae-5061a7b3125e admin admin] The token cache is being invalidate because project 412df787ea954a91a2fac56fba4f96f5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87057) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 17:13:44.799755 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.notifications [None req-4fb3f3ec-90de-4c82-a7ae-5061a7b3125e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87057) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 17:13:44.800380 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.notifications [None req-4fb3f3ec-90de-4c82-a7ae-5061a7b3125e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87057) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 17:13:44.801959 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 1416/2838] 10.222.0.20 () {58 vars in 1233 bytes} [Mon Aug 1 17:13:44 2022] DELETE /identity/v3/projects/412df787ea954a91a2fac56fba4f96f5 => generated 0 bytes in 287 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 17:13:44.805899 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c45970c-e5c1-421f-8206-2893de9e8795 admin admin] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 17:13:44.807322 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.fernet_utils [None req-5c45970c-e5c1-421f-8206-2893de9e8795 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 17:13:44.820473 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c45970c-e5c1-421f-8206-2893de9e8795 admin admin] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 17:13:44.840263 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG dbcounter [-] [87058] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 17:13:44.878241 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-171726a1-c3d8-4223-aee2-2c8cf84f0d03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 17:13:44.878893 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171726a1-c3d8-4223-aee2-2c8cf84f0d03 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 17:13:44.879176 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171726a1-c3d8-4223-aee2-2c8cf84f0d03 admin admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 17:13:44.879454 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171726a1-c3d8-4223-aee2-2c8cf84f0d03 admin admin] PATH_INFO: `/v3/users/c136e1be49d1481ea3ac11a16044d81d` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 17:13:44.880631 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-171726a1-c3d8-4223-aee2-2c8cf84f0d03 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c136e1be49d1481ea3ac11a16044d81d)` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 17:13:44.882621 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-171726a1-c3d8-4223-aee2-2c8cf84f0d03 admin admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 17:13:45.036272 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.notifications [None req-171726a1-c3d8-4223-aee2-2c8cf84f0d03 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c136e1be49d1481ea3ac11a16044d81d'} {{(pid=87058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 17:13:45.036940 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.notifications [None req-171726a1-c3d8-4223-aee2-2c8cf84f0d03 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c136e1be49d1481ea3ac11a16044d81d'} {{(pid=87058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 17:13:45.037547 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.notifications [None req-171726a1-c3d8-4223-aee2-2c8cf84f0d03 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c136e1be49d1481ea3ac11a16044d81d'} {{(pid=87058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 17:13:45.049758 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.notifications [None req-171726a1-c3d8-4223-aee2-2c8cf84f0d03 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c136e1be49d1481ea3ac11a16044d81d'} {{(pid=87058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 17:13:45.056685 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.notifications [None req-171726a1-c3d8-4223-aee2-2c8cf84f0d03 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c136e1be49d1481ea3ac11a16044d81d'} {{(pid=87058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 17:13:45.176387 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 1423/2839] 10.222.0.20 () {58 vars in 1224 bytes} [Mon Aug 1 17:13:44 2022] DELETE /identity/v3/users/c136e1be49d1481ea3ac11a16044d81d => generated 0 bytes in 371 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 17:13:45.577764 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb3f3ec-90de-4c82-a7ae-5061a7b3125e admin admin] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 17:13:45.579913 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.fernet_utils [None req-4fb3f3ec-90de-4c82-a7ae-5061a7b3125e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87057) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 17:13:45.595957 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb3f3ec-90de-4c82-a7ae-5061a7b3125e admin admin] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 17:13:45.648129 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG dbcounter [-] [87057] Writing DB stats keystone:SELECT=87,keystone:DELETE=11,keystone:INSERT=2 {{(pid=87057) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 17:13:45.654162 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f59853e7-c9a7-40cd-bd21-e9f1486ff146 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 17:13:45.654805 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59853e7-c9a7-40cd-bd21-e9f1486ff146 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 17:13:45.655080 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59853e7-c9a7-40cd-bd21-e9f1486ff146 admin admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 17:13:45.655350 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59853e7-c9a7-40cd-bd21-e9f1486ff146 admin admin] PATH_INFO: `/v3/projects/030e9df1bc4f4827bd9a2518dae9ed5e` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 17:13:45.656516 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f59853e7-c9a7-40cd-bd21-e9f1486ff146 admin admin] RBAC: Authorizing `identity:delete_project(project_id=030e9df1bc4f4827bd9a2518dae9ed5e)` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 17:13:45.658499 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f59853e7-c9a7-40cd-bd21-e9f1486ff146 admin admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 17:13:45.748850 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.notifications [None req-f59853e7-c9a7-40cd-bd21-e9f1486ff146 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '030e9df1bc4f4827bd9a2518dae9ed5e', 'request_id': 'req-f59853e7-c9a7-40cd-bd21-e9f1486ff146'} {{(pid=87057) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 17:13:45.755756 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.notifications [None req-f59853e7-c9a7-40cd-bd21-e9f1486ff146 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '030e9df1bc4f4827bd9a2518dae9ed5e', 'request_id': 'req-f59853e7-c9a7-40cd-bd21-e9f1486ff146'} {{(pid=87057) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 17:13:45.761341 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.notifications [None req-f59853e7-c9a7-40cd-bd21-e9f1486ff146 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '030e9df1bc4f4827bd9a2518dae9ed5e', 'request_id': 'req-f59853e7-c9a7-40cd-bd21-e9f1486ff146'} {{(pid=87057) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 17:13:45.832204 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.notifications [None req-f59853e7-c9a7-40cd-bd21-e9f1486ff146 admin admin] The token cache is being invalidate because project 030e9df1bc4f4827bd9a2518dae9ed5e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87057) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 17:13:45.832485 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.notifications [None req-f59853e7-c9a7-40cd-bd21-e9f1486ff146 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87057) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 17:13:45.833093 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.notifications [None req-f59853e7-c9a7-40cd-bd21-e9f1486ff146 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87057) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 17:13:45.834631 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 1417/2840] 10.222.0.20 () {58 vars in 1233 bytes} [Mon Aug 1 17:13:45 2022] DELETE /identity/v3/projects/030e9df1bc4f4827bd9a2518dae9ed5e => generated 0 bytes in 257 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 17:13:45.840538 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ee1aab-2335-4853-b480-77c5d7d2bcaf None None] REQUEST_METHOD: `POST` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 17:13:45.841039 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ee1aab-2335-4853-b480-77c5d7d2bcaf None None] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 17:13:45.841142 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ee1aab-2335-4853-b480-77c5d7d2bcaf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 17:13:45.861358 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.auth.core [None req-46ee1aab-2335-4853-b480-77c5d7d2bcaf None None] MFA Rules not processed for user `1007f8e379b74e169b98fb7c3c0ba5b4`. Rule list: `[]` (Enabled: `True`). {{(pid=87058) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 17:13:45.862527 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.fernet_utils [None req-46ee1aab-2335-4853-b480-77c5d7d2bcaf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 17:13:45.921725 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 1424/2841] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 1 17:13:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 17:13:45.932162 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1df83e3-4a5a-4b6c-8e68-f90c8dc500b8 None None] REQUEST_METHOD: `POST` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 17:13:45.932450 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1df83e3-4a5a-4b6c-8e68-f90c8dc500b8 None None] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 17:13:45.932729 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1df83e3-4a5a-4b6c-8e68-f90c8dc500b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 17:13:45.952870 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.auth.core [None req-c1df83e3-4a5a-4b6c-8e68-f90c8dc500b8 None None] MFA Rules not processed for user `1007f8e379b74e169b98fb7c3c0ba5b4`. Rule list: `[]` (Enabled: `True`). {{(pid=87057) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 17:13:45.954137 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.fernet_utils [None req-c1df83e3-4a5a-4b6c-8e68-f90c8dc500b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87057) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 17:13:45.961976 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 1418/2842] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 1 17:13:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 17:13:45.966190 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46ee1aab-2335-4853-b480-77c5d7d2bcaf None None] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 17:13:45.976606 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46ee1aab-2335-4853-b480-77c5d7d2bcaf None None] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 17:13:45.981278 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7a01c46-a458-43f2-bb60-c7492b4228f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 17:13:45.982184 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a01c46-a458-43f2-bb60-c7492b4228f3 admin admin] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 17:13:45.982464 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a01c46-a458-43f2-bb60-c7492b4228f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 17:13:45.982734 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a01c46-a458-43f2-bb60-c7492b4228f3 admin admin] PATH_INFO: `/v3/domains` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 17:13:45.983431 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7a01c46-a458-43f2-bb60-c7492b4228f3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87058) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 17:13:45.983731 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7a01c46-a458-43f2-bb60-c7492b4228f3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 17:13:45.985940 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7a01c46-a458-43f2-bb60-c7492b4228f3 admin admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 17:13:45.997334 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 1425/2843] 10.222.0.20 () {58 vars in 1153 bytes} [Mon Aug 1 17:13:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 17:13:46.002084 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1df83e3-4a5a-4b6c-8e68-f90c8dc500b8 None None] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 17:13:46.011390 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1df83e3-4a5a-4b6c-8e68-f90c8dc500b8 None None] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 17:13:46.016307 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21097f53-aff2-4d38-af61-8364223a2a76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 17:13:46.016822 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21097f53-aff2-4d38-af61-8364223a2a76 admin admin] REQUEST_METHOD: `POST` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 17:13:46.017105 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21097f53-aff2-4d38-af61-8364223a2a76 admin admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 17:13:46.017404 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21097f53-aff2-4d38-af61-8364223a2a76 admin admin] PATH_INFO: `/v3/projects` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 17:13:46.018100 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21097f53-aff2-4d38-af61-8364223a2a76 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 17:13:46.019999 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21097f53-aff2-4d38-af61-8364223a2a76 admin admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 17:13:46.103028 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 1419/2844] 10.222.0.20 () {60 vars in 1153 bytes} [Mon Aug 1 17:13:46 2022] POST /identity/v3/projects => generated 401 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 17:13:46.107456 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7a01c46-a458-43f2-bb60-c7492b4228f3 admin admin] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 17:13:46.117757 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7a01c46-a458-43f2-bb60-c7492b4228f3 admin admin] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 17:13:46.149033 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG dbcounter [-] [87058] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 17:13:46.176277 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a3e1704-e321-4924-9cc3-a32ed2a2dacf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 17:13:46.177266 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3e1704-e321-4924-9cc3-a32ed2a2dacf admin admin] REQUEST_METHOD: `POST` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 17:13:46.177601 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3e1704-e321-4924-9cc3-a32ed2a2dacf admin admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 17:13:46.177872 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3e1704-e321-4924-9cc3-a32ed2a2dacf admin admin] PATH_INFO: `/v3/users` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 17:13:46.178569 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a3e1704-e321-4924-9cc3-a32ed2a2dacf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 17:13:46.180458 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a3e1704-e321-4924-9cc3-a32ed2a2dacf admin admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 17:13:46.290527 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 1426/2845] 10.222.0.20 () {60 vars in 1144 bytes} [Mon Aug 1 17:13:46 2022] POST /identity/v3/users => generated 301 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 17:13:46.294670 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21097f53-aff2-4d38-af61-8364223a2a76 admin admin] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 17:13:46.304929 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21097f53-aff2-4d38-af61-8364223a2a76 admin admin] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 17:13:46.309491 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d4178b1-d5fa-4e40-80a7-0c3f65ab42d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 17:13:46.310119 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4178b1-d5fa-4e40-80a7-0c3f65ab42d3 admin admin] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 17:13:46.310412 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4178b1-d5fa-4e40-80a7-0c3f65ab42d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 17:13:46.310668 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4178b1-d5fa-4e40-80a7-0c3f65ab42d3 admin admin] PATH_INFO: `/v3/roles` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 17:13:46.311356 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d4178b1-d5fa-4e40-80a7-0c3f65ab42d3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 17:13:46.313612 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d4178b1-d5fa-4e40-80a7-0c3f65ab42d3 admin admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 17:13:46.323353 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 1420/2846] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Aug 1 17:13:46 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 17:13:46.328915 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a3e1704-e321-4924-9cc3-a32ed2a2dacf admin admin] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 17:13:46.339116 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a3e1704-e321-4924-9cc3-a32ed2a2dacf admin admin] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 17:13:46.343742 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf933b1-4e41-4b2a-be5e-23ed119cf40e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 17:13:46.344496 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf933b1-4e41-4b2a-be5e-23ed119cf40e admin admin] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 17:13:46.344782 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf933b1-4e41-4b2a-be5e-23ed119cf40e admin admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 17:13:46.345055 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf933b1-4e41-4b2a-be5e-23ed119cf40e admin admin] PATH_INFO: `/v3/roles` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 17:13:46.345743 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdf933b1-4e41-4b2a-be5e-23ed119cf40e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 17:13:46.347424 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdf933b1-4e41-4b2a-be5e-23ed119cf40e admin admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 17:13:46.358120 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 1427/2847] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Aug 1 17:13:46 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 17:13:46.362234 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d4178b1-d5fa-4e40-80a7-0c3f65ab42d3 admin admin] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 17:13:46.372140 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d4178b1-d5fa-4e40-80a7-0c3f65ab42d3 admin admin] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 17:13:46.377376 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7b4773b-ac84-4365-953a-1b09c401f30c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 17:13:46.378054 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b4773b-ac84-4365-953a-1b09c401f30c admin admin] REQUEST_METHOD: `PUT` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 17:13:46.378414 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b4773b-ac84-4365-953a-1b09c401f30c admin admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 17:13:46.378689 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b4773b-ac84-4365-953a-1b09c401f30c admin admin] PATH_INFO: `/v3/projects/d4ec10c3af95431db5ad0a1105c57022/users/078e764d0c3a45099d1d4df1e1d4fa4f/roles/0999d3da63a545f18ff3252afdc90518` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 17:13:46.394017 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7b4773b-ac84-4365-953a-1b09c401f30c admin admin] RBAC: Authorizing `identity:create_grant(project_id=d4ec10c3af95431db5ad0a1105c57022, user_id=078e764d0c3a45099d1d4df1e1d4fa4f, role_id=0999d3da63a545f18ff3252afdc90518)` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 17:13:46.396810 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7b4773b-ac84-4365-953a-1b09c401f30c admin admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 17:13:46.461776 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 1421/2848] 10.222.0.20 () {60 vars in 1483 bytes} [Mon Aug 1 17:13:46 2022] PUT /identity/v3/projects/d4ec10c3af95431db5ad0a1105c57022/users/078e764d0c3a45099d1d4df1e1d4fa4f/roles/0999d3da63a545f18ff3252afdc90518 => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 17:13:46.473878 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a389392-3080-4835-951a-a6e20291688e None None] REQUEST_METHOD: `POST` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 17:13:46.474201 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a389392-3080-4835-951a-a6e20291688e None None] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 17:13:46.474483 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a389392-3080-4835-951a-a6e20291688e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 17:13:46.502418 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.auth.core [None req-9a389392-3080-4835-951a-a6e20291688e None None] MFA Rules not processed for user `078e764d0c3a45099d1d4df1e1d4fa4f`. Rule list: `[]` (Enabled: `True`). {{(pid=87058) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 17:13:46.503657 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.fernet_utils [None req-9a389392-3080-4835-951a-a6e20291688e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 17:13:46.569293 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 1428/2849] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 1 17:13:46 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 17:13:46.573931 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7b4773b-ac84-4365-953a-1b09c401f30c admin admin] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 17:13:46.635896 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7b4773b-ac84-4365-953a-1b09c401f30c admin admin] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 17:13:46.640669 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6813e8df-e6ab-4c4c-9165-8efcb8c4947d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 17:13:46.643564 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6813e8df-e6ab-4c4c-9165-8efcb8c4947d admin admin] REQUEST_METHOD: `POST` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 17:13:46.643564 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6813e8df-e6ab-4c4c-9165-8efcb8c4947d admin admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 17:13:46.643564 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6813e8df-e6ab-4c4c-9165-8efcb8c4947d admin admin] PATH_INFO: `/v3/projects` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 17:13:46.643564 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6813e8df-e6ab-4c4c-9165-8efcb8c4947d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 17:13:46.645173 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6813e8df-e6ab-4c4c-9165-8efcb8c4947d admin admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 17:13:46.732132 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 1422/2850] 10.222.0.20 () {60 vars in 1153 bytes} [Mon Aug 1 17:13:46 2022] POST /identity/v3/projects => generated 401 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 17:13:46.741163 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a389392-3080-4835-951a-a6e20291688e None None] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 17:13:46.752745 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a389392-3080-4835-951a-a6e20291688e None None] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 17:13:46.808178 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b36ba756-41ce-4c61-a37d-a0ed78d04752 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 17:13:46.808795 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36ba756-41ce-4c61-a37d-a0ed78d04752 admin admin] REQUEST_METHOD: `POST` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 17:13:46.809079 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36ba756-41ce-4c61-a37d-a0ed78d04752 admin admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 17:13:46.809356 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36ba756-41ce-4c61-a37d-a0ed78d04752 admin admin] PATH_INFO: `/v3/users` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 17:13:46.810085 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b36ba756-41ce-4c61-a37d-a0ed78d04752 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 17:13:46.812058 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b36ba756-41ce-4c61-a37d-a0ed78d04752 admin admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 17:13:46.896379 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 1429/2851] 10.222.0.20 () {60 vars in 1144 bytes} [Mon Aug 1 17:13:46 2022] POST /identity/v3/users => generated 307 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 17:13:46.900449 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6813e8df-e6ab-4c4c-9165-8efcb8c4947d admin admin] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 17:13:46.911377 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6813e8df-e6ab-4c4c-9165-8efcb8c4947d admin admin] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 17:13:46.916163 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b59ace-2814-47b3-813a-1af1576d9669 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 17:13:46.916830 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b59ace-2814-47b3-813a-1af1576d9669 admin admin] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 17:13:46.917106 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b59ace-2814-47b3-813a-1af1576d9669 admin admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 17:13:46.917379 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b59ace-2814-47b3-813a-1af1576d9669 admin admin] PATH_INFO: `/v3/roles` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 17:13:46.918114 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70b59ace-2814-47b3-813a-1af1576d9669 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 17:13:46.919722 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70b59ace-2814-47b3-813a-1af1576d9669 admin admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 17:13:46.921888 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG dbcounter [-] [87057] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=87057) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 17:13:46.932057 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 1423/2852] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Aug 1 17:13:46 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 17:13:46.936283 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b36ba756-41ce-4c61-a37d-a0ed78d04752 admin admin] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 17:13:46.947013 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b36ba756-41ce-4c61-a37d-a0ed78d04752 admin admin] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 17:13:46.951630 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5953e886-8c17-458d-b04b-a115d86a1e6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 17:13:46.952281 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5953e886-8c17-458d-b04b-a115d86a1e6e admin admin] REQUEST_METHOD: `PUT` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 17:13:46.952563 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5953e886-8c17-458d-b04b-a115d86a1e6e admin admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 17:13:46.952841 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5953e886-8c17-458d-b04b-a115d86a1e6e admin admin] PATH_INFO: `/v3/projects/dc0e9b8ed5124029b7f8e863cbe0dc77/users/c6cd9f88ce7944d4838bc5ad049cf925/roles/440bb943dd5b46fea038b903b6eaaf81` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 17:13:46.968090 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5953e886-8c17-458d-b04b-a115d86a1e6e admin admin] RBAC: Authorizing `identity:create_grant(project_id=dc0e9b8ed5124029b7f8e863cbe0dc77, user_id=c6cd9f88ce7944d4838bc5ad049cf925, role_id=440bb943dd5b46fea038b903b6eaaf81)` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 17:13:46.970815 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5953e886-8c17-458d-b04b-a115d86a1e6e admin admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 17:13:47.051197 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 1430/2853] 10.222.0.20 () {60 vars in 1483 bytes} [Mon Aug 1 17:13:46 2022] PUT /identity/v3/projects/dc0e9b8ed5124029b7f8e863cbe0dc77/users/c6cd9f88ce7944d4838bc5ad049cf925/roles/440bb943dd5b46fea038b903b6eaaf81 => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 17:13:47.055180 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b59ace-2814-47b3-813a-1af1576d9669 admin admin] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 17:13:47.065162 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b59ace-2814-47b3-813a-1af1576d9669 admin admin] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 17:13:47.121377 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f28c5a43-961b-48bd-94b5-56e3ee30a07e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 17:13:47.121985 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f28c5a43-961b-48bd-94b5-56e3ee30a07e admin admin] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 17:13:47.122289 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f28c5a43-961b-48bd-94b5-56e3ee30a07e admin admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 17:13:47.122560 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f28c5a43-961b-48bd-94b5-56e3ee30a07e admin admin] PATH_INFO: `/v3/roles` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 17:13:47.123284 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f28c5a43-961b-48bd-94b5-56e3ee30a07e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 17:13:47.124870 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f28c5a43-961b-48bd-94b5-56e3ee30a07e admin admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 17:13:47.134399 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 1424/2854] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Aug 1 17:13:47 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 17:13:47.138509 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5953e886-8c17-458d-b04b-a115d86a1e6e admin admin] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 17:13:47.145508 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG dbcounter [-] [87058] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=87058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 17:13:47.149327 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5953e886-8c17-458d-b04b-a115d86a1e6e admin admin] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 17:13:47.153863 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f27d395a-736c-4a07-a23e-1da93c34b475 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 17:13:47.157977 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27d395a-736c-4a07-a23e-1da93c34b475 admin admin] REQUEST_METHOD: `PUT` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 17:13:47.157977 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27d395a-736c-4a07-a23e-1da93c34b475 admin admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 17:13:47.157977 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27d395a-736c-4a07-a23e-1da93c34b475 admin admin] PATH_INFO: `/v3/projects/dc0e9b8ed5124029b7f8e863cbe0dc77/users/c6cd9f88ce7944d4838bc5ad049cf925/roles/440bb943dd5b46fea038b903b6eaaf81` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 17:13:47.157977 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f27d395a-736c-4a07-a23e-1da93c34b475 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dc0e9b8ed5124029b7f8e863cbe0dc77, user_id=c6cd9f88ce7944d4838bc5ad049cf925, role_id=440bb943dd5b46fea038b903b6eaaf81)` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 17:13:47.160204 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f27d395a-736c-4a07-a23e-1da93c34b475 admin admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 17:13:47.170428 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 1431/2855] 10.222.0.20 () {60 vars in 1483 bytes} [Mon Aug 1 17:13:47 2022] PUT /identity/v3/projects/dc0e9b8ed5124029b7f8e863cbe0dc77/users/c6cd9f88ce7944d4838bc5ad049cf925/roles/440bb943dd5b46fea038b903b6eaaf81 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 17:13:47.181900 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6feadb59-1c1f-4be2-ba82-73d21d66b859 None None] REQUEST_METHOD: `POST` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 17:13:47.186866 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6feadb59-1c1f-4be2-ba82-73d21d66b859 None None] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 17:13:47.186866 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6feadb59-1c1f-4be2-ba82-73d21d66b859 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 17:13:47.210667 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.auth.core [None req-6feadb59-1c1f-4be2-ba82-73d21d66b859 None None] MFA Rules not processed for user `c6cd9f88ce7944d4838bc5ad049cf925`. Rule list: `[]` (Enabled: `True`). {{(pid=87057) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 17:13:47.216269 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.fernet_utils [None req-6feadb59-1c1f-4be2-ba82-73d21d66b859 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87057) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 17:13:47.278740 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 1425/2856] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 1 17:13:47 2022] POST /identity/v3/auth/tokens => generated 3335 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 17:13:47.289694 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f27d395a-736c-4a07-a23e-1da93c34b475 admin admin] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 17:13:47.291407 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.fernet_utils [None req-f27d395a-736c-4a07-a23e-1da93c34b475 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 17:13:47.349662 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f27d395a-736c-4a07-a23e-1da93c34b475 admin admin] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 17:13:47.353400 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de7bf8bf-a299-44fa-85e5-a829b23389c4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25970b52ca15448fa1abb1f974d64038', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b992d6a65df4bf9b27c13192d57c5f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 17:13:47.354047 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7bf8bf-a299-44fa-85e5-a829b23389c4 service cinder] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 17:13:47.354335 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7bf8bf-a299-44fa-85e5-a829b23389c4 service cinder] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 17:13:47.354602 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7bf8bf-a299-44fa-85e5-a829b23389c4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 17:13:47.364404 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de7bf8bf-a299-44fa-85e5-a829b23389c4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 17:13:47.365814 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de7bf8bf-a299-44fa-85e5-a829b23389c4 service cinder] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 17:13:47.375816 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 1432/2857] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 1 17:13:47 2022] GET /identity/v3/auth/tokens => generated 3335 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 17:13:47.579312 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6feadb59-1c1f-4be2-ba82-73d21d66b859 None None] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 17:13:47.588917 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6feadb59-1c1f-4be2-ba82-73d21d66b859 None None] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 17:13:47.592472 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc890514-3893-4493-b6ca-ad1357e5b259 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25970b52ca15448fa1abb1f974d64038', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b992d6a65df4bf9b27c13192d57c5f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 17:13:47.596516 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc890514-3893-4493-b6ca-ad1357e5b259 service cinder] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 17:13:47.596516 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc890514-3893-4493-b6ca-ad1357e5b259 service cinder] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 17:13:47.596516 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc890514-3893-4493-b6ca-ad1357e5b259 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 17:13:47.650288 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc890514-3893-4493-b6ca-ad1357e5b259 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 17:13:47.651737 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc890514-3893-4493-b6ca-ad1357e5b259 service cinder] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 17:13:47.661584 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 1426/2858] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 1 17:13:47 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 17:13:52.510244 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de7bf8bf-a299-44fa-85e5-a829b23389c4 service cinder] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 17:13:52.571864 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de7bf8bf-a299-44fa-85e5-a829b23389c4 service cinder] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 17:13:52.576379 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7efc177d-deb2-4610-ae3d-2ec32f32bc75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 17:13:52.577018 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7efc177d-deb2-4610-ae3d-2ec32f32bc75 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 17:13:52.577299 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7efc177d-deb2-4610-ae3d-2ec32f32bc75 admin admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 17:13:52.577573 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7efc177d-deb2-4610-ae3d-2ec32f32bc75 admin admin] PATH_INFO: `/v3/users/078e764d0c3a45099d1d4df1e1d4fa4f` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 17:13:52.578753 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7efc177d-deb2-4610-ae3d-2ec32f32bc75 admin admin] RBAC: Authorizing `identity:delete_user(user_id=078e764d0c3a45099d1d4df1e1d4fa4f)` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 17:13:52.580816 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7efc177d-deb2-4610-ae3d-2ec32f32bc75 admin admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 17:13:52.778729 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.notifications [None req-7efc177d-deb2-4610-ae3d-2ec32f32bc75 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '078e764d0c3a45099d1d4df1e1d4fa4f'} {{(pid=87058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 17:13:52.779409 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.notifications [None req-7efc177d-deb2-4610-ae3d-2ec32f32bc75 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '078e764d0c3a45099d1d4df1e1d4fa4f'} {{(pid=87058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 17:13:52.780067 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.notifications [None req-7efc177d-deb2-4610-ae3d-2ec32f32bc75 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '078e764d0c3a45099d1d4df1e1d4fa4f'} {{(pid=87058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 17:13:52.792704 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.notifications [None req-7efc177d-deb2-4610-ae3d-2ec32f32bc75 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '078e764d0c3a45099d1d4df1e1d4fa4f'} {{(pid=87058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 17:13:52.799454 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.notifications [None req-7efc177d-deb2-4610-ae3d-2ec32f32bc75 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '078e764d0c3a45099d1d4df1e1d4fa4f'} {{(pid=87058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 17:13:52.878432 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 1433/2859] 10.222.0.20 () {58 vars in 1224 bytes} [Mon Aug 1 17:13:52 2022] DELETE /identity/v3/users/078e764d0c3a45099d1d4df1e1d4fa4f => generated 0 bytes in 369 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 17:13:52.883140 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635c7ea2-8333-46a0-8994-e4c551df4432 None None] REQUEST_METHOD: `POST` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 17:13:52.883631 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635c7ea2-8333-46a0-8994-e4c551df4432 None None] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 17:13:52.883750 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635c7ea2-8333-46a0-8994-e4c551df4432 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 17:13:52.903447 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.auth.core [None req-635c7ea2-8333-46a0-8994-e4c551df4432 None None] MFA Rules not processed for user `1007f8e379b74e169b98fb7c3c0ba5b4`. Rule list: `[]` (Enabled: `True`). {{(pid=87057) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 17:13:52.904664 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.fernet_utils [None req-635c7ea2-8333-46a0-8994-e4c551df4432 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87057) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 17:13:52.917359 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG dbcounter [-] [87057] Writing DB stats keystone:SELECT=100 {{(pid=87057) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 17:13:52.964245 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 1427/2860] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 1 17:13:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 17:13:52.974812 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7efc177d-deb2-4610-ae3d-2ec32f32bc75 admin admin] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 17:13:52.976248 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.fernet_utils [None req-7efc177d-deb2-4610-ae3d-2ec32f32bc75 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 17:13:52.990584 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG dbcounter [-] [87058] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=87058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 17:13:53.033053 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7efc177d-deb2-4610-ae3d-2ec32f32bc75 admin admin] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 17:13:53.037236 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b88a07-1407-4fec-88f2-f80334263e3f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d9bd8d649154c979be3744bfd016f95', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b992d6a65df4bf9b27c13192d57c5f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 17:13:53.037896 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b88a07-1407-4fec-88f2-f80334263e3f service neutron] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 17:13:53.038205 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b88a07-1407-4fec-88f2-f80334263e3f service neutron] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 17:13:53.038482 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b88a07-1407-4fec-88f2-f80334263e3f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 17:13:53.047297 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26b88a07-1407-4fec-88f2-f80334263e3f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 17:13:53.048777 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26b88a07-1407-4fec-88f2-f80334263e3f service neutron] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 17:13:53.058915 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 1434/2861] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 1 17:13:52 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 17:13:53.468867 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-635c7ea2-8333-46a0-8994-e4c551df4432 None None] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 17:13:53.470324 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.fernet_utils [None req-635c7ea2-8333-46a0-8994-e4c551df4432 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87057) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 17:13:53.484628 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-635c7ea2-8333-46a0-8994-e4c551df4432 None None] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 17:13:53.490146 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d736a401-ccb8-47bd-bd76-b8757b4cbefb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 17:13:53.490805 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d736a401-ccb8-47bd-bd76-b8757b4cbefb admin admin] REQUEST_METHOD: `DELETE` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 17:13:53.491087 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d736a401-ccb8-47bd-bd76-b8757b4cbefb admin admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 17:13:53.491362 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d736a401-ccb8-47bd-bd76-b8757b4cbefb admin admin] PATH_INFO: `/v3/projects/d4ec10c3af95431db5ad0a1105c57022` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 17:13:53.492550 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d736a401-ccb8-47bd-bd76-b8757b4cbefb admin admin] RBAC: Authorizing `identity:delete_project(project_id=d4ec10c3af95431db5ad0a1105c57022)` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 17:13:53.494638 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d736a401-ccb8-47bd-bd76-b8757b4cbefb admin admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 17:13:53.584296 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.notifications [None req-d736a401-ccb8-47bd-bd76-b8757b4cbefb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd4ec10c3af95431db5ad0a1105c57022', 'request_id': 'req-d736a401-ccb8-47bd-bd76-b8757b4cbefb'} {{(pid=87057) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 17:13:53.590519 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.notifications [None req-d736a401-ccb8-47bd-bd76-b8757b4cbefb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd4ec10c3af95431db5ad0a1105c57022', 'request_id': 'req-d736a401-ccb8-47bd-bd76-b8757b4cbefb'} {{(pid=87057) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 17:13:53.596888 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.notifications [None req-d736a401-ccb8-47bd-bd76-b8757b4cbefb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd4ec10c3af95431db5ad0a1105c57022', 'request_id': 'req-d736a401-ccb8-47bd-bd76-b8757b4cbefb'} {{(pid=87057) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 17:13:53.654838 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.notifications [None req-d736a401-ccb8-47bd-bd76-b8757b4cbefb admin admin] The token cache is being invalidate because project d4ec10c3af95431db5ad0a1105c57022 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87057) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 17:13:53.655113 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.notifications [None req-d736a401-ccb8-47bd-bd76-b8757b4cbefb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87057) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 17:13:53.655762 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.notifications [None req-d736a401-ccb8-47bd-bd76-b8757b4cbefb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87057) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 17:13:53.657446 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 1428/2862] 10.222.0.20 () {58 vars in 1233 bytes} [Mon Aug 1 17:13:53 2022] DELETE /identity/v3/projects/d4ec10c3af95431db5ad0a1105c57022 => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 17:13:53.662385 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b88a07-1407-4fec-88f2-f80334263e3f service neutron] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 17:13:53.672413 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b88a07-1407-4fec-88f2-f80334263e3f service neutron] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 17:13:53.729165 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9030337-86f9-4bc0-ae09-4cda3e791f41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 17:13:53.729811 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9030337-86f9-4bc0-ae09-4cda3e791f41 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 17:13:53.730130 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9030337-86f9-4bc0-ae09-4cda3e791f41 admin admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 17:13:53.730406 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9030337-86f9-4bc0-ae09-4cda3e791f41 admin admin] PATH_INFO: `/v3/users/c6cd9f88ce7944d4838bc5ad049cf925` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 17:13:53.731574 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9030337-86f9-4bc0-ae09-4cda3e791f41 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c6cd9f88ce7944d4838bc5ad049cf925)` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 17:13:53.733788 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9030337-86f9-4bc0-ae09-4cda3e791f41 admin admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 17:13:53.874658 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.notifications [None req-d9030337-86f9-4bc0-ae09-4cda3e791f41 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c6cd9f88ce7944d4838bc5ad049cf925'} {{(pid=87058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 17:13:53.875223 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.notifications [None req-d9030337-86f9-4bc0-ae09-4cda3e791f41 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c6cd9f88ce7944d4838bc5ad049cf925'} {{(pid=87058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 17:13:53.875830 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.notifications [None req-d9030337-86f9-4bc0-ae09-4cda3e791f41 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c6cd9f88ce7944d4838bc5ad049cf925'} {{(pid=87058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 17:13:53.888434 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.notifications [None req-d9030337-86f9-4bc0-ae09-4cda3e791f41 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c6cd9f88ce7944d4838bc5ad049cf925'} {{(pid=87058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 17:13:53.896251 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.notifications [None req-d9030337-86f9-4bc0-ae09-4cda3e791f41 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c6cd9f88ce7944d4838bc5ad049cf925'} {{(pid=87058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 17:13:53.952982 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 1435/2863] 10.222.0.20 () {58 vars in 1224 bytes} [Mon Aug 1 17:13:53 2022] DELETE /identity/v3/users/c6cd9f88ce7944d4838bc5ad049cf925 => generated 0 bytes in 291 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 17:13:54.360313 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d736a401-ccb8-47bd-bd76-b8757b4cbefb admin admin] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 17:13:54.363974 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.fernet_utils [None req-d736a401-ccb8-47bd-bd76-b8757b4cbefb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87057) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 17:13:54.376772 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d736a401-ccb8-47bd-bd76-b8757b4cbefb admin admin] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 17:13:54.436972 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f6c525-98e5-42b0-b927-e0610e4d239a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 17:13:54.437538 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f6c525-98e5-42b0-b927-e0610e4d239a admin admin] REQUEST_METHOD: `DELETE` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 17:13:54.437819 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f6c525-98e5-42b0-b927-e0610e4d239a admin admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 17:13:54.438129 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f6c525-98e5-42b0-b927-e0610e4d239a admin admin] PATH_INFO: `/v3/projects/dc0e9b8ed5124029b7f8e863cbe0dc77` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 17:13:54.439297 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96f6c525-98e5-42b0-b927-e0610e4d239a admin admin] RBAC: Authorizing `identity:delete_project(project_id=dc0e9b8ed5124029b7f8e863cbe0dc77)` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 17:13:54.441820 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96f6c525-98e5-42b0-b927-e0610e4d239a admin admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 17:13:54.550740 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG dbcounter [-] [87057] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87057) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 17:13:54.554385 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.notifications [None req-96f6c525-98e5-42b0-b927-e0610e4d239a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dc0e9b8ed5124029b7f8e863cbe0dc77', 'request_id': 'req-96f6c525-98e5-42b0-b927-e0610e4d239a'} {{(pid=87057) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 17:13:54.560466 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.notifications [None req-96f6c525-98e5-42b0-b927-e0610e4d239a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dc0e9b8ed5124029b7f8e863cbe0dc77', 'request_id': 'req-96f6c525-98e5-42b0-b927-e0610e4d239a'} {{(pid=87057) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 17:13:54.566674 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.notifications [None req-96f6c525-98e5-42b0-b927-e0610e4d239a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dc0e9b8ed5124029b7f8e863cbe0dc77', 'request_id': 'req-96f6c525-98e5-42b0-b927-e0610e4d239a'} {{(pid=87057) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 17:13:54.622233 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.notifications [None req-96f6c525-98e5-42b0-b927-e0610e4d239a admin admin] The token cache is being invalidate because project dc0e9b8ed5124029b7f8e863cbe0dc77 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87057) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 17:13:54.622658 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.notifications [None req-96f6c525-98e5-42b0-b927-e0610e4d239a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87057) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 17:13:54.623392 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.notifications [None req-96f6c525-98e5-42b0-b927-e0610e4d239a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87057) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 17:13:54.625053 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 1429/2864] 10.222.0.20 () {58 vars in 1233 bytes} [Mon Aug 1 17:13:54 2022] DELETE /identity/v3/projects/dc0e9b8ed5124029b7f8e863cbe0dc77 => generated 0 bytes in 265 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 17:13:54.630067 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa60d860-c1a5-4dab-8234-dff22e5d9d6b None None] REQUEST_METHOD: `POST` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 17:13:54.630345 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa60d860-c1a5-4dab-8234-dff22e5d9d6b None None] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 17:13:54.630627 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa60d860-c1a5-4dab-8234-dff22e5d9d6b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 17:13:54.650228 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.auth.core [None req-aa60d860-c1a5-4dab-8234-dff22e5d9d6b None None] MFA Rules not processed for user `1007f8e379b74e169b98fb7c3c0ba5b4`. Rule list: `[]` (Enabled: `True`). {{(pid=87058) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 17:13:54.651442 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.fernet_utils [None req-aa60d860-c1a5-4dab-8234-dff22e5d9d6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 17:13:54.681985 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG dbcounter [-] [87058] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 17:13:54.710398 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 1436/2865] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 1 17:13:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 17:13:54.717971 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48189c9-df5a-4f00-8d3f-28a025389923 None None] REQUEST_METHOD: `POST` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 17:13:54.718284 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48189c9-df5a-4f00-8d3f-28a025389923 None None] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 17:13:54.718558 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48189c9-df5a-4f00-8d3f-28a025389923 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 17:13:54.739969 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.auth.core [None req-f48189c9-df5a-4f00-8d3f-28a025389923 None None] MFA Rules not processed for user `1007f8e379b74e169b98fb7c3c0ba5b4`. Rule list: `[]` (Enabled: `True`). {{(pid=87057) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 17:13:54.741327 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.fernet_utils [None req-f48189c9-df5a-4f00-8d3f-28a025389923 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87057) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 17:13:54.749121 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 1430/2866] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 1 17:13:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 17:13:54.753622 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa60d860-c1a5-4dab-8234-dff22e5d9d6b None None] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 17:13:54.763424 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa60d860-c1a5-4dab-8234-dff22e5d9d6b None None] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 17:13:54.768137 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fa9dd91-3f37-44e0-86b5-2443f2e4d9ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 17:13:54.768743 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa9dd91-3f37-44e0-86b5-2443f2e4d9ad admin admin] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 17:13:54.769025 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa9dd91-3f37-44e0-86b5-2443f2e4d9ad admin admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 17:13:54.769298 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa9dd91-3f37-44e0-86b5-2443f2e4d9ad admin admin] PATH_INFO: `/v3/domains` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 17:13:54.770025 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fa9dd91-3f37-44e0-86b5-2443f2e4d9ad admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87058) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 17:13:54.770355 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fa9dd91-3f37-44e0-86b5-2443f2e4d9ad admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 17:13:54.771976 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fa9dd91-3f37-44e0-86b5-2443f2e4d9ad admin admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 17:13:54.784213 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 1437/2867] 10.222.0.20 () {58 vars in 1153 bytes} [Mon Aug 1 17:13:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 17:13:54.788207 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f48189c9-df5a-4f00-8d3f-28a025389923 None None] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 17:13:54.797443 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f48189c9-df5a-4f00-8d3f-28a025389923 None None] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 17:13:54.802719 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7845c336-2d5f-4655-a133-0e564a9fecfa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 17:13:54.803321 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7845c336-2d5f-4655-a133-0e564a9fecfa admin admin] REQUEST_METHOD: `POST` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 17:13:54.803602 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7845c336-2d5f-4655-a133-0e564a9fecfa admin admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 17:13:54.803876 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7845c336-2d5f-4655-a133-0e564a9fecfa admin admin] PATH_INFO: `/v3/projects` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 17:13:54.804589 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7845c336-2d5f-4655-a133-0e564a9fecfa admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 17:13:54.806530 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7845c336-2d5f-4655-a133-0e564a9fecfa admin admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 17:13:54.886186 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 1431/2868] 10.222.0.20 () {60 vars in 1153 bytes} [Mon Aug 1 17:13:54 2022] POST /identity/v3/projects => generated 381 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 17:13:54.890267 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fa9dd91-3f37-44e0-86b5-2443f2e4d9ad admin admin] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 17:13:54.900640 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fa9dd91-3f37-44e0-86b5-2443f2e4d9ad admin admin] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 17:13:54.959393 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6522ade0-23cc-447f-a08b-34abc24185f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 17:13:54.960008 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6522ade0-23cc-447f-a08b-34abc24185f7 admin admin] REQUEST_METHOD: `POST` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 17:13:54.960284 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6522ade0-23cc-447f-a08b-34abc24185f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 17:13:54.960610 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6522ade0-23cc-447f-a08b-34abc24185f7 admin admin] PATH_INFO: `/v3/users` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 17:13:54.961222 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6522ade0-23cc-447f-a08b-34abc24185f7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 17:13:54.963161 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6522ade0-23cc-447f-a08b-34abc24185f7 admin admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 17:13:55.044706 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 1438/2869] 10.222.0.20 () {60 vars in 1144 bytes} [Mon Aug 1 17:13:54 2022] POST /identity/v3/users => generated 291 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 17:13:55.049006 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7845c336-2d5f-4655-a133-0e564a9fecfa admin admin] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 17:13:55.059485 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7845c336-2d5f-4655-a133-0e564a9fecfa admin admin] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 17:13:55.064145 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de68b091-e1b5-4d27-b00c-b6f90dd98ea7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 17:13:55.064862 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de68b091-e1b5-4d27-b00c-b6f90dd98ea7 admin admin] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 17:13:55.065168 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de68b091-e1b5-4d27-b00c-b6f90dd98ea7 admin admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 17:13:55.065451 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de68b091-e1b5-4d27-b00c-b6f90dd98ea7 admin admin] PATH_INFO: `/v3/roles` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 17:13:55.066281 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de68b091-e1b5-4d27-b00c-b6f90dd98ea7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 17:13:55.067876 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de68b091-e1b5-4d27-b00c-b6f90dd98ea7 admin admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 17:13:55.077615 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 1432/2870] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Aug 1 17:13:55 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 17:13:55.082298 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6522ade0-23cc-447f-a08b-34abc24185f7 admin admin] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 17:13:55.092260 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6522ade0-23cc-447f-a08b-34abc24185f7 admin admin] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 17:13:55.097842 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df3df37d-0cd9-47eb-91be-ea65b6845555 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 17:13:55.098499 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df3df37d-0cd9-47eb-91be-ea65b6845555 admin admin] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 17:13:55.098785 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df3df37d-0cd9-47eb-91be-ea65b6845555 admin admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 17:13:55.099057 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df3df37d-0cd9-47eb-91be-ea65b6845555 admin admin] PATH_INFO: `/v3/roles` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 17:13:55.099802 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df3df37d-0cd9-47eb-91be-ea65b6845555 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 17:13:55.101434 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df3df37d-0cd9-47eb-91be-ea65b6845555 admin admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 17:13:55.113077 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 1439/2871] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Aug 1 17:13:55 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 01 17:13:55.117240 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de68b091-e1b5-4d27-b00c-b6f90dd98ea7 admin admin] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 17:13:55.127558 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de68b091-e1b5-4d27-b00c-b6f90dd98ea7 admin admin] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 17:13:55.132284 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c57538cc-3136-4f35-a0b4-5880978fa6b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 17:13:55.132949 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57538cc-3136-4f35-a0b4-5880978fa6b4 admin admin] REQUEST_METHOD: `PUT` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 17:13:55.133231 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57538cc-3136-4f35-a0b4-5880978fa6b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 17:13:55.133506 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57538cc-3136-4f35-a0b4-5880978fa6b4 admin admin] PATH_INFO: `/v3/projects/a4e07966ee514b28ae789d388e78be1d/users/7a3d4a251bb84ce688f5d990b827e582/roles/0999d3da63a545f18ff3252afdc90518` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 17:13:55.150052 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c57538cc-3136-4f35-a0b4-5880978fa6b4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a4e07966ee514b28ae789d388e78be1d, user_id=7a3d4a251bb84ce688f5d990b827e582, role_id=0999d3da63a545f18ff3252afdc90518)` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 17:13:55.152852 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c57538cc-3136-4f35-a0b4-5880978fa6b4 admin admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 17:13:55.256148 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 1433/2872] 10.222.0.20 () {60 vars in 1483 bytes} [Mon Aug 1 17:13:55 2022] PUT /identity/v3/projects/a4e07966ee514b28ae789d388e78be1d/users/7a3d4a251bb84ce688f5d990b827e582/roles/0999d3da63a545f18ff3252afdc90518 => generated 0 bytes in 139 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 17:13:55.266440 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948458eb-9399-445a-844c-350b646a1575 None None] REQUEST_METHOD: `POST` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 17:13:55.266729 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948458eb-9399-445a-844c-350b646a1575 None None] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 17:13:55.267007 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948458eb-9399-445a-844c-350b646a1575 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 17:13:55.295373 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.auth.core [None req-948458eb-9399-445a-844c-350b646a1575 None None] MFA Rules not processed for user `7a3d4a251bb84ce688f5d990b827e582`. Rule list: `[]` (Enabled: `True`). {{(pid=87058) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 17:13:55.296595 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.fernet_utils [None req-948458eb-9399-445a-844c-350b646a1575 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 17:13:55.360090 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 1440/2873] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 1 17:13:55 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 17:13:55.365033 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c57538cc-3136-4f35-a0b4-5880978fa6b4 admin admin] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 17:13:55.427543 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c57538cc-3136-4f35-a0b4-5880978fa6b4 admin admin] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 17:13:55.432241 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfb09f1a-f3d2-424c-afe4-0e69b39457d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 17:13:55.432863 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb09f1a-f3d2-424c-afe4-0e69b39457d2 admin admin] REQUEST_METHOD: `POST` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 17:13:55.433146 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb09f1a-f3d2-424c-afe4-0e69b39457d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 17:13:55.433422 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb09f1a-f3d2-424c-afe4-0e69b39457d2 admin admin] PATH_INFO: `/v3/projects` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 17:13:55.434128 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfb09f1a-f3d2-424c-afe4-0e69b39457d2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 17:13:55.436072 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfb09f1a-f3d2-424c-afe4-0e69b39457d2 admin admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 17:13:55.514180 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 1434/2874] 10.222.0.20 () {60 vars in 1153 bytes} [Mon Aug 1 17:13:55 2022] POST /identity/v3/projects => generated 379 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 17:13:55.518965 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-948458eb-9399-445a-844c-350b646a1575 None None] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 17:13:55.529628 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-948458eb-9399-445a-844c-350b646a1575 None None] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 17:13:55.557832 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG dbcounter [-] [87058] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 17:13:55.586346 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa3b8966-be27-47bb-8915-1af31879ef46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 17:13:55.586967 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3b8966-be27-47bb-8915-1af31879ef46 admin admin] REQUEST_METHOD: `POST` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 17:13:55.587250 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3b8966-be27-47bb-8915-1af31879ef46 admin admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 17:13:55.587530 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3b8966-be27-47bb-8915-1af31879ef46 admin admin] PATH_INFO: `/v3/users` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 17:13:55.588208 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa3b8966-be27-47bb-8915-1af31879ef46 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 17:13:55.590196 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa3b8966-be27-47bb-8915-1af31879ef46 admin admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 17:13:55.667188 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 1441/2875] 10.222.0.20 () {60 vars in 1144 bytes} [Mon Aug 1 17:13:55 2022] POST /identity/v3/users => generated 296 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 17:13:55.671080 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfb09f1a-f3d2-424c-afe4-0e69b39457d2 admin admin] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 17:13:55.681507 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfb09f1a-f3d2-424c-afe4-0e69b39457d2 admin admin] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 17:13:55.686443 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a043710-d10c-49a6-b3fa-15f85802ff32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 17:13:55.687040 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a043710-d10c-49a6-b3fa-15f85802ff32 admin admin] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 17:13:55.687318 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a043710-d10c-49a6-b3fa-15f85802ff32 admin admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 17:13:55.687589 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a043710-d10c-49a6-b3fa-15f85802ff32 admin admin] PATH_INFO: `/v3/roles` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 17:13:55.688275 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a043710-d10c-49a6-b3fa-15f85802ff32 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 17:13:55.689853 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a043710-d10c-49a6-b3fa-15f85802ff32 admin admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 17:13:55.701141 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 1435/2876] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Aug 1 17:13:55 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 01 17:13:55.705155 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa3b8966-be27-47bb-8915-1af31879ef46 admin admin] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 17:13:55.716294 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa3b8966-be27-47bb-8915-1af31879ef46 admin admin] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 17:13:55.720908 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e5147e-fd37-43d8-8cdd-cafe198c307f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 17:13:55.721552 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e5147e-fd37-43d8-8cdd-cafe198c307f admin admin] REQUEST_METHOD: `PUT` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 17:13:55.722018 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e5147e-fd37-43d8-8cdd-cafe198c307f admin admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 17:13:55.722323 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e5147e-fd37-43d8-8cdd-cafe198c307f admin admin] PATH_INFO: `/v3/projects/7e2d53053652497a8f358c18d22e8776/users/4a15dd1729f94f23b6362e6cbe34f124/roles/440bb943dd5b46fea038b903b6eaaf81` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 17:13:55.737376 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57e5147e-fd37-43d8-8cdd-cafe198c307f admin admin] RBAC: Authorizing `identity:create_grant(project_id=7e2d53053652497a8f358c18d22e8776, user_id=4a15dd1729f94f23b6362e6cbe34f124, role_id=440bb943dd5b46fea038b903b6eaaf81)` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 17:13:55.740710 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57e5147e-fd37-43d8-8cdd-cafe198c307f admin admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 17:13:55.805014 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 1442/2877] 10.222.0.20 () {60 vars in 1483 bytes} [Mon Aug 1 17:13:55 2022] PUT /identity/v3/projects/7e2d53053652497a8f358c18d22e8776/users/4a15dd1729f94f23b6362e6cbe34f124/roles/440bb943dd5b46fea038b903b6eaaf81 => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 17:13:55.808901 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a043710-d10c-49a6-b3fa-15f85802ff32 admin admin] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 17:13:55.819673 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a043710-d10c-49a6-b3fa-15f85802ff32 admin admin] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 17:13:55.858500 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG dbcounter [-] [87057] Writing DB stats keystone:SELECT=92,keystone:DELETE=4,keystone:INSERT=4 {{(pid=87057) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 17:13:55.874643 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-393cc58f-aa2e-4618-8fb5-5911b2ca0806 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 17:13:55.875257 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393cc58f-aa2e-4618-8fb5-5911b2ca0806 admin admin] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 17:13:55.875536 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393cc58f-aa2e-4618-8fb5-5911b2ca0806 admin admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 17:13:55.875811 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393cc58f-aa2e-4618-8fb5-5911b2ca0806 admin admin] PATH_INFO: `/v3/roles` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 17:13:55.876500 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-393cc58f-aa2e-4618-8fb5-5911b2ca0806 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 17:13:55.878692 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-393cc58f-aa2e-4618-8fb5-5911b2ca0806 admin admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 17:13:55.888587 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 1436/2878] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Aug 1 17:13:55 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 17:13:55.892712 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e5147e-fd37-43d8-8cdd-cafe198c307f admin admin] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 17:13:55.904066 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e5147e-fd37-43d8-8cdd-cafe198c307f admin admin] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 17:13:55.909168 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e276e9-831c-4ef5-8fda-3d623d9f236c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 17:13:55.921978 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e276e9-831c-4ef5-8fda-3d623d9f236c admin admin] REQUEST_METHOD: `PUT` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 17:13:55.921978 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e276e9-831c-4ef5-8fda-3d623d9f236c admin admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 17:13:55.921978 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e276e9-831c-4ef5-8fda-3d623d9f236c admin admin] PATH_INFO: `/v3/projects/7e2d53053652497a8f358c18d22e8776/users/4a15dd1729f94f23b6362e6cbe34f124/roles/440bb943dd5b46fea038b903b6eaaf81` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 17:13:55.921978 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21e276e9-831c-4ef5-8fda-3d623d9f236c admin admin] RBAC: Authorizing `identity:create_grant(project_id=7e2d53053652497a8f358c18d22e8776, user_id=4a15dd1729f94f23b6362e6cbe34f124, role_id=440bb943dd5b46fea038b903b6eaaf81)` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 17:13:55.921978 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21e276e9-831c-4ef5-8fda-3d623d9f236c admin admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 17:13:55.926896 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 1443/2879] 10.222.0.20 () {60 vars in 1483 bytes} [Mon Aug 1 17:13:55 2022] PUT /identity/v3/projects/7e2d53053652497a8f358c18d22e8776/users/4a15dd1729f94f23b6362e6cbe34f124/roles/440bb943dd5b46fea038b903b6eaaf81 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 17:13:55.937855 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db398cee-e80d-4809-99a9-dd3ec7b81c35 None None] REQUEST_METHOD: `POST` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 17:13:55.938176 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db398cee-e80d-4809-99a9-dd3ec7b81c35 None None] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 17:13:55.938454 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db398cee-e80d-4809-99a9-dd3ec7b81c35 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 17:13:55.966809 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.auth.core [None req-db398cee-e80d-4809-99a9-dd3ec7b81c35 None None] MFA Rules not processed for user `4a15dd1729f94f23b6362e6cbe34f124`. Rule list: `[]` (Enabled: `True`). {{(pid=87057) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 17:13:55.968007 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.fernet_utils [None req-db398cee-e80d-4809-99a9-dd3ec7b81c35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87057) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 17:13:56.035963 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 1437/2880] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 1 17:13:55 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 17:13:56.046551 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e276e9-831c-4ef5-8fda-3d623d9f236c admin admin] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 17:13:56.048236 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.fernet_utils [None req-21e276e9-831c-4ef5-8fda-3d623d9f236c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 17:13:56.105311 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e276e9-831c-4ef5-8fda-3d623d9f236c admin admin] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 17:13:56.109130 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-228148c7-3b7c-41e3-b10a-e72693a27917 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25970b52ca15448fa1abb1f974d64038', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b992d6a65df4bf9b27c13192d57c5f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 17:13:56.109764 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228148c7-3b7c-41e3-b10a-e72693a27917 service cinder] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 17:13:56.110061 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228148c7-3b7c-41e3-b10a-e72693a27917 service cinder] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 17:13:56.110349 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228148c7-3b7c-41e3-b10a-e72693a27917 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 17:13:56.120377 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-228148c7-3b7c-41e3-b10a-e72693a27917 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 17:13:56.121799 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-228148c7-3b7c-41e3-b10a-e72693a27917 service cinder] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 17:13:56.131213 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 1444/2881] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 1 17:13:56 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 01 17:13:56.733915 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db398cee-e80d-4809-99a9-dd3ec7b81c35 None None] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 17:13:56.794952 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db398cee-e80d-4809-99a9-dd3ec7b81c35 None None] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 17:13:56.799519 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-311a1725-2e52-4d87-bf30-10b8be2cac15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 17:13:56.800181 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311a1725-2e52-4d87-bf30-10b8be2cac15 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 17:13:56.800464 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311a1725-2e52-4d87-bf30-10b8be2cac15 admin admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 17:13:56.800733 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311a1725-2e52-4d87-bf30-10b8be2cac15 admin admin] PATH_INFO: `/v3/users/7a3d4a251bb84ce688f5d990b827e582` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 17:13:56.801938 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-311a1725-2e52-4d87-bf30-10b8be2cac15 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7a3d4a251bb84ce688f5d990b827e582)` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 17:13:56.803971 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-311a1725-2e52-4d87-bf30-10b8be2cac15 admin admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 17:13:56.978329 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.notifications [None req-311a1725-2e52-4d87-bf30-10b8be2cac15 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7a3d4a251bb84ce688f5d990b827e582'} {{(pid=87057) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 17:13:56.978969 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.notifications [None req-311a1725-2e52-4d87-bf30-10b8be2cac15 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7a3d4a251bb84ce688f5d990b827e582'} {{(pid=87057) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 17:13:57.032883 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.notifications [None req-311a1725-2e52-4d87-bf30-10b8be2cac15 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7a3d4a251bb84ce688f5d990b827e582'} {{(pid=87057) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 17:13:57.033488 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.notifications [None req-311a1725-2e52-4d87-bf30-10b8be2cac15 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7a3d4a251bb84ce688f5d990b827e582'} {{(pid=87057) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 17:13:57.041376 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.notifications [None req-311a1725-2e52-4d87-bf30-10b8be2cac15 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7a3d4a251bb84ce688f5d990b827e582'} {{(pid=87057) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 17:13:57.043429 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG dbcounter [-] [87057] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=87057) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 17:13:57.059453 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 1438/2882] 10.222.0.20 () {58 vars in 1224 bytes} [Mon Aug 1 17:13:56 2022] DELETE /identity/v3/users/7a3d4a251bb84ce688f5d990b827e582 => generated 0 bytes in 326 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 17:13:57.063931 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a57de1-4153-41aa-9752-7ee8f8e3ab4e None None] REQUEST_METHOD: `POST` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 17:13:57.064450 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a57de1-4153-41aa-9752-7ee8f8e3ab4e None None] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 17:13:57.064450 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a57de1-4153-41aa-9752-7ee8f8e3ab4e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 17:13:57.083530 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.auth.core [None req-03a57de1-4153-41aa-9752-7ee8f8e3ab4e None None] MFA Rules not processed for user `1007f8e379b74e169b98fb7c3c0ba5b4`. Rule list: `[]` (Enabled: `True`). {{(pid=87058) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 17:13:57.084724 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.fernet_utils [None req-03a57de1-4153-41aa-9752-7ee8f8e3ab4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 17:13:57.144545 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 1445/2883] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 1 17:13:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 17:13:57.154208 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-311a1725-2e52-4d87-bf30-10b8be2cac15 admin admin] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 17:13:57.155613 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.fernet_utils [None req-311a1725-2e52-4d87-bf30-10b8be2cac15 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87057) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 17:13:57.212758 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-311a1725-2e52-4d87-bf30-10b8be2cac15 admin admin] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 17:13:57.216451 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5283fd54-dcf1-4a0f-b671-f960bd0ecc11 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d9bd8d649154c979be3744bfd016f95', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b992d6a65df4bf9b27c13192d57c5f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 17:13:57.217067 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5283fd54-dcf1-4a0f-b671-f960bd0ecc11 service neutron] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 17:13:57.217345 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5283fd54-dcf1-4a0f-b671-f960bd0ecc11 service neutron] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 17:13:57.217616 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5283fd54-dcf1-4a0f-b671-f960bd0ecc11 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 17:13:57.227545 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5283fd54-dcf1-4a0f-b671-f960bd0ecc11 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 17:13:57.229035 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5283fd54-dcf1-4a0f-b671-f960bd0ecc11 service neutron] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 17:13:57.238884 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 1439/2884] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 1 17:13:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 17:13:57.663267 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03a57de1-4153-41aa-9752-7ee8f8e3ab4e None None] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 17:13:57.664686 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.fernet_utils [None req-03a57de1-4153-41aa-9752-7ee8f8e3ab4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 17:13:57.676794 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03a57de1-4153-41aa-9752-7ee8f8e3ab4e None None] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 17:13:57.681546 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e39761d-89d8-4f0b-9d35-a079c40fd7ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 17:13:57.682218 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e39761d-89d8-4f0b-9d35-a079c40fd7ed admin admin] REQUEST_METHOD: `DELETE` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 17:13:57.682493 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e39761d-89d8-4f0b-9d35-a079c40fd7ed admin admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 17:13:57.682873 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e39761d-89d8-4f0b-9d35-a079c40fd7ed admin admin] PATH_INFO: `/v3/projects/a4e07966ee514b28ae789d388e78be1d` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 17:13:57.684045 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e39761d-89d8-4f0b-9d35-a079c40fd7ed admin admin] RBAC: Authorizing `identity:delete_project(project_id=a4e07966ee514b28ae789d388e78be1d)` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 17:13:57.685934 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e39761d-89d8-4f0b-9d35-a079c40fd7ed admin admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 17:13:57.710343 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG dbcounter [-] [87058] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=87058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 17:13:57.811699 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.notifications [None req-0e39761d-89d8-4f0b-9d35-a079c40fd7ed admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a4e07966ee514b28ae789d388e78be1d', 'request_id': 'req-0e39761d-89d8-4f0b-9d35-a079c40fd7ed'} {{(pid=87058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 17:13:57.818202 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.notifications [None req-0e39761d-89d8-4f0b-9d35-a079c40fd7ed admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a4e07966ee514b28ae789d388e78be1d', 'request_id': 'req-0e39761d-89d8-4f0b-9d35-a079c40fd7ed'} {{(pid=87058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 17:13:57.824293 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.notifications [None req-0e39761d-89d8-4f0b-9d35-a079c40fd7ed admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a4e07966ee514b28ae789d388e78be1d', 'request_id': 'req-0e39761d-89d8-4f0b-9d35-a079c40fd7ed'} {{(pid=87058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 17:13:57.895072 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.notifications [None req-0e39761d-89d8-4f0b-9d35-a079c40fd7ed admin admin] The token cache is being invalidate because project a4e07966ee514b28ae789d388e78be1d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87058) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 17:13:57.895359 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.notifications [None req-0e39761d-89d8-4f0b-9d35-a079c40fd7ed admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 17:13:57.895987 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.notifications [None req-0e39761d-89d8-4f0b-9d35-a079c40fd7ed admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 17:13:57.897791 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 1446/2885] 10.222.0.20 () {58 vars in 1233 bytes} [Mon Aug 1 17:13:57 2022] DELETE /identity/v3/projects/a4e07966ee514b28ae789d388e78be1d => generated 0 bytes in 235 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 17:13:57.901768 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5283fd54-dcf1-4a0f-b671-f960bd0ecc11 service neutron] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 17:13:57.912433 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5283fd54-dcf1-4a0f-b671-f960bd0ecc11 service neutron] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 17:13:57.967563 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c6b715f-b8d8-4136-af70-9301bdadbea5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 17:13:57.968229 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6b715f-b8d8-4136-af70-9301bdadbea5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 17:13:57.968512 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6b715f-b8d8-4136-af70-9301bdadbea5 admin admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 17:13:57.968788 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6b715f-b8d8-4136-af70-9301bdadbea5 admin admin] PATH_INFO: `/v3/users/4a15dd1729f94f23b6362e6cbe34f124` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 17:13:57.969947 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c6b715f-b8d8-4136-af70-9301bdadbea5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4a15dd1729f94f23b6362e6cbe34f124)` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 17:13:57.971904 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c6b715f-b8d8-4136-af70-9301bdadbea5 admin admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 17:13:58.150804 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.notifications [None req-1c6b715f-b8d8-4136-af70-9301bdadbea5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4a15dd1729f94f23b6362e6cbe34f124'} {{(pid=87057) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 17:13:58.151428 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.notifications [None req-1c6b715f-b8d8-4136-af70-9301bdadbea5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4a15dd1729f94f23b6362e6cbe34f124'} {{(pid=87057) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 17:13:58.210206 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.notifications [None req-1c6b715f-b8d8-4136-af70-9301bdadbea5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4a15dd1729f94f23b6362e6cbe34f124'} {{(pid=87057) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 17:13:58.210830 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.notifications [None req-1c6b715f-b8d8-4136-af70-9301bdadbea5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4a15dd1729f94f23b6362e6cbe34f124'} {{(pid=87057) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 17:13:58.217987 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.notifications [None req-1c6b715f-b8d8-4136-af70-9301bdadbea5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4a15dd1729f94f23b6362e6cbe34f124'} {{(pid=87057) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 17:13:58.236101 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 1440/2886] 10.222.0.20 () {58 vars in 1224 bytes} [Mon Aug 1 17:13:57 2022] DELETE /identity/v3/users/4a15dd1729f94f23b6362e6cbe34f124 => generated 0 bytes in 335 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 17:13:58.594507 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e39761d-89d8-4f0b-9d35-a079c40fd7ed admin admin] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 17:13:58.596131 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.fernet_utils [None req-0e39761d-89d8-4f0b-9d35-a079c40fd7ed admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 17:13:58.609528 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e39761d-89d8-4f0b-9d35-a079c40fd7ed admin admin] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 17:13:58.665699 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb2d7266-ec5c-4fbd-97ed-c1863be8c9de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 17:13:58.666367 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2d7266-ec5c-4fbd-97ed-c1863be8c9de admin admin] REQUEST_METHOD: `DELETE` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 17:13:58.666645 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2d7266-ec5c-4fbd-97ed-c1863be8c9de admin admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 17:13:58.666913 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2d7266-ec5c-4fbd-97ed-c1863be8c9de admin admin] PATH_INFO: `/v3/projects/7e2d53053652497a8f358c18d22e8776` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 17:13:58.668050 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb2d7266-ec5c-4fbd-97ed-c1863be8c9de admin admin] RBAC: Authorizing `identity:delete_project(project_id=7e2d53053652497a8f358c18d22e8776)` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 17:13:58.669944 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb2d7266-ec5c-4fbd-97ed-c1863be8c9de admin admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 17:13:58.764007 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.notifications [None req-fb2d7266-ec5c-4fbd-97ed-c1863be8c9de admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7e2d53053652497a8f358c18d22e8776', 'request_id': 'req-fb2d7266-ec5c-4fbd-97ed-c1863be8c9de'} {{(pid=87058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 17:13:58.770066 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.notifications [None req-fb2d7266-ec5c-4fbd-97ed-c1863be8c9de admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7e2d53053652497a8f358c18d22e8776', 'request_id': 'req-fb2d7266-ec5c-4fbd-97ed-c1863be8c9de'} {{(pid=87058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 17:13:58.775497 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.notifications [None req-fb2d7266-ec5c-4fbd-97ed-c1863be8c9de admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7e2d53053652497a8f358c18d22e8776', 'request_id': 'req-fb2d7266-ec5c-4fbd-97ed-c1863be8c9de'} {{(pid=87058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 17:13:58.837601 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.notifications [None req-fb2d7266-ec5c-4fbd-97ed-c1863be8c9de admin admin] The token cache is being invalidate because project 7e2d53053652497a8f358c18d22e8776 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87058) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 17:13:58.837884 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.notifications [None req-fb2d7266-ec5c-4fbd-97ed-c1863be8c9de admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 17:13:58.838556 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.notifications [None req-fb2d7266-ec5c-4fbd-97ed-c1863be8c9de admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 17:13:58.840096 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 1447/2887] 10.222.0.20 () {58 vars in 1233 bytes} [Mon Aug 1 17:13:58 2022] DELETE /identity/v3/projects/7e2d53053652497a8f358c18d22e8776 => generated 0 bytes in 246 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 17:13:58.845969 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1533cbf8-8dd8-42e1-b58d-a41c9f983cbc None None] REQUEST_METHOD: `POST` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 17:13:58.849958 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1533cbf8-8dd8-42e1-b58d-a41c9f983cbc None None] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 17:13:58.849958 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1533cbf8-8dd8-42e1-b58d-a41c9f983cbc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 17:13:58.855637 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG dbcounter [-] [87057] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=87057) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 17:13:58.867303 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.auth.core [None req-1533cbf8-8dd8-42e1-b58d-a41c9f983cbc None None] MFA Rules not processed for user `1007f8e379b74e169b98fb7c3c0ba5b4`. Rule list: `[]` (Enabled: `True`). {{(pid=87057) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 17:13:58.870057 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.fernet_utils [None req-1533cbf8-8dd8-42e1-b58d-a41c9f983cbc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87057) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 17:13:58.930603 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 1441/2888] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 1 17:13:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 17:13:58.938572 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08676fe7-76d7-4322-a758-f8451969d9a4 None None] REQUEST_METHOD: `POST` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 17:13:58.938907 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08676fe7-76d7-4322-a758-f8451969d9a4 None None] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 17:13:58.939190 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08676fe7-76d7-4322-a758-f8451969d9a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 17:13:58.959868 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.auth.core [None req-08676fe7-76d7-4322-a758-f8451969d9a4 None None] MFA Rules not processed for user `1007f8e379b74e169b98fb7c3c0ba5b4`. Rule list: `[]` (Enabled: `True`). {{(pid=87058) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 17:13:58.961078 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.fernet_utils [None req-08676fe7-76d7-4322-a758-f8451969d9a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 17:13:58.969252 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 1448/2889] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 1 17:13:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 17:13:58.974388 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1533cbf8-8dd8-42e1-b58d-a41c9f983cbc None None] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 17:13:58.984354 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1533cbf8-8dd8-42e1-b58d-a41c9f983cbc None None] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 17:13:58.989080 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a72b9eb-8d1a-4043-8d3f-48c15f79f249 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 17:13:58.989692 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a72b9eb-8d1a-4043-8d3f-48c15f79f249 admin admin] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 17:13:58.989975 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a72b9eb-8d1a-4043-8d3f-48c15f79f249 admin admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 17:13:58.990309 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a72b9eb-8d1a-4043-8d3f-48c15f79f249 admin admin] PATH_INFO: `/v3/domains` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 17:13:58.991008 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a72b9eb-8d1a-4043-8d3f-48c15f79f249 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87057) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 17:13:58.991338 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a72b9eb-8d1a-4043-8d3f-48c15f79f249 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 17:13:58.992931 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a72b9eb-8d1a-4043-8d3f-48c15f79f249 admin admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 17:13:59.004525 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 1442/2890] 10.222.0.20 () {58 vars in 1153 bytes} [Mon Aug 1 17:13:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 17:13:59.008737 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08676fe7-76d7-4322-a758-f8451969d9a4 None None] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 17:13:59.018281 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08676fe7-76d7-4322-a758-f8451969d9a4 None None] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 17:13:59.023695 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca624e2-7ff3-452e-a61e-5419f386478d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 17:13:59.024326 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca624e2-7ff3-452e-a61e-5419f386478d admin admin] REQUEST_METHOD: `POST` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 17:13:59.024609 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca624e2-7ff3-452e-a61e-5419f386478d admin admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 17:13:59.024906 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca624e2-7ff3-452e-a61e-5419f386478d admin admin] PATH_INFO: `/v3/projects` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 17:13:59.025567 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ca624e2-7ff3-452e-a61e-5419f386478d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 17:13:59.027496 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ca624e2-7ff3-452e-a61e-5419f386478d admin admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 17:13:59.114515 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 1449/2891] 10.222.0.20 () {60 vars in 1153 bytes} [Mon Aug 1 17:13:59 2022] POST /identity/v3/projects => generated 373 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 17:13:59.118858 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a72b9eb-8d1a-4043-8d3f-48c15f79f249 admin admin] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 17:13:59.129359 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a72b9eb-8d1a-4043-8d3f-48c15f79f249 admin admin] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 17:13:59.184640 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2bb0add-5593-4157-8e50-d67cbb1e5d70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 17:13:59.185184 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2bb0add-5593-4157-8e50-d67cbb1e5d70 admin admin] REQUEST_METHOD: `POST` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 17:13:59.185468 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2bb0add-5593-4157-8e50-d67cbb1e5d70 admin admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 17:13:59.185940 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2bb0add-5593-4157-8e50-d67cbb1e5d70 admin admin] PATH_INFO: `/v3/users` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 17:13:59.186650 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2bb0add-5593-4157-8e50-d67cbb1e5d70 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 17:13:59.188578 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2bb0add-5593-4157-8e50-d67cbb1e5d70 admin admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 17:13:59.284542 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 1443/2892] 10.222.0.20 () {60 vars in 1144 bytes} [Mon Aug 1 17:13:59 2022] POST /identity/v3/users => generated 287 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 17:13:59.289055 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca624e2-7ff3-452e-a61e-5419f386478d admin admin] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 17:13:59.299699 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca624e2-7ff3-452e-a61e-5419f386478d admin admin] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 17:13:59.304365 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b656e3-401f-4210-ac7e-aaf1e21f98b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 17:13:59.304967 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b656e3-401f-4210-ac7e-aaf1e21f98b3 admin admin] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 17:13:59.305259 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b656e3-401f-4210-ac7e-aaf1e21f98b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 17:13:59.305521 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b656e3-401f-4210-ac7e-aaf1e21f98b3 admin admin] PATH_INFO: `/v3/roles` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 17:13:59.306299 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1b656e3-401f-4210-ac7e-aaf1e21f98b3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 17:13:59.307897 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1b656e3-401f-4210-ac7e-aaf1e21f98b3 admin admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 17:13:59.313136 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG dbcounter [-] [87058] Writing DB stats keystone:DELETE=14,keystone:SELECT=83,keystone:INSERT=3 {{(pid=87058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 17:13:59.318789 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 1450/2893] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Aug 1 17:13:59 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 17:13:59.325790 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2bb0add-5593-4157-8e50-d67cbb1e5d70 admin admin] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 17:13:59.336729 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2bb0add-5593-4157-8e50-d67cbb1e5d70 admin admin] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 17:13:59.341452 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f185c509-d94c-450e-aae1-68930d58265e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 17:13:59.342086 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f185c509-d94c-450e-aae1-68930d58265e admin admin] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 17:13:59.347226 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f185c509-d94c-450e-aae1-68930d58265e admin admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 17:13:59.347226 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f185c509-d94c-450e-aae1-68930d58265e admin admin] PATH_INFO: `/v3/roles` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 17:13:59.347226 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f185c509-d94c-450e-aae1-68930d58265e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 17:13:59.347226 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f185c509-d94c-450e-aae1-68930d58265e admin admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 17:13:59.355896 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 1444/2894] 10.222.0.20 () {58 vars in 1122 bytes} [Mon Aug 1 17:13:59 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 17:13:59.360177 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b656e3-401f-4210-ac7e-aaf1e21f98b3 admin admin] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 17:13:59.370443 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b656e3-401f-4210-ac7e-aaf1e21f98b3 admin admin] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 17:13:59.375080 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-010a8a09-9a59-4deb-92f6-c26a39b6f8dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 17:13:59.375729 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010a8a09-9a59-4deb-92f6-c26a39b6f8dd admin admin] REQUEST_METHOD: `PUT` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 17:13:59.376117 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010a8a09-9a59-4deb-92f6-c26a39b6f8dd admin admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 17:13:59.376396 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010a8a09-9a59-4deb-92f6-c26a39b6f8dd admin admin] PATH_INFO: `/v3/projects/373368f3276a4b75bdc37f4510ea6b8b/users/24a9a164af284f04a1f3580bc7f133e5/roles/0999d3da63a545f18ff3252afdc90518` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 17:13:59.391894 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-010a8a09-9a59-4deb-92f6-c26a39b6f8dd admin admin] RBAC: Authorizing `identity:create_grant(project_id=373368f3276a4b75bdc37f4510ea6b8b, user_id=24a9a164af284f04a1f3580bc7f133e5, role_id=0999d3da63a545f18ff3252afdc90518)` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 17:13:59.398885 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-010a8a09-9a59-4deb-92f6-c26a39b6f8dd admin admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 17:13:59.518876 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 1451/2895] 10.222.0.20 () {60 vars in 1483 bytes} [Mon Aug 1 17:13:59 2022] PUT /identity/v3/projects/373368f3276a4b75bdc37f4510ea6b8b/users/24a9a164af284f04a1f3580bc7f133e5/roles/0999d3da63a545f18ff3252afdc90518 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 17:13:59.524043 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bae7207-9b24-480e-9393-b50e3e9cc21f None None] REQUEST_METHOD: `POST` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 17:13:59.524327 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bae7207-9b24-480e-9393-b50e3e9cc21f None None] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 17:13:59.524601 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bae7207-9b24-480e-9393-b50e3e9cc21f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 17:13:59.543388 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.auth.core [None req-6bae7207-9b24-480e-9393-b50e3e9cc21f None None] MFA Rules not processed for user `1007f8e379b74e169b98fb7c3c0ba5b4`. Rule list: `[]` (Enabled: `True`). {{(pid=87057) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 17:13:59.544580 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.fernet_utils [None req-6bae7207-9b24-480e-9393-b50e3e9cc21f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87057) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 17:13:59.601927 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 1445/2896] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 1 17:13:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 17:13:59.611585 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-010a8a09-9a59-4deb-92f6-c26a39b6f8dd admin admin] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 17:13:59.613833 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.fernet_utils [None req-010a8a09-9a59-4deb-92f6-c26a39b6f8dd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 17:13:59.671646 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-010a8a09-9a59-4deb-92f6-c26a39b6f8dd admin admin] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 17:13:59.675335 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6adcbec0-dc29-4202-b1c4-1afbe0825333 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d9bd8d649154c979be3744bfd016f95', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b992d6a65df4bf9b27c13192d57c5f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 17:13:59.676457 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6adcbec0-dc29-4202-b1c4-1afbe0825333 service neutron] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 17:13:59.676738 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6adcbec0-dc29-4202-b1c4-1afbe0825333 service neutron] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 17:13:59.677009 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6adcbec0-dc29-4202-b1c4-1afbe0825333 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 17:13:59.685940 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6adcbec0-dc29-4202-b1c4-1afbe0825333 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 17:13:59.687392 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6adcbec0-dc29-4202-b1c4-1afbe0825333 service neutron] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 17:13:59.697341 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 1452/2897] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 1 17:13:59 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 17:14:00.777809 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f2f7dd-b8ab-4912-a6f1-9ac6c240529c None None] REQUEST_METHOD: `POST` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 17:14:00.778399 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f2f7dd-b8ab-4912-a6f1-9ac6c240529c None None] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 17:14:00.778522 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f2f7dd-b8ab-4912-a6f1-9ac6c240529c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 17:14:00.800826 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG dbcounter [-] [87057] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87057) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 17:14:00.809631 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.auth.core [None req-a7f2f7dd-b8ab-4912-a6f1-9ac6c240529c None None] MFA Rules not processed for user `24a9a164af284f04a1f3580bc7f133e5`. Rule list: `[]` (Enabled: `True`). {{(pid=87057) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 17:14:00.810882 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.fernet_utils [None req-a7f2f7dd-b8ab-4912-a6f1-9ac6c240529c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87057) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 17:14:00.882160 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 1446/2898] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 1 17:14:00 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 17:14:00.890470 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6adcbec0-dc29-4202-b1c4-1afbe0825333 service neutron] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 17:14:00.892028 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.fernet_utils [None req-6adcbec0-dc29-4202-b1c4-1afbe0825333 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 17:14:00.955267 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6adcbec0-dc29-4202-b1c4-1afbe0825333 service neutron] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 17:14:00.960033 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48db2901-3a6b-4329-a280-e62c2808308a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25970b52ca15448fa1abb1f974d64038', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b992d6a65df4bf9b27c13192d57c5f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 17:14:00.960665 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48db2901-3a6b-4329-a280-e62c2808308a service cinder] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 17:14:00.960948 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48db2901-3a6b-4329-a280-e62c2808308a service cinder] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 17:14:00.961225 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48db2901-3a6b-4329-a280-e62c2808308a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 17:14:00.969890 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48db2901-3a6b-4329-a280-e62c2808308a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 17:14:00.971371 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48db2901-3a6b-4329-a280-e62c2808308a service cinder] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 17:14:00.980724 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 1453/2899] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 1 17:14:00 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 17:14:02.736866 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca5109d-578a-48d4-bc82-9a8bacc25b8b None None] REQUEST_METHOD: `POST` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 17:14:02.737514 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca5109d-578a-48d4-bc82-9a8bacc25b8b None None] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 17:14:02.737514 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca5109d-578a-48d4-bc82-9a8bacc25b8b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 17:14:02.756867 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.auth.core [None req-dca5109d-578a-48d4-bc82-9a8bacc25b8b None None] MFA Rules not processed for user `24a9a164af284f04a1f3580bc7f133e5`. Rule list: `[]` (Enabled: `True`). {{(pid=87057) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 17:14:02.758102 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.fernet_utils [None req-dca5109d-578a-48d4-bc82-9a8bacc25b8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87057) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 17:14:02.765036 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 1447/2900] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 1 17:14:02 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 17:14:03.002687 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48db2901-3a6b-4329-a280-e62c2808308a service cinder] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 17:14:03.004122 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.fernet_utils [None req-48db2901-3a6b-4329-a280-e62c2808308a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 17:14:03.016506 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48db2901-3a6b-4329-a280-e62c2808308a service cinder] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 17:14:03.020068 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48824443-1c17-40b7-bce5-b9feb96da27a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d9bd8d649154c979be3744bfd016f95', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b992d6a65df4bf9b27c13192d57c5f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 17:14:03.020665 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48824443-1c17-40b7-bce5-b9feb96da27a service neutron] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 17:14:03.021479 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48824443-1c17-40b7-bce5-b9feb96da27a service neutron] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 17:14:03.021752 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48824443-1c17-40b7-bce5-b9feb96da27a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 17:14:03.023477 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.fernet_utils [None req-48824443-1c17-40b7-bce5-b9feb96da27a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 17:14:03.032743 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48824443-1c17-40b7-bce5-b9feb96da27a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 17:14:03.034188 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48824443-1c17-40b7-bce5-b9feb96da27a service neutron] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 17:14:03.042318 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 1454/2901] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 1 17:14:03 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 17:14:12.751829 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG dbcounter [-] [87057] Writing DB stats keystone:SELECT=30 {{(pid=87057) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 17:14:13.038727 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG dbcounter [-] [87058] Writing DB stats keystone:SELECT=68,keystone:INSERT=1 {{(pid=87058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 17:14:20.854443 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ce77e1-ab3b-41aa-a125-203a7521968a None None] REQUEST_METHOD: `POST` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 17:14:20.854928 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ce77e1-ab3b-41aa-a125-203a7521968a None None] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 17:14:20.855037 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ce77e1-ab3b-41aa-a125-203a7521968a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 17:14:20.874837 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.auth.core [None req-75ce77e1-ab3b-41aa-a125-203a7521968a None None] MFA Rules not processed for user `24a9a164af284f04a1f3580bc7f133e5`. Rule list: `[]` (Enabled: `True`). {{(pid=87057) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 17:14:20.876064 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.fernet_utils [None req-75ce77e1-ab3b-41aa-a125-203a7521968a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87057) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 17:14:20.883153 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 1448/2902] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 1 17:14:20 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 17:14:30.870021 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG dbcounter [-] [87057] Writing DB stats keystone:SELECT=3 {{(pid=87057) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 17:14:35.979335 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48824443-1c17-40b7-bce5-b9feb96da27a service neutron] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 17:14:35.988919 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48824443-1c17-40b7-bce5-b9feb96da27a service neutron] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 17:14:35.992448 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c843fe-37b0-49c6-8fd3-d87b200c515f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25970b52ca15448fa1abb1f974d64038', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b992d6a65df4bf9b27c13192d57c5f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 17:14:35.993637 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c843fe-37b0-49c6-8fd3-d87b200c515f service cinder] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 17:14:35.993915 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c843fe-37b0-49c6-8fd3-d87b200c515f service cinder] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 17:14:35.994217 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c843fe-37b0-49c6-8fd3-d87b200c515f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 17:14:35.995958 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.fernet_utils [None req-11c843fe-37b0-49c6-8fd3-d87b200c515f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 17:14:36.053756 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11c843fe-37b0-49c6-8fd3-d87b200c515f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 17:14:36.055723 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11c843fe-37b0-49c6-8fd3-d87b200c515f service cinder] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 17:14:36.064603 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 1455/2903] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 1 17:14:35 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 17:14:46.060542 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG dbcounter [-] [87058] Writing DB stats keystone:SELECT=28 {{(pid=87058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 17:14:48.693554 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c76977e-f764-40a9-b320-daffe52f7e74 None None] REQUEST_METHOD: `POST` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 17:14:48.694124 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c76977e-f764-40a9-b320-daffe52f7e74 None None] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 17:14:48.694244 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c76977e-f764-40a9-b320-daffe52f7e74 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 17:14:48.714112 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.auth.core [None req-5c76977e-f764-40a9-b320-daffe52f7e74 None None] MFA Rules not processed for user `8ca353ec7cb948c9a803e6422ea3476e`. Rule list: `[]` (Enabled: `True`). {{(pid=87057) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 17:14:48.715315 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.fernet_utils [None req-5c76977e-f764-40a9-b320-daffe52f7e74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87057) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 17:14:48.722291 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 1449/2904] 10.222.0.20 () {58 vars in 958 bytes} [Mon Aug 1 17:14:48 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 17:14:53.033571 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44d9dc7-bd91-49d9-9637-a0d0fe5acf86 None None] REQUEST_METHOD: `POST` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 17:14:53.034277 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44d9dc7-bd91-49d9-9637-a0d0fe5acf86 None None] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 17:14:53.034277 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44d9dc7-bd91-49d9-9637-a0d0fe5acf86 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 17:14:53.079780 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.auth.core [None req-a44d9dc7-bd91-49d9-9637-a0d0fe5acf86 None None] MFA Rules not processed for user `b6a93d1c6c82489099adc8b413280fc0`. Rule list: `[]` (Enabled: `True`). {{(pid=87058) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 17:14:53.081142 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.fernet_utils [None req-a44d9dc7-bd91-49d9-9637-a0d0fe5acf86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 17:14:53.182548 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 1456/2905] 10.222.0.20 () {58 vars in 1009 bytes} [Mon Aug 1 17:14:53 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 17:14:53.327803 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c76977e-f764-40a9-b320-daffe52f7e74 None None] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 17:14:53.329327 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.fernet_utils [None req-5c76977e-f764-40a9-b320-daffe52f7e74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87057) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 17:14:53.418133 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c76977e-f764-40a9-b320-daffe52f7e74 None None] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 17:14:53.423418 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b9bf1b-8b9c-41df-8f99-fd1c47a11302 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ab4add1faa04faa811ef7a9341cc35c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b992d6a65df4bf9b27c13192d57c5f0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 17:14:53.424052 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b9bf1b-8b9c-41df-8f99-fd1c47a11302 service swift] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 17:14:53.424334 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b9bf1b-8b9c-41df-8f99-fd1c47a11302 service swift] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 17:14:53.424644 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b9bf1b-8b9c-41df-8f99-fd1c47a11302 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 17:14:53.435569 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86b9bf1b-8b9c-41df-8f99-fd1c47a11302 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 17:14:53.437201 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86b9bf1b-8b9c-41df-8f99-fd1c47a11302 service swift] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 17:14:53.446133 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 1450/2906] 10.222.0.20 () {60 vars in 1384 bytes} [Mon Aug 1 17:14:53 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Aug 01 17:15:03.174436 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG dbcounter [-] [87058] Writing DB stats keystone:SELECT=32 {{(pid=87058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 17:15:03.444030 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG dbcounter [-] [87057] Writing DB stats keystone:SELECT=40 {{(pid=87057) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 17:15:09.045491 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a44d9dc7-bd91-49d9-9637-a0d0fe5acf86 None None] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 17:15:09.055301 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a44d9dc7-bd91-49d9-9637-a0d0fe5acf86 None None] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 17:15:09.059737 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac49ef9-cfe4-458f-b05b-ec6fcbd661a3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d9bd8d649154c979be3744bfd016f95', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b992d6a65df4bf9b27c13192d57c5f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 17:15:09.060418 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac49ef9-cfe4-458f-b05b-ec6fcbd661a3 service neutron] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 17:15:09.060648 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac49ef9-cfe4-458f-b05b-ec6fcbd661a3 service neutron] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 17:15:09.061034 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac49ef9-cfe4-458f-b05b-ec6fcbd661a3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 17:15:09.062938 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.fernet_utils [None req-2ac49ef9-cfe4-458f-b05b-ec6fcbd661a3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 17:15:09.128589 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ac49ef9-cfe4-458f-b05b-ec6fcbd661a3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 17:15:09.130059 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ac49ef9-cfe4-458f-b05b-ec6fcbd661a3 service neutron] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 17:15:09.139932 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 1457/2907] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 1 17:15:09 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 17:15:10.182467 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13338491-854d-4bcf-846d-5b19cfa3b09e None None] REQUEST_METHOD: `POST` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 17:15:10.182963 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13338491-854d-4bcf-846d-5b19cfa3b09e None None] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 17:15:10.183083 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13338491-854d-4bcf-846d-5b19cfa3b09e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 17:15:10.203864 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.auth.core [None req-13338491-854d-4bcf-846d-5b19cfa3b09e None None] MFA Rules not processed for user `b6a93d1c6c82489099adc8b413280fc0`. Rule list: `[]` (Enabled: `True`). {{(pid=87057) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 17:15:10.205090 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.fernet_utils [None req-13338491-854d-4bcf-846d-5b19cfa3b09e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87057) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 17:15:10.212506 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 1451/2908] 10.222.0.20 () {58 vars in 967 bytes} [Mon Aug 1 17:15:10 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 17:15:10.224688 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac49ef9-cfe4-458f-b05b-ec6fcbd661a3 service neutron] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 17:15:10.237246 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac49ef9-cfe4-458f-b05b-ec6fcbd661a3 service neutron] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 17:15:10.248903 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4536ca15-4234-484a-9654-6817c5c3eefa service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ab4add1faa04faa811ef7a9341cc35c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b992d6a65df4bf9b27c13192d57c5f0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 17:15:10.249537 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4536ca15-4234-484a-9654-6817c5c3eefa service swift] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 17:15:10.249818 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4536ca15-4234-484a-9654-6817c5c3eefa service swift] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 17:15:10.250144 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4536ca15-4234-484a-9654-6817c5c3eefa service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 17:15:10.259587 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4536ca15-4234-484a-9654-6817c5c3eefa service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 17:15:10.261647 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4536ca15-4234-484a-9654-6817c5c3eefa service swift] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 17:15:10.271120 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 1458/2909] 10.222.0.20 () {60 vars in 1384 bytes} [Mon Aug 1 17:15:10 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 01 17:15:13.094525 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13338491-854d-4bcf-846d-5b19cfa3b09e None None] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 17:15:13.107704 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13338491-854d-4bcf-846d-5b19cfa3b09e None None] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 17:15:13.112538 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c61b12bd-ebcd-4111-aff6-c0b5a16b7ae1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 17:15:13.113200 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61b12bd-ebcd-4111-aff6-c0b5a16b7ae1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 17:15:13.113473 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61b12bd-ebcd-4111-aff6-c0b5a16b7ae1 admin admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 17:15:13.113744 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61b12bd-ebcd-4111-aff6-c0b5a16b7ae1 admin admin] PATH_INFO: `/v3/users/24a9a164af284f04a1f3580bc7f133e5` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 17:15:13.114934 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c61b12bd-ebcd-4111-aff6-c0b5a16b7ae1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=24a9a164af284f04a1f3580bc7f133e5)` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 17:15:13.116946 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c61b12bd-ebcd-4111-aff6-c0b5a16b7ae1 admin admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 17:15:13.233453 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.notifications [None req-c61b12bd-ebcd-4111-aff6-c0b5a16b7ae1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '24a9a164af284f04a1f3580bc7f133e5'} {{(pid=87057) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 17:15:13.234142 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.notifications [None req-c61b12bd-ebcd-4111-aff6-c0b5a16b7ae1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '24a9a164af284f04a1f3580bc7f133e5'} {{(pid=87057) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 17:15:13.293771 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.notifications [None req-c61b12bd-ebcd-4111-aff6-c0b5a16b7ae1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '24a9a164af284f04a1f3580bc7f133e5'} {{(pid=87057) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 17:15:13.294557 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.notifications [None req-c61b12bd-ebcd-4111-aff6-c0b5a16b7ae1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '24a9a164af284f04a1f3580bc7f133e5'} {{(pid=87057) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 17:15:13.301921 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.notifications [None req-c61b12bd-ebcd-4111-aff6-c0b5a16b7ae1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '24a9a164af284f04a1f3580bc7f133e5'} {{(pid=87057) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 17:15:13.319863 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 1452/2910] 10.222.0.20 () {58 vars in 1224 bytes} [Mon Aug 1 17:15:13 2022] DELETE /identity/v3/users/24a9a164af284f04a1f3580bc7f133e5 => generated 0 bytes in 226 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 17:15:13.564343 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4536ca15-4234-484a-9654-6817c5c3eefa service swift] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 17:15:13.575245 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4536ca15-4234-484a-9654-6817c5c3eefa service swift] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 17:15:13.632923 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc9a1c68-b712-4c9c-99b4-655ca6190a7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 17:15:13.633534 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9a1c68-b712-4c9c-99b4-655ca6190a7e admin admin] REQUEST_METHOD: `DELETE` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 17:15:13.633772 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9a1c68-b712-4c9c-99b4-655ca6190a7e admin admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 17:15:13.634075 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9a1c68-b712-4c9c-99b4-655ca6190a7e admin admin] PATH_INFO: `/v3/projects/373368f3276a4b75bdc37f4510ea6b8b` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 17:15:13.635232 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc9a1c68-b712-4c9c-99b4-655ca6190a7e admin admin] RBAC: Authorizing `identity:delete_project(project_id=373368f3276a4b75bdc37f4510ea6b8b)` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 17:15:13.637142 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc9a1c68-b712-4c9c-99b4-655ca6190a7e admin admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 17:15:13.746135 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.notifications [None req-dc9a1c68-b712-4c9c-99b4-655ca6190a7e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '373368f3276a4b75bdc37f4510ea6b8b', 'request_id': 'req-dc9a1c68-b712-4c9c-99b4-655ca6190a7e'} {{(pid=87058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 17:15:13.752292 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.notifications [None req-dc9a1c68-b712-4c9c-99b4-655ca6190a7e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '373368f3276a4b75bdc37f4510ea6b8b', 'request_id': 'req-dc9a1c68-b712-4c9c-99b4-655ca6190a7e'} {{(pid=87058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 17:15:13.757761 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.notifications [None req-dc9a1c68-b712-4c9c-99b4-655ca6190a7e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '373368f3276a4b75bdc37f4510ea6b8b', 'request_id': 'req-dc9a1c68-b712-4c9c-99b4-655ca6190a7e'} {{(pid=87058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 17:15:13.829949 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.notifications [None req-dc9a1c68-b712-4c9c-99b4-655ca6190a7e admin admin] The token cache is being invalidate because project 373368f3276a4b75bdc37f4510ea6b8b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87058) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 17:15:13.830273 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.notifications [None req-dc9a1c68-b712-4c9c-99b4-655ca6190a7e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 17:15:13.830937 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.notifications [None req-dc9a1c68-b712-4c9c-99b4-655ca6190a7e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 17:15:13.832518 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 1459/2911] 10.222.0.20 () {58 vars in 1233 bytes} [Mon Aug 1 17:15:13 2022] DELETE /identity/v3/projects/373368f3276a4b75bdc37f4510ea6b8b => generated 0 bytes in 269 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 17:15:23.316304 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG dbcounter [-] [87057] Writing DB stats keystone:SELECT=32,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87057) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 17:15:23.826735 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG dbcounter [-] [87058] Writing DB stats keystone:SELECT=87,keystone:DELETE=7,keystone:INSERT=1 {{(pid=87058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 17:16:24.438878 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c61b12bd-ebcd-4111-aff6-c0b5a16b7ae1 admin admin] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 17:16:24.440386 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.fernet_utils [None req-c61b12bd-ebcd-4111-aff6-c0b5a16b7ae1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87057) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 17:16:24.498156 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c61b12bd-ebcd-4111-aff6-c0b5a16b7ae1 admin admin] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 17:16:24.545572 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b1cab6-dae5-46ee-a4bf-0a4c6c7dbdd0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d9bd8d649154c979be3744bfd016f95', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b992d6a65df4bf9b27c13192d57c5f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 17:16:24.546226 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b1cab6-dae5-46ee-a4bf-0a4c6c7dbdd0 service neutron] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 17:16:24.546511 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b1cab6-dae5-46ee-a4bf-0a4c6c7dbdd0 service neutron] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 17:16:24.546788 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b1cab6-dae5-46ee-a4bf-0a4c6c7dbdd0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 17:16:24.548553 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.fernet_utils [None req-59b1cab6-dae5-46ee-a4bf-0a4c6c7dbdd0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87057) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 17:16:24.557888 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59b1cab6-dae5-46ee-a4bf-0a4c6c7dbdd0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 17:16:24.559405 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59b1cab6-dae5-46ee-a4bf-0a4c6c7dbdd0 service neutron] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 17:16:24.567431 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 1453/2912] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 1 17:16:24 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 17:16:34.563797 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG dbcounter [-] [87057] Writing DB stats keystone:SELECT=46 {{(pid=87057) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 17:17:13.382596 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc9a1c68-b712-4c9c-99b4-655ca6190a7e admin admin] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 17:17:13.384203 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.fernet_utils [None req-dc9a1c68-b712-4c9c-99b4-655ca6190a7e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 17:17:13.493483 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc9a1c68-b712-4c9c-99b4-655ca6190a7e admin admin] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 17:17:13.499987 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d7bc55e-b6f3-4fc1-bd0f-803e7def16f1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6533106d03eb4ca185a8839336d2aff9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b992d6a65df4bf9b27c13192d57c5f0', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 17:17:13.500654 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7bc55e-b6f3-4fc1-bd0f-803e7def16f1 service nova] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 17:17:13.500929 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7bc55e-b6f3-4fc1-bd0f-803e7def16f1 service nova] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 17:17:13.501332 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7bc55e-b6f3-4fc1-bd0f-803e7def16f1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 17:17:13.503132 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.fernet_utils [None req-4d7bc55e-b6f3-4fc1-bd0f-803e7def16f1 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 17:17:13.516747 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d7bc55e-b6f3-4fc1-bd0f-803e7def16f1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 17:17:13.518360 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d7bc55e-b6f3-4fc1-bd0f-803e7def16f1 service nova] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 17:17:13.529669 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 1460/2913] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 1 17:17:13 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 17:17:13.787732 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b1cab6-dae5-46ee-a4bf-0a4c6c7dbdd0 service neutron] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 17:17:13.789148 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.fernet_utils [None req-59b1cab6-dae5-46ee-a4bf-0a4c6c7dbdd0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87057) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 17:17:13.800121 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b1cab6-dae5-46ee-a4bf-0a4c6c7dbdd0 service neutron] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 17:17:13.804034 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c25744-2fd5-47f6-b2f5-fc50da66ee97 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d9bd8d649154c979be3744bfd016f95', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b992d6a65df4bf9b27c13192d57c5f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 17:17:13.804654 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c25744-2fd5-47f6-b2f5-fc50da66ee97 service neutron] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 17:17:13.804933 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c25744-2fd5-47f6-b2f5-fc50da66ee97 service neutron] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 17:17:13.805205 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c25744-2fd5-47f6-b2f5-fc50da66ee97 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 17:17:13.806980 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.fernet_utils [None req-84c25744-2fd5-47f6-b2f5-fc50da66ee97 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87057) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 17:17:13.821389 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84c25744-2fd5-47f6-b2f5-fc50da66ee97 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 17:17:13.822862 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84c25744-2fd5-47f6-b2f5-fc50da66ee97 service neutron] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 17:17:13.834203 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 1454/2914] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 1 17:17:13 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 17:17:15.018454 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d7bc55e-b6f3-4fc1-bd0f-803e7def16f1 service nova] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 17:17:15.019918 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.fernet_utils [None req-4d7bc55e-b6f3-4fc1-bd0f-803e7def16f1 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 17:17:15.032972 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d7bc55e-b6f3-4fc1-bd0f-803e7def16f1 service nova] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 17:17:15.036983 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a75ab3fe-13af-4c39-aae5-50451fe5f1fd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d9bd8d649154c979be3744bfd016f95', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b992d6a65df4bf9b27c13192d57c5f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 17:17:15.037609 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75ab3fe-13af-4c39-aae5-50451fe5f1fd service neutron] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 17:17:15.037891 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75ab3fe-13af-4c39-aae5-50451fe5f1fd service neutron] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 17:17:15.038196 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75ab3fe-13af-4c39-aae5-50451fe5f1fd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 17:17:15.039984 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.fernet_utils [None req-a75ab3fe-13af-4c39-aae5-50451fe5f1fd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 17:17:15.056317 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a75ab3fe-13af-4c39-aae5-50451fe5f1fd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 17:17:15.057766 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a75ab3fe-13af-4c39-aae5-50451fe5f1fd service neutron] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 17:17:15.069502 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 1461/2915] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 1 17:17:15 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 17:17:16.239319 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c25744-2fd5-47f6-b2f5-fc50da66ee97 service neutron] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 17:17:16.240766 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.fernet_utils [None req-84c25744-2fd5-47f6-b2f5-fc50da66ee97 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87057) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 17:17:16.255857 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c25744-2fd5-47f6-b2f5-fc50da66ee97 service neutron] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 17:17:16.275704 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0709cf24-5baa-4d8c-9513-1f10249242f1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6533106d03eb4ca185a8839336d2aff9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b992d6a65df4bf9b27c13192d57c5f0', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 17:17:16.276349 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0709cf24-5baa-4d8c-9513-1f10249242f1 service nova] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 17:17:16.276625 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0709cf24-5baa-4d8c-9513-1f10249242f1 service nova] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 17:17:16.276894 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0709cf24-5baa-4d8c-9513-1f10249242f1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 17:17:16.278840 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.fernet_utils [None req-0709cf24-5baa-4d8c-9513-1f10249242f1 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87057) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 17:17:16.299352 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0709cf24-5baa-4d8c-9513-1f10249242f1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 17:17:16.301071 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0709cf24-5baa-4d8c-9513-1f10249242f1 service nova] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 17:17:16.314414 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 1455/2916] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 1 17:17:16 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 76 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 17:17:25.064377 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG dbcounter [-] [87058] Writing DB stats keystone:SELECT=49 {{(pid=87058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 17:17:26.308692 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG dbcounter [-] [87057] Writing DB stats keystone:SELECT=12 {{(pid=87057) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 17:17:27.775714 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a75ab3fe-13af-4c39-aae5-50451fe5f1fd service neutron] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 17:17:27.785614 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a75ab3fe-13af-4c39-aae5-50451fe5f1fd service neutron] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 17:17:27.789323 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14931bff-b709-4418-9777-ccef83372543 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d9bd8d649154c979be3744bfd016f95', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b992d6a65df4bf9b27c13192d57c5f0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 17:17:27.789928 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14931bff-b709-4418-9777-ccef83372543 service neutron] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 17:17:27.790248 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14931bff-b709-4418-9777-ccef83372543 service neutron] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 17:17:27.790522 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14931bff-b709-4418-9777-ccef83372543 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 17:17:27.792249 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.fernet_utils [None req-14931bff-b709-4418-9777-ccef83372543 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 17:17:27.854773 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14931bff-b709-4418-9777-ccef83372543 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 17:17:27.856188 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14931bff-b709-4418-9777-ccef83372543 service neutron] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 17:17:27.866497 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 1462/2917] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 1 17:17:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 01 17:17:31.319808 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0709cf24-5baa-4d8c-9513-1f10249242f1 service nova] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 17:17:31.326629 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.fernet_utils [None req-0709cf24-5baa-4d8c-9513-1f10249242f1 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87057) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 17:17:31.390705 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0709cf24-5baa-4d8c-9513-1f10249242f1 service nova] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 17:17:31.395787 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e1d0b0-70b0-4879-b139-dddf2e6e2424 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8362293261614623bcdabc4120a97a9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b992d6a65df4bf9b27c13192d57c5f0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 17:17:31.396408 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e1d0b0-70b0-4879-b139-dddf2e6e2424 service placement] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 17:17:31.396688 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e1d0b0-70b0-4879-b139-dddf2e6e2424 service placement] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 17:17:31.396965 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e1d0b0-70b0-4879-b139-dddf2e6e2424 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 17:17:31.398736 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.fernet_utils [None req-46e1d0b0-70b0-4879-b139-dddf2e6e2424 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87057) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 17:17:31.410351 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46e1d0b0-70b0-4879-b139-dddf2e6e2424 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 17:17:31.411922 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46e1d0b0-70b0-4879-b139-dddf2e6e2424 service placement] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 17:17:31.422640 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 1456/2918] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 1 17:17:31 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 17:17:34.737524 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14931bff-b709-4418-9777-ccef83372543 service neutron] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 17:17:34.739035 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.fernet_utils [None req-14931bff-b709-4418-9777-ccef83372543 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 17:17:34.751468 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14931bff-b709-4418-9777-ccef83372543 service neutron] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 17:17:34.756229 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-839bdc2b-e21b-423e-9733-172618336dff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 17:17:34.756872 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839bdc2b-e21b-423e-9733-172618336dff admin admin] REQUEST_METHOD: `DELETE` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 17:17:34.757170 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839bdc2b-e21b-423e-9733-172618336dff admin admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 17:17:34.757431 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839bdc2b-e21b-423e-9733-172618336dff admin admin] PATH_INFO: `/v3/users/8ca353ec7cb948c9a803e6422ea3476e` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 17:17:34.758648 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-839bdc2b-e21b-423e-9733-172618336dff admin admin] RBAC: Authorizing `identity:delete_user(user_id=8ca353ec7cb948c9a803e6422ea3476e)` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 17:17:34.760583 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-839bdc2b-e21b-423e-9733-172618336dff admin admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 17:17:34.921831 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.notifications [None req-839bdc2b-e21b-423e-9733-172618336dff admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8ca353ec7cb948c9a803e6422ea3476e'} {{(pid=87058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 17:17:34.922477 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.notifications [None req-839bdc2b-e21b-423e-9733-172618336dff admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8ca353ec7cb948c9a803e6422ea3476e'} {{(pid=87058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 17:17:34.923149 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.notifications [None req-839bdc2b-e21b-423e-9733-172618336dff admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8ca353ec7cb948c9a803e6422ea3476e'} {{(pid=87058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 17:17:34.935040 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.notifications [None req-839bdc2b-e21b-423e-9733-172618336dff admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8ca353ec7cb948c9a803e6422ea3476e'} {{(pid=87058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 17:17:34.942005 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.notifications [None req-839bdc2b-e21b-423e-9733-172618336dff admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8ca353ec7cb948c9a803e6422ea3476e'} {{(pid=87058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 17:17:35.044422 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 1463/2919] 10.222.0.20 () {58 vars in 1224 bytes} [Mon Aug 1 17:17:34 2022] DELETE /identity/v3/users/8ca353ec7cb948c9a803e6422ea3476e => generated 0 bytes in 307 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 17:17:35.252625 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e1d0b0-70b0-4879-b139-dddf2e6e2424 service placement] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 17:17:35.264462 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e1d0b0-70b0-4879-b139-dddf2e6e2424 service placement] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 17:17:35.324151 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-792d875e-984f-4009-b8bf-b4cd99c51e20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 17:17:35.324889 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792d875e-984f-4009-b8bf-b4cd99c51e20 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 17:17:35.325171 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792d875e-984f-4009-b8bf-b4cd99c51e20 admin admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 17:17:35.325447 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792d875e-984f-4009-b8bf-b4cd99c51e20 admin admin] PATH_INFO: `/v3/projects/c2cff009abd049a7aff67696680687ca` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 17:17:35.326649 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-792d875e-984f-4009-b8bf-b4cd99c51e20 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c2cff009abd049a7aff67696680687ca)` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 17:17:35.329234 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-792d875e-984f-4009-b8bf-b4cd99c51e20 admin admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 17:17:35.456060 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.notifications [None req-792d875e-984f-4009-b8bf-b4cd99c51e20 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c2cff009abd049a7aff67696680687ca', 'request_id': 'req-792d875e-984f-4009-b8bf-b4cd99c51e20'} {{(pid=87057) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 17:17:35.462351 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.notifications [None req-792d875e-984f-4009-b8bf-b4cd99c51e20 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c2cff009abd049a7aff67696680687ca', 'request_id': 'req-792d875e-984f-4009-b8bf-b4cd99c51e20'} {{(pid=87057) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 17:17:35.468018 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.notifications [None req-792d875e-984f-4009-b8bf-b4cd99c51e20 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c2cff009abd049a7aff67696680687ca', 'request_id': 'req-792d875e-984f-4009-b8bf-b4cd99c51e20'} {{(pid=87057) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 17:17:35.535233 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.notifications [None req-792d875e-984f-4009-b8bf-b4cd99c51e20 admin admin] The token cache is being invalidate because project c2cff009abd049a7aff67696680687ca was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87057) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 17:17:35.535521 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.notifications [None req-792d875e-984f-4009-b8bf-b4cd99c51e20 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87057) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 17:17:35.536135 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.notifications [None req-792d875e-984f-4009-b8bf-b4cd99c51e20 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87057) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 17:17:35.537680 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 1457/2920] 10.222.0.20 () {58 vars in 1233 bytes} [Mon Aug 1 17:17:35 2022] DELETE /identity/v3/projects/c2cff009abd049a7aff67696680687ca => generated 0 bytes in 286 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 17:17:35.541827 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-839bdc2b-e21b-423e-9733-172618336dff admin admin] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 17:17:35.543387 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.fernet_utils [None req-839bdc2b-e21b-423e-9733-172618336dff admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 17:17:35.557196 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-839bdc2b-e21b-423e-9733-172618336dff admin admin] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 17:17:35.613366 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f26c9f7a-d523-4421-b175-e3a084fa4574 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 17:17:35.614007 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26c9f7a-d523-4421-b175-e3a084fa4574 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 17:17:35.614319 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26c9f7a-d523-4421-b175-e3a084fa4574 admin admin] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 17:17:35.614589 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26c9f7a-d523-4421-b175-e3a084fa4574 admin admin] PATH_INFO: `/v3/users/916b40e076f34ef090e6c68882f25657` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 17:17:35.615759 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f26c9f7a-d523-4421-b175-e3a084fa4574 admin admin] RBAC: Authorizing `identity:delete_user(user_id=916b40e076f34ef090e6c68882f25657)` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 17:17:35.617725 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f26c9f7a-d523-4421-b175-e3a084fa4574 admin admin] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 17:17:35.623266 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG dbcounter [-] [87058] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 17:17:35.800113 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.notifications [None req-f26c9f7a-d523-4421-b175-e3a084fa4574 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '916b40e076f34ef090e6c68882f25657'} {{(pid=87058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 17:17:35.800751 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.notifications [None req-f26c9f7a-d523-4421-b175-e3a084fa4574 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '916b40e076f34ef090e6c68882f25657'} {{(pid=87058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 17:17:35.801261 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.notifications [None req-f26c9f7a-d523-4421-b175-e3a084fa4574 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '916b40e076f34ef090e6c68882f25657'} {{(pid=87058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 17:17:35.814663 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.notifications [None req-f26c9f7a-d523-4421-b175-e3a084fa4574 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '916b40e076f34ef090e6c68882f25657'} {{(pid=87058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 17:17:35.822212 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.notifications [None req-f26c9f7a-d523-4421-b175-e3a084fa4574 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '916b40e076f34ef090e6c68882f25657'} {{(pid=87058) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 17:17:35.884443 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 1464/2921] 10.222.0.20 () {58 vars in 1224 bytes} [Mon Aug 1 17:17:35 2022] DELETE /identity/v3/users/916b40e076f34ef090e6c68882f25657 => generated 0 bytes in 343 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 17:17:36.165874 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-792d875e-984f-4009-b8bf-b4cd99c51e20 admin admin] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 17:17:36.167348 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.fernet_utils [None req-792d875e-984f-4009-b8bf-b4cd99c51e20 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87057) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 17:17:36.181427 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-792d875e-984f-4009-b8bf-b4cd99c51e20 admin admin] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 17:17:36.198119 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG dbcounter [-] [87057] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=87057) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 17:17:36.239301 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb03060-caa9-4bf3-a641-629a5e3e4397 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1007f8e379b74e169b98fb7c3c0ba5b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e43f22f36744d079b60704aff4969d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 17:17:36.239947 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb03060-caa9-4bf3-a641-629a5e3e4397 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 17:17:36.240229 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb03060-caa9-4bf3-a641-629a5e3e4397 admin admin] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 17:17:36.240505 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb03060-caa9-4bf3-a641-629a5e3e4397 admin admin] PATH_INFO: `/v3/projects/e82a966c39d74959b691be5aac26f0dd` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 17:17:36.241656 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eb03060-caa9-4bf3-a641-629a5e3e4397 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e82a966c39d74959b691be5aac26f0dd)` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 17:17:36.243605 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eb03060-caa9-4bf3-a641-629a5e3e4397 admin admin] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 17:17:36.348130 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.notifications [None req-4eb03060-caa9-4bf3-a641-629a5e3e4397 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e82a966c39d74959b691be5aac26f0dd', 'request_id': 'req-4eb03060-caa9-4bf3-a641-629a5e3e4397'} {{(pid=87057) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 17:17:36.354664 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.notifications [None req-4eb03060-caa9-4bf3-a641-629a5e3e4397 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e82a966c39d74959b691be5aac26f0dd', 'request_id': 'req-4eb03060-caa9-4bf3-a641-629a5e3e4397'} {{(pid=87057) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 17:17:36.360841 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.notifications [None req-4eb03060-caa9-4bf3-a641-629a5e3e4397 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e82a966c39d74959b691be5aac26f0dd', 'request_id': 'req-4eb03060-caa9-4bf3-a641-629a5e3e4397'} {{(pid=87057) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 17:17:36.415104 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.notifications [None req-4eb03060-caa9-4bf3-a641-629a5e3e4397 admin admin] The token cache is being invalidate because project e82a966c39d74959b691be5aac26f0dd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87057) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 17:17:36.415326 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.notifications [None req-4eb03060-caa9-4bf3-a641-629a5e3e4397 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87057) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 17:17:36.416014 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.notifications [None req-4eb03060-caa9-4bf3-a641-629a5e3e4397 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87057) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 17:17:36.417653 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 1458/2922] 10.222.0.20 () {58 vars in 1233 bytes} [Mon Aug 1 17:17:36 2022] DELETE /identity/v3/projects/e82a966c39d74959b691be5aac26f0dd => generated 0 bytes in 252 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 17:17:45.880689 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG dbcounter [-] [87058] Writing DB stats keystone:SELECT=23,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 17:17:46.411887 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG dbcounter [-] [87057] Writing DB stats keystone:SELECT=42,keystone:DELETE=7,keystone:INSERT=1 {{(pid=87057) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 17:18:19.674790 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f26c9f7a-d523-4421-b175-e3a084fa4574 admin admin] Authenticating user token {{(pid=87058) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 17:18:19.676198 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.fernet_utils [None req-f26c9f7a-d523-4421-b175-e3a084fa4574 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 17:18:19.744834 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f26c9f7a-d523-4421-b175-e3a084fa4574 admin admin] Validating token access rules against request {{(pid=87058) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 17:18:19.803067 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-468b82de-e258-4c40-918c-330039371b70 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8362293261614623bcdabc4120a97a9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b992d6a65df4bf9b27c13192d57c5f0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87058) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 17:18:19.803684 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468b82de-e258-4c40-918c-330039371b70 service placement] REQUEST_METHOD: `GET` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 17:18:19.803962 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468b82de-e258-4c40-918c-330039371b70 service placement] SCRIPT_NAME: `/identity` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 17:18:19.804231 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468b82de-e258-4c40-918c-330039371b70 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=87058) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 17:18:19.805977 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.fernet_utils [None req-468b82de-e258-4c40-918c-330039371b70 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87058) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 17:18:19.817664 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-468b82de-e258-4c40-918c-330039371b70 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 17:18:19.819328 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-468b82de-e258-4c40-918c-330039371b70 service placement] RBAC: Authorization granted {{(pid=87058) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 17:18:19.829619 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: [pid: 87058|app: 0|req: 1465/2923] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 1 17:18:19 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 155 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 17:18:29.824830 dv-cinder-849890-2-iscsi devstack@keystone.service[87058]: DEBUG dbcounter [-] [87058] Writing DB stats keystone:SELECT=58 {{(pid=87058) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 17:23:25.659341 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb03060-caa9-4bf3-a641-629a5e3e4397 admin admin] Authenticating user token {{(pid=87057) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 17:23:25.660745 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.fernet_utils [None req-4eb03060-caa9-4bf3-a641-629a5e3e4397 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87057) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 17:23:25.732298 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb03060-caa9-4bf3-a641-629a5e3e4397 admin admin] Validating token access rules against request {{(pid=87057) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 17:23:25.737608 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e7757f-2ba5-46b9-90a0-714ddad0989d service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8362293261614623bcdabc4120a97a9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b992d6a65df4bf9b27c13192d57c5f0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87057) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 17:23:25.738299 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e7757f-2ba5-46b9-90a0-714ddad0989d service placement] REQUEST_METHOD: `GET` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 17:23:25.738583 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e7757f-2ba5-46b9-90a0-714ddad0989d service placement] SCRIPT_NAME: `/identity` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 17:23:25.738863 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e7757f-2ba5-46b9-90a0-714ddad0989d service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=87057) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 17:23:25.748746 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0e7757f-2ba5-46b9-90a0-714ddad0989d service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 17:23:25.750375 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0e7757f-2ba5-46b9-90a0-714ddad0989d service placement] RBAC: Authorization granted {{(pid=87057) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 17:23:25.760507 dv-cinder-849890-2-iscsi devstack@keystone.service[87057]: [pid: 87057|app: 0|req: 1459/2924] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Aug 1 17:23:25 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0)